arxiv.org/abs/2608.08730 Dispatch Overhead, Not Kernel Quality: The Performance Bottleneck in Single-Stream WebGPU LLM Inference
The study introduces a sequential-dispatch measurement method and finds that, for single-stream (batch size 1) LLM inference on WebGPU, per-dispatch overhead rather than kernel quality is the bottleneck. It concludes that reducing dispatch count is the effective optimization.
— via arXiv Distributed Computing, J\k{e}drzej Maczan
