arxiv.org/abs/2610.31479 Evaluating the Applicability of LLM Inference Optimizations on Apple Silicon
A benchmark study of four LLM inference optimization families on four Apple machines, with RTX 4090 comparisons, finds that gains do not transfer reliably. KV-cache pruning reached up to 1.75 times throughput on Apple, while all tested speculative decoding methods reduced it and ported CUDA kernels helped only partially, by up to 1.20 times.
— via arXiv Distributed Computing, Kyrylo Yemets, Andrii Yaroshevych, Victor Muryn, Ostap Khomenko, Pavlo Kryven, Maksym Kmet, Maksym Shamrai, Mariya Hirna
