Newest papers

Evaluating the Applicability of LLM Inference Optimizations on Apple Silicon

Kyrylo Yemets, Andrii Yaroshevych, Victor Muryn, Ostap Khomenko, Pavlo Kryven, Maksym Kmet, Maksym Shamrai, Mariya Hirna · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31479 · cited by 0 works (OpenAlex) · open-access PDF

German

Apple Silicon ist eine der am häufigsten verwendeten Plattformen für lokale LLM-Untersuchungen, aber fast alle Inferenzoptimierungen werden auf NVIDIA-GPUs der Serverklasse unter Batch-Serving-Workloads entworfen und validiert. Welche Optimierungen behalten ihre Vorteile, wenn sie auf Apple Silicon verschoben werden, und was bestimmt ihre Anwendbarkeit? Wir bewerten vier Optimierungsfamilien: KV-Cache-Abschneiden, spekulative Dekodierung, portierte GPU-Kerne und Post-Training-Quantizierung. Experimente umfassen vier Apple-Maschinen unter einem gemeinsamen Single-Request-Decode-Workload, mit übereinstimmten RTX 4090-Vergleichungen, wo möglich. KV-Cache-Abschneiden kann bis zu

Machine translation (Meta M2M100); the English original follows.

Abstract

Apple Silicon is among the most widely used platforms for local LLM inference, yet nearly all inference optimizations are designed and validated on server-class NVIDIA GPUs under batched serving workloads. Which optimizations retain their benefits when moved to Apple Silicon, and what determines their applicability? We evaluate four optimization families: KV-cache pruning, speculative decoding, ported GPU kernels, and post-training quantization. Experiments span four Apple machines under a common single-request decode workload, with matched RTX 4090 comparisons where possible. KV-cache pruning can reach up to 1.75 times end-to-end throughput on Apple but can reduce throughput on desktop NVIDIA GPUs. Under the common benchmark, all evaluated speculative decoding methods reduce throughput on Apple, despite reported 2-6 times gains on server GPUs. CUDA kernel designs transfer only partially: public Metal lacks asynchronous copy, and batch-one decode makes several serving-oriented mechanisms inapplicable, although transferable components yield up to 1.20 times decode throughput. Quantization gains vary by machine. Across four families, a reported speedup is not a portable property of a technique alone but a joint function of technique, runtime, hardware contract, and workload regime.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Porting, Parallel computing, Throughput, Speedup, Kernel (algebra)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange