Newest papers

Evaluating the Applicability of LLM Inference Optimizations on Apple Silicon

Kyrylo Yemets, Andrii Yaroshevych, Victor Muryn, Ostap Khomenko, Pavlo Kryven, Maksym Kmet, Maksym Shamrai, Mariya Hirna · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31479 · cited by 0 works (OpenAlex) · open-access PDF

Korean

애플 실리콘은 로컬 LLM 추론을위한 가장 널리 사용되는 플랫폼 중 하나이지만, 거의 모든 추론 최적화는 서버 클래스 NVIDIA GPU에서 배치된 서비스 워크loads에 따라 설계되고 검증됩니다. 어떤 최적화는 애플 실리콘으로 이동할 때 그 혜택을 유지하고 무엇이 그 적용 가능성을 결정합니까? 우리는 4 개의 최적화 가족을 평가합니다: KV-cache pruning, 추측적 인 디코딩, 포트 GPU 코어 및 훈련 후 양성화. 실험은 일반적인 단일 요청 디코딩 워크loads에 따라 4 개의 애플 기계를 확장합니다. 가능한 경우 RTX 4090 비교와 일치합니다. KV-cache pruning는 애플을 통해 1.75 번의 엔드-에

Machine translation (Meta M2M100); the English original follows.

Abstract

Apple Silicon is among the most widely used platforms for local LLM inference, yet nearly all inference optimizations are designed and validated on server-class NVIDIA GPUs under batched serving workloads. Which optimizations retain their benefits when moved to Apple Silicon, and what determines their applicability? We evaluate four optimization families: KV-cache pruning, speculative decoding, ported GPU kernels, and post-training quantization. Experiments span four Apple machines under a common single-request decode workload, with matched RTX 4090 comparisons where possible. KV-cache pruning can reach up to 1.75 times end-to-end throughput on Apple but can reduce throughput on desktop NVIDIA GPUs. Under the common benchmark, all evaluated speculative decoding methods reduce throughput on Apple, despite reported 2-6 times gains on server GPUs. CUDA kernel designs transfer only partially: public Metal lacks asynchronous copy, and batch-one decode makes several serving-oriented mechanisms inapplicable, although transferable components yield up to 1.20 times decode throughput. Quantization gains vary by machine. Across four families, a reported speedup is not a portable property of a technique alone but a joint function of technique, runtime, hardware contract, and workload regime.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Porting, Parallel computing, Throughput, Speedup, Kernel (algebra)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange