Newest papers

Tandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUs

Zhuomin Chen, Jingchao Ni, Xu Zheng, Yong Chen, Mo Sha, Wei Cheng, Dongsheng Luo · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26660 · cited by 0 works (OpenAlex) · open-access PDF

Korean

스마트 폰 스크린 에이전트에서 스크린 에이전트는 화면을 반복적으로 읽고 다음 작업에 대한 모델을 요청함으로써 모바일 앱에서 작업을 완료합니다. 모바일 신경 처리 단위 (NPUs)는 로컬 실행을 가능하게하고 화면 콘텐츠를 휴대 전화에 보관하지만 모델 늦은 시간은 각 액션으로 축적됩니다. 하이브리드 선형 관심 모델은 메모리 요구 사항을 줄이지만 반복적 인 상태를 다시 사용하면 적절한 입력 위치에서 체크 포인트를 필요로합니다. 에이전트 요청은 지시를 유지하지만 최근 기록을 다시 작성하므로 채팅 지속을 위해 설계된 체크 포인트 정책은 재사용 가능한 프리픽스를 놓칠 수 있습니다. 추측적 인 디코딩을위한 롤백 상태를 유지하는 것은 또한

Machine translation (Meta M2M100); the English original follows.

Abstract

Smartphone GUI agents complete tasks in mobile apps by repeatedly reading the screen and asking a model for the next action. Mobile neural processing units (NPUs) enable local execution, keeping screen contents on the phone, but model latency accumulates with every action. Hybrid linear attention models reduce memory requirements, yet reusing their recurrent state requires checkpoints at the appropriate input positions. Agent requests retain instructions but rewrite recent history, so checkpoint policies designed for chat continuation can miss reusable prefixes. Retaining rollback states for speculative decoding can also move prompt processing off the NPU's matrix unit, adding overhead that outweighs decoding savings on short action outputs. A loaded model can further hold the NPU awake between tasks. We present Tandem, a runtime for hybrid GUI agents on mobile NPUs. Tandem saves recurrent state where consecutive prompts diverge, allowing later requests to reuse previously processed context. It keeps most prefill on the matrix unit and produces rollback states through a short per-token tail. Its verification kernel reads each weight block once for all proposed tokens. Between request bursts, Tandem releases NPU power votes without unloading the model. We implement Tandem in llama.cpp and evaluate it on recorded agent interactions across two Snapdragon generations. On a Snapdragon 8 Elite, Tandem reduces per-step latency by 30% for text-based screen descriptions and 16% for screenshots relative to llama.cpp's fastest NPU configuration, reduces energy per text step by 33%, and lowers standby power with the model loaded from 0.76 to 0.22 W.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Latency (audio), Operating system, Decoding methods, Reuse, Rollback

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange