Newest papers

SAKURA: Serving Attention Key-Value State Under Rate Adaptation for Private Image Editing on Phones

Shuncheng Jie, Desen Sun, Ri Hong, Yuyang Tian, Hong Zhang, Sihang Liu · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29733 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

拡散変換器は、画像編集サービスで広く使用されています。効率性とプライバシーの両方を維持するために、既存のシステムは、一般にクラウドデバイス協力に依存します。デバイス上のリクエスト特有の推測を維持すると、個人画像やプロンプトをクラウドに送信することは避けられます。これらのワークロードは、頻繁に顔交換や仮想トライオンなどの共通のテンプレートを使用しますが、KVステータスは推測パスに置かれており、リクエストで再利用することもできます。しかし、この再利用可能な状態は幅広く、テンプレートの数とスケールされ、モバイルデバイスにそれらをすべて保存することは非実用的です。クラウドに保存することは、この容量の問題

Machine translation (Meta M2M100); the English original follows.

Abstract

Diffusion transformers are widely used in image-editing services. To preserve both efficiency and privacy, existing systems commonly rely on cloud-device collaboration. Keeping request-specific inference on the device avoids sending personal images and prompts to the cloud. These workloads often use common templates, such as in face swapping and virtual try-on, enabling KV state to be precomputed and reused across requests. However, this reusable state is large and scales with the number of templates, making it impractical to store all of them on mobile devices. Storing them in the cloud addresses this capacity issue, but cache transfer is placed onto the inference path and can even be slower than on-device computation, especially under fluctuating mobile networks. We introduce SAKURA, a cloud-edge architecture that delivers reusable attention state to mobile devices under a bandwidth budget. It first uses a fine-grained token-importance-aware compression method to preserve image quality under limited transfer budgets. At runtime, SAKURA adaptively selects the cache representation for each upcoming shard based on network bandwidth. To enable efficient execution, we develop a pipelined runtime that overlaps cache transfer and reconstruction with on-device execution. We evaluate SAKURA using three editing tasks and two diffusion models, using a Snapdragon 8 Elite Gen 5 phone. SAKURA reduces latency by 1.82x and 3.27x over CacheGen adapted to DiT and on-device generation, respectively. It also reduces cache storage by 3.06x over uncompressed cache, while preserving image quality.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Cache, Mobile device, Mobile processor, Latency (audio), Inference

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange