Newest papers

SAKURA: Serving Attention Key-Value State Under Rate Adaptation for Private Image Editing on Phones

Shuncheng Jie, Desen Sun, Ri Hong, Yuyang Tian, Hong Zhang, Sihang Liu · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29733 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

Os transformadores de latência de difusão são amplamente utilizados em serviços de edição de imagens. Para preservar a eficiência e a privacidade, os sistemas existentes geralmente dependem da colaboração nuvem-dispositivo. Manter a inferência específica de solicitação no dispositivo evita o envio de imagens pessoais e prompts para a nuvem. Essas cargas de trabalho muitas vezes usam modelos comuns, como na troca de rostos e na tentativa virtual, permitindo que o estado KV seja pré-computado e reutilizado em solicitações. No entanto, este estado reutilizável é grande e escala com o número de modelos, tornando impraticável armazená-los todos em dispositivos móveis. Armazená-los na nuvem aborda este problema de capacidade, mas a transferência de cache é colocada no caminho de inferência virtual e pode

Machine translation (Meta M2M100); the English original follows.

Abstract

Diffusion transformers are widely used in image-editing services. To preserve both efficiency and privacy, existing systems commonly rely on cloud-device collaboration. Keeping request-specific inference on the device avoids sending personal images and prompts to the cloud. These workloads often use common templates, such as in face swapping and virtual try-on, enabling KV state to be precomputed and reused across requests. However, this reusable state is large and scales with the number of templates, making it impractical to store all of them on mobile devices. Storing them in the cloud addresses this capacity issue, but cache transfer is placed onto the inference path and can even be slower than on-device computation, especially under fluctuating mobile networks. We introduce SAKURA, a cloud-edge architecture that delivers reusable attention state to mobile devices under a bandwidth budget. It first uses a fine-grained token-importance-aware compression method to preserve image quality under limited transfer budgets. At runtime, SAKURA adaptively selects the cache representation for each upcoming shard based on network bandwidth. To enable efficient execution, we develop a pipelined runtime that overlaps cache transfer and reconstruction with on-device execution. We evaluate SAKURA using three editing tasks and two diffusion models, using a Snapdragon 8 Elite Gen 5 phone. SAKURA reduces latency by 1.82x and 3.27x over CacheGen adapted to DiT and on-device generation, respectively. It also reduces cache storage by 3.06x over uncompressed cache, while preserving image quality.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Cache, Mobile device, Mobile processor, Latency (audio), Inference

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange