Newest papers

SpecKCast: Spectral Kalman Forecasting for Diffusion Acceleration

Javad Rajabi, Hakkı Can Karaimer, David B. Lindell, Babak Taati, Michael S. Brown · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26837 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Diffusion Transformers (DiTs) melaporkan kualitas gambar dan produksi video yang tinggi, tetapi mereka menderita biaya inferensi yang signifikan karena evaluasi transformator berulang di berbagai langkah denoising. Caching fitur mengurangi biaya ini dengan menggunakan kembali fitur-fitur di berbagai langkah denoising. Metode prediksi terbaru juga mengabaikan gagasan ini dengan mengekstrak fitur-fitur ke langkah-langkah yang dilewatkan daripada menggunakannya kembali. Namun, metode prediksi yang ada mengekstrak fitur-fitur yang mereka prediksi secara serentak, meskipun komponen frekuensi spasial dari fitur-fitur ini paling banyak berubah pada tahap-tahap yang berbeda dari denoising. Mereka juga mengabaikan bahwa denoising berkembang tidak serentak, sehingga interval langkah yang sama dapat sesuai dengan jumlah yang sangat berbeda dari perubahan. Metode prediksi ini tidak cocok untuk memperkuat kesalahan

Machine translation (Meta M2M100); the English original follows.

Abstract

Diffusion Transformers (DiTs) achieve high-fidelity image and video generation, but they suffer from substantial inference costs due to repeated transformer evaluations across many denoising steps. Feature caching reduces this cost by reusing features across denoising steps. Recent forecasting methods extend this idea by extrapolating features to skipped steps instead of reusing them. However, existing forecasting methods extrapolate the features they predict uniformly, even though the spatial-frequency components of these features change most at different stages of denoising. They also overlook that denoising progresses nonuniformly, so equal step intervals can correspond to very different amounts of change. These mismatches amplify forecast errors as more steps are skipped, limiting the achievable acceleration. We propose SpecKCast, a training-free method that forecasts the model output in the frequency domain. SpecKCast applies a 2D discrete cosine transform (DCT) to the output and tracks each coefficient with a lightweight Kalman filter along a band-specific time coordinate derived from the noise schedule. The filter tracks each coefficient's value and rate of change, and the transformer is evaluated only when the filter's forecast uncertainty exceeds a threshold. Experiments on FLUX.1-dev, Qwen-Image, Wan2.1-14B, and HunyuanVideo show that SpecKCast achieves a better trade-off between acceleration and fidelity than existing caching and forecasting methods, reaching up to 6x speedup with less than 0.05 GiB of additional memory.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Kalman filter, Computer science, Granularity, Inference, Algorithm, Noise reduction

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange