Newest papers

CaptionFuse: Training-Free Composed Product Retrieval Via VLM Intent Captioning and Weighted Embedding Fusion

Zhao Gao, Mina Metias, Mohamed Noordeen Alaudeen, Bassel Saleh · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26661 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Composed Image Retrieval (CIR) mengumpulkan produk yang cocok dengan gambar referensi yang dimodifikasi oleh bahasa alami---sebuah kemampuan inti untuk pencarian eCommerce. Metode-metode bebas pelatihan yang ada menggabungkan embeddings CLIP melalui aritmetik, tetapi kami mengidentifikasi keterbatasan fundamental: modifikasi teks pengguna adalah atribut-sparse---pengguna yang mengetik "make it red with shorter sleeves" menentukan apa yang harus diubah tetapi tidak apa yang harus dipenuhi (gambar bunga, leher V, siluet fit-and-flare). Kami memperkenalkan fusion bahasa-ruang: sebuah Vision-Language Model (VLM) secara bersama-sama menafsirkan gambar referensi dan modifikasi untuk menghasilkan judul produk yang didasarkan secara visual---misalnya, "Red floral fit-and-flare dress

Machine translation (Meta M2M100); the English original follows.

Abstract

Composed Image Retrieval (CIR) retrieves products matching a reference image modified by natural language---a core capability for eCommerce search. Existing training-free methods fuse CLIP embeddings via arithmetic, but we identify a fundamental limitation: user text modifications are attribute-sparse---a user who types ``make it red with shorter sleeves'' specifies what to change but not what to keep (the floral pattern, the V-neck, the fit-and-flare silhouette). We introduce language-space fusion: a Vision-Language Model (VLM) jointly interprets the reference image and modification to generate a visually grounded product-title caption---e.g., ``Red floral fit-and-flare dress with short sleeves and V-neck''---that fills in the missing attributes from the image. Our method, CaptionFuse, requires no task-specific training and is model-agnostic. A controlled experiment confirms that attribute content--- not linguistic style---drives the gains: imperative and descriptive captions with identical attributes achieve equal R@10, while both outperform raw modifications by +6.5 points.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Closed captioning, Computer science, Fuse (electrical), Artificial intelligence, Embedding, Matching (statistics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange