Newest papers

CaptionFuse: Training-Free Composed Product Retrieval Via VLM Intent Captioning and Weighted Embedding Fusion

Zhao Gao, Mina Metias, Mohamed Noordeen Alaudeen, Bassel Saleh · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26661 · cited by 0 works (OpenAlex) · open-access PDF

German

Composite Image Retrieval (CIR) erfasst Produkte, die ein Referenzbild mit natürlicher Sprache modifizieren - eine Kernfunktion für die E-Commerce-Suche. Bestehende Training-freie Methoden fusionieren CLIP-Embeddings über Arithmetik, aber wir identifizieren eine grundlegende Einschränkung: Benutzertextänderungen sind Attribute-sparse---ein Benutzer, der "machen es rot mit kürzeren Ärmeln" typisiert, spezifiziert, was zu ändern ist, aber nicht, was zu behalten ist (das florale Muster, der V-Hals, die fit-and-flare Silhouette). Wir führen eine Sprach-Space-Fusion ein: ein Vision-Language-Modell (VLM) interpretiert gemeinsam das Referenzbild

Machine translation (Meta M2M100); the English original follows.

Abstract

Composed Image Retrieval (CIR) retrieves products matching a reference image modified by natural language---a core capability for eCommerce search. Existing training-free methods fuse CLIP embeddings via arithmetic, but we identify a fundamental limitation: user text modifications are attribute-sparse---a user who types ``make it red with shorter sleeves'' specifies what to change but not what to keep (the floral pattern, the V-neck, the fit-and-flare silhouette). We introduce language-space fusion: a Vision-Language Model (VLM) jointly interprets the reference image and modification to generate a visually grounded product-title caption---e.g., ``Red floral fit-and-flare dress with short sleeves and V-neck''---that fills in the missing attributes from the image. Our method, CaptionFuse, requires no task-specific training and is model-agnostic. A controlled experiment confirms that attribute content--- not linguistic style---drives the gains: imperative and descriptive captions with identical attributes achieve equal R@10, while both outperform raw modifications by +6.5 points.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Closed captioning, Computer science, Fuse (electrical), Artificial intelligence, Embedding, Matching (statistics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange