Newest papers

CaptionFuse: Training-Free Composed Product Retrieval Via VLM Intent Captioning and Weighted Embedding Fusion

Zhao Gao, Mina Metias, Mohamed Noordeen Alaudeen, Bassel Saleh · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26661 · cited by 0 works (OpenAlex) · open-access PDF

French

Composite Image Retrieval (CIR) récupère des produits correspondant à une image de référence modifiée par le langage naturel---une capacité de base pour la recherche en e-commerce. Les méthodes existantes sans formation fusionnent les embeddings CLIP via l'arithmétique, mais nous identifions une limitation fondamentale: les modifications de texte utilisateur sont attribut-sparse---un utilisateur qui typique "make it red with shorter sleeves" spécifie ce qu'il faut changer mais pas ce qu'il faut garder (le motif floral, le collier V, la silhouette fit-and-flare). Nous introduisons la fusion langue-espace: un modèle Vision-Language (VLM) interprète conjointement l'image de référence et la modification pour génér

Machine translation (Meta M2M100); the English original follows.

Abstract

Composed Image Retrieval (CIR) retrieves products matching a reference image modified by natural language---a core capability for eCommerce search. Existing training-free methods fuse CLIP embeddings via arithmetic, but we identify a fundamental limitation: user text modifications are attribute-sparse---a user who types ``make it red with shorter sleeves'' specifies what to change but not what to keep (the floral pattern, the V-neck, the fit-and-flare silhouette). We introduce language-space fusion: a Vision-Language Model (VLM) jointly interprets the reference image and modification to generate a visually grounded product-title caption---e.g., ``Red floral fit-and-flare dress with short sleeves and V-neck''---that fills in the missing attributes from the image. Our method, CaptionFuse, requires no task-specific training and is model-agnostic. A controlled experiment confirms that attribute content--- not linguistic style---drives the gains: imperative and descriptive captions with identical attributes achieve equal R@10, while both outperform raw modifications by +6.5 points.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Closed captioning, Computer science, Fuse (electrical), Artificial intelligence, Embedding, Matching (statistics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange