Newest papers

CRANE: Post-Hoc Counterfactual Retrieval for Multimodal Classifiers

Franco Rugolon, Ioanna Miliou, Panagiotis Papapetrou · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31594 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Metode penjelasan tunggal counterfactual telah diperkenalkan untuk pengaturan unimodal, dengan kerangka kerja khusus untuk tabel, timeshare, teks, dan data gambar, namun tidak ada metode post-hoc yang dirancang untuk klasifikator yang belajar dari input multimodal. Metode penjelasan yang ada untuk klasifikator multimodal menerapkan atribusi spesifik modal secara independen untuk setiap input, yang gagal menghasilkan penjelasan yang konsisten dan tidak mentransfer ke arsitektur multimodal yang telah dilatih sebelumnya. Untuk mengatasi kekurangan ini, kami memperkenalkan CRANE (Counterfactual Retrieval via Agnostic Nearest-Neighbour Explanations), kerangka kerja model-agnostic pertama untuk generasi counterfactual multimodal terdekat yang mengumpulkan contoh pelatihan nyata dari kelas sasaran sebagai penjelasan, memastikan plausibilitas oleh konstruksi. Melalui empat

Machine translation (Meta M2M100); the English original follows.

Abstract

Counterfactual explanation methods have been introduced for unimodal settings, with dedicated frameworks for tabular, timeseries, text, and image data, yet no post-hoc method has been designed for classifiers that learn from multimodal inputs. Existing explainability methods for multimodal classifiers apply modality-specific attribution independently to each input, which fails to yield coherent explanations and does not transfer to arbitrary pre-trained multimodal architectures. To address these shortcomings, we present CRANE (Counterfactual Retrieval via Agnostic Nearest-neighbour Explanations), the first post-hoc, model-agnostic framework for nearest-neighbour multimodal counterfactual generation that retrieves real training examples from the target class as candidate explanations, ensuring plausibility by construction. Across four heterogeneous datasets and three fusion strategies, we show that Multimodal Consensus Retrieval (MC-R), which intersects per-modality neighbour sets to identify instances jointly close to the query in every modality, emerges as the strongest multimodal strategy, achieving the best or joint-best outcome on three of four datasets while grounding every explanation in the joint training distribution. Unimodal strategies remain competitive when a single modality dominates the decision boundary.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Counterfactual thinking, Computer science, Artificial intelligence, Machine learning, Class (philosophy), Modality (human–computer interaction)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange