Newest papers

GazeLens for Efficient Egocentric Video Question Answering with Simple Gaze-Driven Adaptive Token Reduction

Trong-Thang Pham, Anh Nguyen, Hien Van Nguyen, Ngan Le · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28502 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Los lentes inteligentes pueden ayudar a los usuarios a comprender sus actividades y entornos, pero los modelos de lenguaje de visión pequeños deben conservar pruebas útiles dentro de la computación limitada. Esto es difícil porque los marcos redundantes consumen el presupuesto visual, mientras que la compresión espacial agresiva puede descartar los detalles relevantes. Aunque la mirada identifica el enfoque del portador, la pregunta puede requerir pruebas de la escena circundante. Para abordar estas restricciones, proponemos GazeLens, que combina la selección de marcos con la reducción de tokens guiados por la mirada y la pregunta. En primer lugar, las transiciones de HOG y la agudeza de la imagen seleccionan los marcos antes de la codificación visual. Para cada marco seleccionado, la agrupación gaussiana luego forma los resúmenes focaliz

Machine translation (Meta M2M100); the English original follows.

Abstract

Smart glasses can help users understand their activities and surroundings, but small vision-language models must preserve useful evidence within limited computation. This is difficult because redundant frames consume the visual budget, while aggressive spatial compression can discard relevant details. Although gaze identifies the wearer's focus, the question may require evidence from the surrounding scene. To address these constraints, we propose GazeLens, which combines frame selection with gaze- and question-guided token reduction. First, HOG transitions and image sharpness select frames before visual encoding. For each selected frame, Gaussian pooling then forms gaze-focused and complementary summaries. Because their relevance depends on the question, a lightweight gate blends them into a primary token. Together with a global scene token, this retains both focused and contextual summaries within two tokens per frame. On StreamGaze, GazeLens achieves higher macro and micro accuracy than the evaluated full-grid and token-reduction baselines, with lower input-forward FLOPs than the shared-backbone token-reduction baselines.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Security token, Pooling, Relevance (law), Artificial intelligence, Frame (networking)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange