Newest papers

GazeLens for Efficient Egocentric Video Question Answering with Simple Gaze-Driven Adaptive Token Reduction

Trong-Thang Pham, Anh Nguyen, Hien Van Nguyen, Ngan Le · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28502 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

スマートメガネは、ユーザーの活動や環境を理解するのに役立ちますが、小さなビジョン言語モデルは、限られた計算の範囲内で有用な証拠を保持しなければなりません。これは困難であるため、過剰なフレームは視覚予算を消費し、攻撃的な空間圧縮は、関連する詳細を捨てることができます。 視線は、使用者の焦点を特定しますが、問題は、周囲のシーンからの証拠を必要とします。 これらの制限に対処するために、我々はGazeLensを提案し、フレーム選択とGazeおよび質問ガイドされたトークン削減を組み合わせます。 まず、HOGの移行と画像の鋭さは、視覚的なエンコードの前にフレームを選択します。 各選択されたフレーム

Machine translation (Meta M2M100); the English original follows.

Abstract

Smart glasses can help users understand their activities and surroundings, but small vision-language models must preserve useful evidence within limited computation. This is difficult because redundant frames consume the visual budget, while aggressive spatial compression can discard relevant details. Although gaze identifies the wearer's focus, the question may require evidence from the surrounding scene. To address these constraints, we propose GazeLens, which combines frame selection with gaze- and question-guided token reduction. First, HOG transitions and image sharpness select frames before visual encoding. For each selected frame, Gaussian pooling then forms gaze-focused and complementary summaries. Because their relevance depends on the question, a lightweight gate blends them into a primary token. Together with a global scene token, this retains both focused and contextual summaries within two tokens per frame. On StreamGaze, GazeLens achieves higher macro and micro accuracy than the evaluated full-grid and token-reduction baselines, with lower input-forward FLOPs than the shared-backbone token-reduction baselines.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Security token, Pooling, Relevance (law), Artificial intelligence, Frame (networking)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange