Newest papers

NeuroCodec: EEG-Guided Latent-Domain Generative Model for High-Fidelity Target Speaker Extraction

Julian Silva, Jaliya Kodithuwakku, Navindu Sirimanna, Shaveen Herath, Wageesha N. Manamperi, Joshua P. Kulasingham · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29815 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

为了解决这一问题,听力技术使用通过电动脑传输(EEG)录制的神经信号来识别注意力,以引导TSE,使目标扬声器提取(TSE)在多话环境中获得了显著的关注。然而,大多数现有的技术依赖于时间频率掩护,这在光谱时重叠的区域中受到低忠诚度的影响,并且通常仅限于狭带音频。相比之下,我们引入了NeuroCodec,这在我们所知的最佳程度上是第一个基于EEG引导的TSE内容的生成框架,该框架在公共神经音频代码的隐藏空间中运作。 具体来说,NeuroCodec 提供了通过图形转换和FEG模型进行空间依赖的EEG模型,我们通过

Machine translation (Meta M2M100); the English original follows.

Abstract

Driven by recent advancements in hearing assistive devices, target speaker extraction (TSE) in multi-talker environments has drawn significant attention. To address this, hearing technologies identify attention using neural signals recorded via electroencephalography (EEG) to guide TSE, enabling neuro-steered auditory attention decoding. However, most existing techniques rely on time-frequency masking, which suffers from low fidelity in spectro-temporally overlapping regions and is often restricted to narrowband audio. In contrast, we introduce NeuroCodec, which, to the best of our knowledge, is the first generative EEG-guided TSE framework operating in the latent space of a neural audio codec. Specifically, NeuroCodec models spatial EEG dependencies via graph convolutions and fuses cross-modal EEG and speech representations through attention mechanisms and Mamba-based temporal modeling. We further introduce a six-phase training curriculum that progressively incorporates latent reconstruction, multi-scale spectral, and adversarial objectives to ensure stable, high-fidelity speech generation. Experiments on the public Cocktail Party and KUL datasets show NeuroCodec outperforms baselines, across several perceptual metrics. A listening experiment with 20 participants validated that NeuroCodec offers greater perceptual accuracy, achieving a mean score of 82.24 compared with 49.63 for M3ANet. Additionally, the proposed method is seen to enable 44.1 kHz reconstruction with spectral content extending up to 20 kHz, further supporting the perceptual results. Finally, we extend NeuroCodec with a causal streaming formulation suited for real-time processing, providing a basis for hearing assistive devices.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Speech recognition, Artificial intelligence, Fidelity, Generative model, Perception

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange