Newest papers

From Common Information to Canonical Correlation Analysis and Beyond: Bridging Information Theory, Signal Processing, and Machine Learning

Paris A. Karakasis, Walid Saad · 2026 · Entropy

DOI: 10.3390/e28101161 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

许多问题在信号处理,机器学习和神经成像提供多个共同观察的一种过程的测量部分,每一个混合一个常见的组件与一个组件私有本身,因此需要共同的组件被恢复而不受监督。这个问题已被研究在几个社区,大多是独立的。统计学家提出了加拿大相关分析(CCA)的主要结果和工具,由Hotelling在1936年引入了这个问题,信息理论家提出了两种常见信息的概念,由于Gács和Körner和Winner。工程师开发了传感器阵列和多用户通信的子空间方法,以设计多视觉接收器。本教程介绍了主要的结果和每个工具社区在一个单一的展览中开发了这个问题,并解释了他们遇到的连接,从特殊案例和类似

Machine translation (Meta M2M100); the English original follows.

Abstract

Many problems in signal processing, machine learning, and neuroimaging provide several jointly observed measurements of one process, each mixing a component common to all of them with a component private to itself, and require the common component to be recovered without supervision. This problem has been studied in several communities, largely independently. Statisticians proposed canonical correlation analysis (CCA), introduced by Hotelling in 1936. Information theorists proposed two notions of common information, due to Gács and Körner and to Wyner. Engineers developed subspace methods for sensor arrays and multiuser communications to design multiview receivers. This tutorial presents the main results and tools each community developed for this problem in a single exposition, and explains the connections that emerge where they meet, distinguishing exact equivalences from special cases and analogies. We develop the information-theoretic foundation from first principles. We then introduce CCA and its extension to several views, followed by the engineering thread from blind multiuser detection to CCA as a blind receiver. Next, we turn to nonlinear CCA, covering kernel, deep, and probabilistic formulations, the conditions under which the common factor is identifiable, and connections to self-supervised learning. We conclude with a discussion of further extensions and open directions. The exposition is self-contained, so that readers from each of these communities can follow the parts developed by the others.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Canonical correlation, Probabilistic logic, Computer science, Artificial intelligence, Bridging (networking), Subspace topology

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange