Newest papers

From Common Information to Canonical Correlation Analysis and Beyond: Bridging Information Theory, Signal Processing, and Machine Learning

Paris A. Karakasis, Walid Saad · 2026 · Entropy

DOI: 10.3390/e28101161 · cited by 0 works (OpenAlex) · open-access PDF

Korean

신호 처리, 기계 학습 및 신경 이미징의 많은 문제는 하나의 프로세스의 여러 공동 관찰 측정을 제공하며, 각각은 그들 모두에게 공통적인 구성 요소를 혼합하여 자체에 개인적인 구성 요소를 복구하도록 요구합니다.이 문제는 여러 지역 사회에서 주로 독립적으로 연구되었습니다. 통계학자들은 1936 년 Hotelling에 의해 도입 된 카노닉 상관 관계 분석 (CCA)의 주요 결과를 제안했습니다. 정보 이론가들은 Gács와 Körner와 Wyner로 인해 공통 정보의 두 개의 개념을 제안했습니다. 엔지니어들은 센서 아레이지 및 멀티 사용자 통신을위한 하위 공간 방법을 개발했습니다.이 튜토리얼은 주요 결과와이 문제를위한 하나의 전시에서 개발 된 각 도구를 제

Machine translation (Meta M2M100); the English original follows.

Abstract

Many problems in signal processing, machine learning, and neuroimaging provide several jointly observed measurements of one process, each mixing a component common to all of them with a component private to itself, and require the common component to be recovered without supervision. This problem has been studied in several communities, largely independently. Statisticians proposed canonical correlation analysis (CCA), introduced by Hotelling in 1936. Information theorists proposed two notions of common information, due to Gács and Körner and to Wyner. Engineers developed subspace methods for sensor arrays and multiuser communications to design multiview receivers. This tutorial presents the main results and tools each community developed for this problem in a single exposition, and explains the connections that emerge where they meet, distinguishing exact equivalences from special cases and analogies. We develop the information-theoretic foundation from first principles. We then introduce CCA and its extension to several views, followed by the engineering thread from blind multiuser detection to CCA as a blind receiver. Next, we turn to nonlinear CCA, covering kernel, deep, and probabilistic formulations, the conditions under which the common factor is identifiable, and connections to self-supervised learning. We conclude with a discussion of further extensions and open directions. The exposition is self-contained, so that readers from each of these communities can follow the parts developed by the others.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Canonical correlation, Probabilistic logic, Computer science, Artificial intelligence, Bridging (networking), Subspace topology

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange