Source Separation, ICA via Unsupervised Learning: Information-theoretic Approach and Conditions on Cross-cumulants

The problem of separation of independent source signals from their instantaneous mixtures, is addressed by using two diierent approaches: (1) information-theoretic approach; (2) higher-order cross-cumulants approach. Natural gradient 4] is known as eecient and fast convergent for on-line learning. First, we present further results on natural gradient-based source separation algorithms. Second, we discuss source separation criteria based on higher-order cross-cumulants. Blind source separation is a fundamental problem encountered in many applications such as array signal processing, sonar, digital communications, some biomedical applications. The goal of blind source separation is to recover the independent source signals from their linear mixtures without the knowledge of mixing matrix. For instance, the narrow-band signals received by antenna sensors in communication systems are often a linear transformation of statistically independent source signals, and it is desirable to restore these source signals without the knowledge of the linear transformation. The problem is formulated as follows. The received signals x(t) 2 IR m consist of instantaneous mixtures of independent source signals s(t) 2 IR n by a mixing matrix A 2 IR mn , i.e., x(t) = As(t): (1) In source separation as shown in Figure 1 , it is required to update a demixing networkW(t), i.e., y(t) = W(t)x(t); (2) such that the global system G(t) = W(t)A converges as t ! 1 to a matrix G = PP; (3) for some permutation matrix P and nonsingular diagonal matrix. Unknown Mixing Demixing PSfrag replacements s(t) x(t) y(t) A W(t) Figure 1. Schematic diagram for source separation via unsupervised learning Source separation is often called as ICA, which can be viewed as a nonlinear extension of principal component analysis (PCA) where only second-order statistics is considered. Neural computational approach to blind source separation was rst introduced by Jutten and Herault 18]. Introducing nonlinear functions in a linear feedback network, the network was trained in unsu-pervised manner to nd a linear transformation which forces higher-order correlations between network outputs to vanish zero. Later this was further developed by Cichocki et al 17] by employing self-connections for normalization. Bell and Sejnowski 6] presented a simple learning algorithm derived from information-theoretic approach, while past algorithms were based on heuristic investigation. Amari et al 1] introduced natural gradient and presented a algorithm with approximating a marginal entropy using Gram-Charlier