What Is Source Separation?

Blind signal separation was first proposed by Herault and Jutten in 1985, which refers to analyzing unobserved original signals from multiple observed mixed signals. Commonly observed mixed signals come from the output of multiple sensors, and the output signals of the sensors are independent (linearly uncorrelated). The word "blind" of the blind signal emphasizes two points: 1) the original signal is unknown and 2) the method of signal mixing is unknown.

In the literal sense, signal separation is to separate or recover the original source signal from the received mixed signal (typically the signal of interest + interference + noise). Signal separation is a basic problem in signal processing. Various time-domain filters, frequency-domain filters, space-domain filters, or code-domain filters can all be regarded as a kind of signal separator to complete the task of signal separation. It is just that the "separation" at this time is only to separate the required signals of interest, which is slightly different from the "separation" mentioned earlier. It can be seen that signal separation is the embodiment of many signal processing common features. When the prior knowledge of the source signal and the transmission channel is known, the above pass

If you can directly model the way signals are mixed, this is certainly the best way. However, in blind signal separation, we do not know how signals are mixed, so only statistical methods can be used. The algorithm makes the following assumptions: has m independent signal sources

with

Independent observations

Observation and signal source have the following relationship

among them

Is an

Coefficient matrix, the original problem becomes known

with

Independence

Estimation problem. Assume the following formula

among them

is true

Estimate, W is a

Coefficient matrix, the question becomes how to effectively estimate the matrix W.

The basic assumptions of the problem are as follows: 1) Each source signal

All are zero-mean signals, real random variables, and the signals are statistically independent. If the source signal

The probability density is

then

The probability density is:

. 2) Number of source signals

Less than or equal to the number of observation signals

, which is

. Mixed matrix

Is an

Matrix. assumed

Full rank. 3) Only one Gaussian distribution is allowed in the source signal. When there is more than one Gaussian distribution, the source signal becomes indivisible.

The calculation formula of the natural gradient method is:

among them

We need an estimated matrix.

Step size

Is a non-linear transformation, such as

, In actual calculation

For one

matrix,

Is the number of original signals,

Is the number of sampling points. The basic steps of the algorithm are:

1) Initialization

Identity matrix

2) Cycle through the following steps until

versus

The difference is less than the specified value

(The method of calculating the matrix difference can be artificially specified), and sometimes the number of iterations is also artificially specified

3) Use formula

(among them

)

4) Use formula

The real progress of blind separation was in the late 1980s. The pioneering work was mainly done by Jutten and Herault in 1986: they proposed an adaptive algorithm to complete the separation of the two aliasing source signals. Later, Jutten, Herault, Comon, Sorouchyari, and others published three classic articles on blind source separation in "Signal Processing" in 1991, marking a significant progress in blind separation research. At the same time, L. Tong et al. Conducted a preliminary study on the identifiability of the blind separation problem in 1991, and it was not until 1996 that Cao Xiren completely solved the solvability condition of blind separation. In 1994, Comon systematically analyzed the problem of instantaneous blind signal separation, and explicitly introduced the concept of Independent Component Analysis (ICA), proving that as long as the independence of each signal in the mixed signal is restored, the source signal can be completed. Of separation. It can be said that the work of Comon actually made the research on blind signal separation algorithms into the cost function of independent component analysis and its optimization algorithm, which made the future algorithm design have a clear theoretical basis. Since then, a large number of excellent blind separation algorithms have emerged. AJBell and TJSejnowski proposed the information maximization method (Informax) in 1995, and T. Lee improved the algorithm (ExInformax) in 1999; JFCardoso proposed the non-linear PCA algorithm in 1996 and the largest in 1997. Likelihood algorithm. Amari proposed a mutual information minimum algorithm based on natural gradients in 1998, which reduced the calculation amount of the algorithm. Hyvarinen and Oja proposed the fixed-point algorithm FastICA in blind separation in 1997, laying a solid foundation for further practical applications. After entering the new century, as the basic theory of blind separation has gradually matured, more and more scholars have invested in the research of extended blind separation, which has promoted noisy blind separation, underdetermined blind separation, convolutional blind separation and non-blind separation. Development of linear blind separation, etc. On this basis, relevant theoretical summaries and monographs continue to emerge, further promoting the development of the discipline. "Proceedings of IEEE" in October 1998 is a blind signal processing album, Haykin, Hyvarinen, Cichocki and others have published monographs on blind signal separation.

Since the pioneering work of Herault and Jutten, a lot of research work has been done in this field, and many effective blind separation algorithms have been proposed from different perspectives. In order to better understand and compare the principles and characteristics of these algorithms, it is necessary to classify them according to certain principles. Classification according to the statistical information of the BSS used. Generally, according to the statistical information of the source signals that the algorithm depends on, blind source signal separation algorithms can be divided into the following three categories:

Blind separation algorithm based on information theory or likelihood estimation

This type of algorithm is based on information theory. The criterion for judging signal separation is to maximize the statistical independence of the output signals of the separation system (minimum of mutual information, maximization of negative entropy, etc.). In addition to requiring independent source signals, this type of algorithm also requires that the source signal contains at most one Gaussian signal. Cardoso et al. Proved that the likelihood estimation algorithm is equivalent to the information theory algorithm, so the likelihood estimation algorithm and the information theory algorithm can be classified into the same category. Typical algorithms based on information theory include Amari's neural network-based natural gradient algorithm, Imformax algorithm, and so on. In addition, non-linear algorithms fall into this category. Although the starting point of the non-linear method is different from the information theory and the likelihood estimation method, they are similar in algorithm and both are implemented by a non-linear neural network.

Information theory-based blind separation algorithms are usually adaptive online learning algorithms. The disadvantage of this type of algorithm is that the nonlinear excitation function is related to the statistical distribution characteristics of the signal (whether it is a sub-Gaussian or super-Gaussian distribution). When the source signal has both a sub-Gaussian signal and a super-Gaussian signal, it will cause trouble. There are two ways to solve this problem. One is to adaptively estimate the type of excitation function, and then select a suitable function method in a given excitation function. Another method is to perform Edegworth or Gram Charlier expansion on the probability density function of the source signal, so as to express the non-linear excitation function as a function of separating the cumulants of each order of the signal, and adaptively estimate these statistics. Information theory-based BSS methods usually have good stability and convergence.

Blind separation algorithm based on second-order statistics

Such algorithms are also called decorrelation algorithms. This type of algorithm requires statistical uncorrelation between the source signals. In addition, the source signal is required to be non-white or non-stationary. In other words, the decorrelation algorithm cannot separate statistically independent, smooth white noise processes (regardless of their probability distribution). The main advantage of the decorrelation algorithm is that the algorithm is relatively simple and has good stability, which is suitable for source signals with any probability distribution. Under the framework of second-order moment theory, to fully describe a non-white and non-stationary random process, its two-dimensional autocorrelation function must be used. From the perspective of blind signal separation, the non-whiteness and non-stationarity of the source signal are equivalent ^[2]

In statistics, independent component analysis or Independent Components Analysis (Abbreviation: ICA) is a method that uses statistical principles to perform calculations. It is a linear transformation. This transformation separates data or signals into a linear combination of statistically independent non-Gaussian signal sources. Independent component analysis is a special case of blind source separation. The most important assumption of independent component analysis is that the signal sources are statistically independent. This assumption is true in most cases of blind signal separation. Even when this assumption is not satisfied, independent component analysis can still be used to statistically separate the observation signals to further analyze the characteristics of the data. The classic problem of independent component analysis is the "cocktail party problem". This question describes how to separate the independent signals of each person speaking at the same time in a cocktail party, given a mixed signal. When there is

Signal sources, it is usually assumed that the observation signal also has

(For example

Microphone or recorder). This assumption means that the mixing matrix is a square matrix, that is,

,among them

Is the dimension of the input data,

Is the dimension of the system model. for

with

There are also different studies in academia. Independent component analysis cannot fully recover the specific value of the signal source, nor can it solve the sign of the signal source, the number of levels of the signal, or the range of the signal. Independent component analysis is an important method to study blind signal separation, and it has many applications in practice.