• Source: Kernel-independent component analysis

In statistics, kernel-independent component analysis (kernel ICA) is an efficient algorithm for independent component analysis which estimates source components by optimizing a generalized variance contrast function, which is based on representations in a reproducing kernel Hilbert space. Those contrast functions use the notion of mutual information as a measure of statistical independence.




Main idea


Kernel ICA is based on the idea that correlations between two random variables can be represented in a reproducing kernel Hilbert space (RKHS), denoted by





F




{\displaystyle {\mathcal {F}}}

, associated with a feature map




L

x


:


F


↦

R



{\displaystyle L_{x}:{\mathcal {F}}\mapsto \mathbb {R} }

defined for a fixed



x
∈

R



{\displaystyle x\in \mathbb {R} }

. The





F




{\displaystyle {\mathcal {F}}}

-correlation between two random variables



X


{\displaystyle X}

and



Y


{\displaystyle Y}

is defined as





ρ


F



(
X
,
Y
)
=

max

f
,
g
∈


F




corr
⁡
(
⟨

L

X


,
f
⟩
,
⟨

L

Y


,
g
⟩
)


{\displaystyle \rho _{\mathcal {F}}(X,Y)=\max _{f,g\in {\mathcal {F}}}\operatorname {corr} (\langle L_{X},f\rangle ,\langle L_{Y},g\rangle )}


where the functions



f
,
g
:

R

→

R



{\displaystyle f,g:\mathbb {R} \to \mathbb {R} }

range over





F




{\displaystyle {\mathcal {F}}}

and




corr
⁡
(
⟨

L

X


,
f
⟩
,
⟨

L

Y


,
g
⟩
)
:=



cov
⁡
(
f
(
X
)
,
g
(
Y
)
)


var
⁡
(
f
(
X
)

)

1

/

2


var
⁡
(
g
(
Y
)

)

1

/

2







{\displaystyle \operatorname {corr} (\langle L_{X},f\rangle ,\langle L_{Y},g\rangle ):={\frac {\operatorname {cov} (f(X),g(Y))}{\operatorname {var} (f(X))^{1/2}\operatorname {var} (g(Y))^{1/2}}}}


for fixed



f
,
g
∈


F




{\displaystyle f,g\in {\mathcal {F}}}

. Note that the reproducing property implies that



f
(
x
)
=
⟨

L

x


,
f
⟩


{\displaystyle f(x)=\langle L_{x},f\rangle }

for fixed



x
∈

R



{\displaystyle x\in \mathbb {R} }

and



f
∈


F




{\displaystyle f\in {\mathcal {F}}}

. It follows then that the





F




{\displaystyle {\mathcal {F}}}

-correlation between two independent random variables is zero.
This notion of





F




{\displaystyle {\mathcal {F}}}

-correlations is used for defining contrast functions that are optimized in the Kernel ICA algorithm. Specifically, if




X

:=
(

x

i
j


)
∈


R


n
×
m




{\displaystyle \mathbf {X} :=(x_{ij})\in \mathbb {R} ^{n\times m}}

is a prewhitened data matrix, that is, the sample mean of each column is zero and the sample covariance of the rows is the



m
×
m


{\displaystyle m\times m}

dimensional identity matrix, Kernel ICA estimates a



m
×
m


{\displaystyle m\times m}

dimensional orthogonal matrix




A



{\displaystyle \mathbf {A} }

so as to minimize finite-sample





F




{\displaystyle {\mathcal {F}}}

-correlations between the columns of




S

:=

X



A


′




{\displaystyle \mathbf {S} :=\mathbf {X} \mathbf {A} ^{\prime }}

.




References

Kata Kunci Pencarian:

• Kernel-independent component analysis
• Independent component analysis
• Kernel principal component analysis
• Component analysis
• Principal component analysis
• Kernel density estimation
• Kernel (linear algebra)
• Amari distance
• Linux kernel
• Principal component regression