Aapo Hyvärinen: Publications

Publications menu

Bayesian networks & causal analysis Estimation theory Unsupervised learning: misc. Natural image statistics & visual cortex Neuroscience: misc. The FastICA algorithm Blind separation of sources with temporal structure Independent component analysis: misc.   Publications front page Back to home page

Independent component analysis: misc.

[A collection of papers on various topics on the theory of ICA]

Reviews on ICA

[You may first want to see the what is independent component analysis page]

A. Hyvärinen and E. Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks, 13(4-5):411-430, 2000.
html (and Japanese version)  Postscript  gzipped PostScript  pdf
[A tutorial text on ICA in general, and FastICA in particular.]

A. Hyvärinen, J. Karhunen and E. Oja. Independent Component Analysis.
[The main reference book on ICA]

A. Hyvärinen, J. Karhunen and E. Oja. Introductory Chapter of the book Independent Component Analysis.
Postscript   pdf
[A short very accessible introduction to ICA (and our book).]

A. Hyvärinen. Survey on Independent Component Analysis. Neural Computing Surveys 2:94--128, 1999.
html  Postscript   gzipped PostScript   pdf
[A comprehensive review on ICA estimation methods.]

A. Hyvärinen and Y. Kano. Independent component analysis for non-normal factor analysis. Proc. International Meeting of the Psychometric Society (IMPS2001), Osaka, Japan.
Postscript  pdf
[Tutorial of ICA for people who already are familiar with factor analysis.]

Validation and testing

S. Shimizu, A. Hyvärinen, Y. Kano, P. O. Hoyer and A. J. Kerminen. Testing significance of mixing and demixing coefficients in ICA. Proc. International Symposium on Independent Component Analysis and Blind Signal Separation (ICA2006), Charleston, SC, USA, 2006.
pdf matlab code
[Proposes tests to see whether coefficients in the ICA matrices are significantly different from zero or not.]

J. Himberg, A. Hyvärinen and F. Esposito. Validating the independent components of neuroimaging time-series via clustering and visualization. NeuroImage 22(3):1214-1222, 2004.
pdf    software
[A method for analyzing the reliability and stability of estimated independent components by re-running the algorithm many times and visualizing the relationships of the obtained component. Features an easy-to-use software package for Matlab.]

Dependent components

A. Hyvärinen and S. Shimizu. A quasi-stochastic gradient algorithm for variance-dependent component analysis. In Proc. International Conference on Artificial Neural Networks (ICANN2006), Athens, Greece, pp. 211-220, 2006.
pdf
[Proposes a new algorithm to blindly separate sources which are dependent due to correlations in the variances, i.e. general activity levels.]

A. Hyvärinen, P.O. Hoyer and M. Inki. Topographic Independent Component Analysis. Neural Computation, 13(7):1527-1558, 2001.
Postscript  gzipped PostScript   pdf
[Introduces an extension of ICA. The dependencies of the estimated "independent" components are visualized as a topographic order. A new principle for topographic organization, based on higher-order statistics. Applied on image data, both topography and complex cell properties emerge, see section on the visual cortex models.]

[Many more papers on this topic can be found in the sections on sparse coding in the visual cortex and one also here

Nonlinear ICA

A. Hyvärinen and P. Pajunen. Nonlinear Independent Component Analysis: Existence and Uniqueness results. Neural Networks 12(3): 429--439, 1999.
Postscript  gzipped PostScript  pdf
[Shows that the solution of the nonlinear ICA problem is highly non-unique, and proposes a restriction of the model that does make the solution unique.]

P. Pajunen, A. Hyvärinen and J. Karhunen. Non-Linear Blind Source Separation by Self-Organizing Maps. In Proc. Int. Conf. on Neural Information Processing, Hong Kong, pp. 1207-1210, 1996.
Postscript  gzipped PostScript
[Describes a simple if limited method for doing nonlinear (nonparametric) ICA for subgaussian signals.]

Noisy data and denoising

A. Hyvärinen. Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation. Neural Computation, 11(7):1739--1768, 1999.
Postscript  gzipped PostScript  pdf
[Describes sparse code shrinkage, which is a new method for denoising of images etc. It is a kind of a combination of independent component analysis and wavelet shrinkage ideas.]

A. Hyvärinen, P. Hoyer and E. Oja. Image Denoising by Sparse Code Shrinkage. In S. Haykin and B. Kosko (eds), Intelligent Signal Processing, IEEE Press, 2001.
Postscript  gzipped PostScript  pdf
[Describes sparse code shrinkage (see preceding paper) in more detail.]

A. Hyvärinen. Independent Component Analysis in the Presence of Gaussian Noise by Maximizing Joint Likelihood. Neurocomputing, 22:49-67, 1998.
Postscript  gzipped PostScript.
[Older work on noisy independent component analysis and its connections to competitive learning.]

(See also the FastICA section for a method based on "Gaussian Moments" for noisy ICA)

Other topics

A. Hyvärinen and R. Karthikesh. Imposing sparsity on the mixing matrix in independent component analysis. Neurocomputing, 49:151-162, 2002 (Special Issue on ICA and BSS).
Postscript  gzipped PostScript  pdf
[Shows how to implement the prior information on the sparsity of the mixing matrix in a very simple way as conjugate priors.]

A. Hyvärinen, J. Särelä and R. Vigário. Bumps and Spikes: Artifacts Generated by Independent Component Analysis with Insufficient Sample Size. In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA'99), pp. 425-429, Aussois, France, 1999.
gzipped PostScript   pdf
[Describes the phenomenon of overlearning in ICA.]

A. Hyvärinen and E. Bingham. Connection between multi-layer perceptrons and regression using independent component analysis. Neurocomputing, 50(C):211-222, 2003.
Postscript  gzipped PostScript  pdf
[Shows that MLP's can be interpreted as estimating an ICA model for the data, and doing regression using that model.]

J. Himberg and A. Hyvärinen. Independent component analysis for binary data: An experimental study . In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA2001), San Diego, California, 2001.
Postscript  gzipped PostScript  pdf
[Shows how to do ICA on binary data using ordinary FastICA.]

A. Hyvärinen and M. Inki. Estimating overcomplete independent component bases for image windows. Journal of Mathematical Imaging and Vision, 17:139-152, 2002.
Postscript  gzipped PostScript  pdf
[Discusses different methods for overcomplete basis estimation from images, and proposes two new, computationally efficient algorithms.]

A. Hyvärinen and E. Oja. Independent Component Analysis by General Non-linear Hebbian-like Learning Rules.  Signal Processing,  64(3):301-313, 1998.
Postscript  gzipped PostScript  pdf .
[Introduces one-unit adaptive algorithms related to FastICA. Shows how to estimate a coefficient that allows the estimation of both sub- and super-Gaussian independent component using a single nonlinearity.]