Aapo Hyvärinen: Publications

Publications by topic

Publications front page

Home page

Independent component analysis: misc.

Reviews on ICA

A Hyvärinen. Independent Component Analysis: Recent Advances. Philosophical Transactions of the Royal Society A, 371:20110534, 2013.
Open Access Article
[An update on what has happened in ICA during the 10 years or so up to 2013.]

A. Hyvärinen and E. Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks, 13(4-5):411-430, 2000.
html (and Japanese version)  Postscript  gzipped PostScript  pdf
[A tutorial text on ICA in general, and FastICA in particular.]

A. Hyvärinen, J. Karhunen and E. Oja. Independent Component Analysis.
[The main reference book on ICA]

A. Hyvärinen. Survey on Independent Component Analysis. Neural Computing Surveys 2:94--128, 1999.
html  Postscript   gzipped PostScript   pdf
[A more detailed review on ICA estimation methods.]

Validation and testing

A. Hyvärinen. Testing the ICA mixing matrix based on inter-subject or inter-session consistency. NeuroImage, 58:122-136, 2011.
pdf  Matlab code
[A method for assigning a statistical significance (p-value) to each independent component, based on whether an independent component with the same mixing coefficients was found in different data sets. The datasets can be from different subjects in brain imaging, or just different parts of the same larger data set.]

A. Hyvärinen and P. Ramkumar. Testing independent component patterns by inter-subject or inter-session consistency, Frontiers in Human Neuroscience, 7:94, 2013.
Open Access article
[Extends the theory of the preceding paper to testing the values of the independent components themselves.]

J. Himberg, A. Hyvärinen and F. Esposito. Validating the independent components of neuroimaging time-series via clustering and visualization. NeuroImage 22(3):1214-1222, 2004.
pdf    software
[A method for analyzing the reliability and stability of estimated independent components by re-running the algorithm many times and visualizing the relationships of the obtained component. Features an easy-to-use software package for Matlab.]

S. Shimizu, A. Hyvärinen, Y. Kano, P. O. Hoyer and A. J. Kerminen. Testing significance of mixing and demixing coefficients in ICA. Proc. International Symposium on Independent Component Analysis and Blind Signal Separation (ICA2006), Charleston, SC, USA, 2006.
pdf matlab code
[Proposes tests to see whether coefficients in the ICA matrices are significantly different from zero or not.]

Dependent components

H. Sasaki, M. U. Gutmann, H. Shouno and A. Hyvärinen. Correlated Topographic Analysis: Estimating an Ordering of Correlated Components. Machine Learning, 92:285-317, 2013.
pdf  
[A new principle for topographic formation, and a new model of depdendencies of linear components, featuring linear correlation instead of energy correlations.]

A. Hyvärinen and S. Shimizu. A quasi-stochastic gradient algorithm for variance-dependent component analysis. In Proc. International Conference on Artificial Neural Networks (ICANN2006), Athens, Greece, pp. 211-220, 2006.
pdf
[Proposes a new algorithm to blindly separate sources which are dependent due to correlations in the variances, i.e. general activity levels.]

A. Hyvärinen, P.O. Hoyer and M. Inki. Topographic Independent Component Analysis. Neural Computation, 13(7):1527-1558, 2001.
Postscript  gzipped PostScript   pdf
[Introduces an extension of ICA. The dependencies of the estimated "independent" components are visualized as a topographic order. A new principle for topographic organization, based on higher-order statistics. Applied on image data, both topography and complex cell properties emerge, see section on the visual cortex models.]

[Many more papers on this topic can be found in the sections on sparse coding in the visual cortex and one also here and here

Nonlinear ICA

A. Hyvärinen and P. Pajunen. Nonlinear Independent Component Analysis: Existence and Uniqueness results. Neural Networks 12(3): 429--439, 1999.
Postscript  gzipped PostScript  pdf
[Shows that the solution of the nonlinear ICA problem is highly non-unique, and proposes a restriction of the model that does make the solution unique.]

P. Pajunen, A. Hyvärinen and J. Karhunen. Non-Linear Blind Source Separation by Self-Organizing Maps. In Proc. Int. Conf. on Neural Information Processing, Hong Kong, pp. 1207-1210, 1996.
Postscript  gzipped PostScript
[Describes a simple if limited method for doing nonlinear (nonparametric) ICA for subgaussian signals.]

Noisy data and denoising

A. Hyvärinen. Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation. Neural Computation, 11(7):1739--1768, 1999.
Postscript  gzipped PostScript  pdf
[Describes sparse code shrinkage, which is a new method for denoising of images etc. It is a kind of a combination of independent component analysis and wavelet shrinkage ideas.]

A. Hyvärinen, P. Hoyer and E. Oja. Image Denoising by Sparse Code Shrinkage. In S. Haykin and B. Kosko (eds), Intelligent Signal Processing, IEEE Press, 2001.
Postscript  gzipped PostScript  pdf
[Describes sparse code shrinkage (see preceding paper) in more detail.]

A. Hyvärinen. Independent Component Analysis in the Presence of Gaussian Noise by Maximizing Joint Likelihood. Neurocomputing, 22:49-67, 1998.
Postscript  gzipped PostScript. pdf.
[Work on noisy independent component analysis and its connections to competitive learning (clustering).]

(See also the FastICA section for a method based on "Gaussian Moments" for noisy ICA)

Blind separation of sources with temporal structure

K. Zhang and A. Hyvärinen. A general linear non-Gaussian state-space model: Identifiability, identification, and applications.Proc. Asian Conf. on Machine Learning (ACML2011), JMLR~W&CP, Taoyuan, Taiwan.
pdf
[A non-Gaussian state-space model which can separate even linearly correlated signals.]

A. Hyvärinen, P. Ramkumar, L. Parkkonen, and R. Hari. Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis. NeuroImage, 49(1):257-271, 2010.
pdf
[Proposes a source separation method tailor-made to EEG and MEG signals. Basically, preprocess the data by short-time Fourier transforms and then do ICA. Shows that this takes temporal correlations into account, and combines them with non-Gaussianity.]

A. Hyvärinen. A unifying model for blind separation of independent sources. Signal Processing, 85(7):1419-1427, 2005.
pdf   Postscript   Matlab code
[A framework and algorithm that unifies the three commonly used separation principles: nongaussianity, nonstationary variances, and second-order autocorrelations.]

A. Hyvärinen and J. Hurri. Blind separation of sources that have spatiotemporal dependencies. Signal Processing, 84(2):247-254, 2004 (special issue on nonlinear and non-independent source separation).
Postscript  gzipped PostScript  pdf
[Introduces double-blind source separation, which means separation of sources that are not independent, without a parametric model of the dependencies. Uses a cumulant-based criterion similar to our temporal coherence models of natural image sequences.]

A. Hyvärinen. Complexity Pursuit: Separating interesting components from time-series. Neural Computation, 13(4):883--898, 2001.
Postscript  gzipped PostScript  pdf
[Introduces the concept of complexity pursuit, which means finding projections of time series (signals) that have minimum complexity. Also introduces simple approximations of complexity that take into account both nongaussianity and autocorrelations.]

A. Hyvärinen. Blind source separation by nonstationarity of variance: A cumulant-based approach. IEEE Trans. on Neural Networks, 12(6):1471-1474, 2001.
Postscript  gzipped PostScript   pdf
[Formulates the less-known separation criterion on variance nonstationarity using cumulants, and proposes a fast fixed-point algorithm.]

A. Hyvärinen. Independent Component Analysis for Time-dependent Stochastic Processes.  In Proc. Int. Conf. on Artificial Neural Networks (ICANN'98), Skövde, Sweden, pp. 541-546, 1998.
Postscript   gzipped PostScript   pdf
[This paper shows that in ICA, it is often useful to preprocess the data by computing the innovation processes.]

Other theoretical topics

J. Puuronen and A. Hyvärinen. Hermite Polynomials and Measures of Non-Gaussianity . In Proc. International Conference on Artificial Neural Networks (ICANN2011), Helsinki, Finland, 2011.
pdf
[Proposes a general and statistically robust approximation of non-Gaussianity.]

A. Hyvärinen and R. Karthikesh. Imposing sparsity on the mixing matrix in independent component analysis. Neurocomputing, 49:151-162, 2002 (Special Issue on ICA and BSS).
Postscript  gzipped PostScript  pdf
[Shows how to implement the prior information on the sparsity of the mixing matrix in a very simple way as conjugate priors.]

A. Hyvärinen, J. Särelä and R. Vigário. Bumps and Spikes: Artifacts Generated by Independent Component Analysis with Insufficient Sample Size. In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA'99), pp. 425-429, Aussois, France, 1999.
gzipped PostScript   pdf
[Describes the phenomenon of overlearning in ICA.]

A. Hyvärinen and E. Bingham. Connection between multi-layer perceptrons and regression using independent component analysis. Neurocomputing, 50(C):211-222, 2003.
Postscript  gzipped PostScript  pdf
[Shows that MLP's can be interpreted as estimating an ICA model for the data, and doing regression using that model.]

J. Himberg and A. Hyvärinen. Independent component analysis for binary data: An experimental study . In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA2001), San Diego, California, 2001.
Postscript  gzipped PostScript  pdf
[Shows how to do ICA on binary data using ordinary FastICA.]

A. Hyvärinen and M. Inki. Estimating overcomplete independent component bases for image windows. Journal of Mathematical Imaging and Vision, 17:139-152, 2002.
Postscript  gzipped PostScript  pdf
[Discusses different methods for overcomplete basis estimation from images, and proposes two new, computationally efficient algorithms.]

A. Hyvärinen and E. Oja. Independent Component Analysis by General Non-linear Hebbian-like Learning Rules.  Signal Processing,  64(3):301-313, 1998.
Postscript  gzipped PostScript  pdf .
[Introduces one-unit adaptive algorithms related to FastICA. Shows how to estimate a coefficient that allows the estimation of both sub- and super-Gaussian independent component using a single nonlinearity.]

A. Hyvärinen. A unified probabilistic model for independent and principal component analysis.  In Advances in Independent Component Analysis and Learning Machines (Festschrift to Erkki Oja), Academic Press, 2015.
pdf 
[Proposes a single probabilistic model which can do either PCA or ICA, depending on the data and its preprocessing.]

Applications

T. Honkela, A. Hyvärinen, and J. Väyrynen WordICA - Emergence of Feature Representations for Words by Independent Component Analysis. Natural Language Engineering, 16(3):277-308, 2010.
pdf
[Shows how interesting semantic and syntactic categories (attributes) emerge when ICA is applied on descriptions of the contexts of words.]

J. Perkiö and A. Hyvärinen. Modelling image complexity by independent component analysis, with application to content-based image retrieval. Proc. Int. Conf. on Artificial Neural Networks (ICANN2009), Limassol, Cyprus, 2009.
pdf
[Proposes a simple approximator of image complexity based on ICA, and uses it for CBIR.]

ICA and inverse modelling

J. Puuronen and A. Hyvärinen. A Bayesian Inverse Solution using ICA. Neural Networks, 50:47-59, 2014.
pdf
[Combines inverse modelling, as found in EEG and MEG for example, with the ICA model.]