Several authors (Spirtes et al. 1993; Pearl 2000) have recently formalized concepts related to causality using probability distributions defined on directed acyclic graphs. This line of research emphasizes the importance of understanding the process which generated the data, rather than only characterizing the joint distribution of the observed variables. The reasoning is that a causal understanding of the data is essential to be able to predict the consequences of interventions, such as setting a given variable to some specified value.
One of the main questions one can answer using this kind of theoretical framework is: 'Under what circumstances and in what way can one determine causal structure based on data which is not obtained by controlled experiments but by passive observation only?'. In many cases it is impossible or too expensive to perform controlled experiments, and hence methods for discovering likely causal relations from uncontrolled ('observational') data would be very valuable.
We have developed a method, abbreviated LiNGAM, for identifying Linear, Non-Gaussian, Acyclic causal Models based on purely observational, continuous-valued data. This method can be seen as an extension of the standard SEM (Structural Equation Model; see, for instance, Bollen 1989) framework. The key aspect of our method is that when the data is non-gaussian it is possible to identify more of the generating structure than is possible in the traditional Gaussian setting.
The basic method assumes that (a) there are no hidden confounders, and (b) all (or all but one) of the error terms are non-gaussian. Under these conditions it can be shown that the full generating model can be identified in the limit of an infinite sample. Furthermore, our estimation method based on ICA works well even for relatively small sample sizes.
We distribute a complete Matlab/Octave code package for performing this standard LiNGAM analysis. The code conforms to the method (and notation) as described in our JMLR paper (see below).
Please see the following papers for details on standard LiNGAM:
S. Shimizu, P.O. Hoyer, A. Hyvärinen, and A.J. Kerminen,
"A linear, non-gaussian acyclic model for causal discovery,"
Journal of Machine Learning Research, 7:2003-2030, 2006.
S. Shimizu, A. Hyvärinen, Y. Kano, and P.O. Hoyer,
"Discovery of non-gaussian linear causal models using ICA,"
Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI-2005),
pp. 526-533, 2005.
S. Shimizu, A. Hyvärinen, P.O. Hoyer, and Y. Kano,
"Finding a causal ordering via independent component analysis,"
Computational Statistics & Data Analysis, 50(11): 3278-3293, 2006.
lvLiNGAM
(Patrik O. Hoyer, Shohei Shimizu, Antti J. Kerminen, and Markus Palviainen)
Even when hidden confounding variables may be present, it is theoretically possible to identify a large part of the generating model based on non-gaussianity. In the code and papers below, we explore this possibility in more details. The main result is that although in theory much can be identified, in practice it is very difficult and very large sample sizes are needed to get good results even for small models.
We distribute a small Matlab code package for performing the basic lvLiNGAM experiments described in the PGM'06 paper (see below).
Please see the following papers for details on lvLiNGAM:
P.O. Hoyer, S. Shimizu, A.J. Kerminen, and M. Palviainen,
"Estimation of causal effects using linear
non-gaussian causal models with hidden variables,"
International Journal of Approximate Reasoning, in press.
P.O. Hoyer, S. Shimizu, and A.J. Kerminen,
"Estimation of linear, non-gaussian
causal models in the presence of confounding latent variables,"
Proceedings of the Third European Workshop on Probabilistic Graphical Models (PGM'06),
pp. 155-162, 2006.
LiNGAM with hidden classes
(Shohei Shimizu and Aapo Hyvärinen)
Extends the basic LiNGAM method to cases where latent classes are present. The new method finds hidden groups of samples that have similar DAG structures.
Please see the following paper for details:
S. Shimizu and A. Hyvärinen,
"Discovery of linear non-gaussian acyclic
models in the presence of latent classes,"
Proceedings of the 14th International Conference on Neural Information Processing (ICONIP-2007), in press.
Shohei Shimizu
The institute of Statistical Mathematics
Japan
Aapo Hyvärinen &
Patrik O. Hoyer
Helsinki Institute for Information Technology
Department of Computer Science
University of Helsinki
Finland