Unsupervised Machine Learning
|Tue 14-16||C222||Aapo Hyvärinen||13.03.2012-27.04.2012|
|Thu 14-16||C222||Aapo Hyvärinen||13.03.2012-27.04.2012|
|Fri 14-16||C222||Aapo Hyvärinen||13.03.2012-27.04.2012|
Information for international students
The course will be completely in English.
Master's students in computer science (specialization in algorithms & machine learning, or bioinformatics), applied mathematics (specialization statistical machine learning or e.g. stochastics), or statistics.
Unsupervised learning is one of the main streams of machine learning, and closely related to exploratory data analysis and data mining. This course describes some of the main methods in unsupervised learning.
In recent years, machine learning has become heavily dependent on statistical theory which is why this course is somewhere on the borderline between statistics and computer science. Emphasis is put both on the statistical formulation of the methods as well as on their computational implementation. The goal is not only to introduce the methods on a theoretical level but also to show how they can be implemented in scientific computing environments such as Matlab or R.
Computer projects are an important part of the course, but they are given separate credits, see Projects for Unsupervised Machine Learning. The projects will be given in the exercice session marked below in the schedule.
One of the weekly sessions (usually Friday) will be an exercice session, detailed timetable will be as follows:
|Tue 13 Mar||Lecture||*||Thu 15 Mar||Lecture||*||Fri 16 Mar||Lecture|
|Tue 20 Mar||Lecture||*||Thu 22 Mar||Lecture||*||Fri 23 Mar||
Intro to computer assignments
|Tue 27 Mar||Exercices||*||Thu 29 Mar||Lecture||*||Fri 30 Mar||Exercices|
|Tue 3 Apr||Lecture||*||Thu 5 Apr||Easter break||*||Fri 6 Apr||Easter break|
|Tue 10 Apr||Easter break||*||Thu 12 Apr||Lecture||*||Fri 13 Apr||Exercices|
|Tue 17 Apr||Lecture||*||Thu 19 Apr||Lecture||*||Fri 20 Apr||Exercices|
|Tue 24 Apr||Lecture
||*||Thu 26 Apr||Lecture||*||Fri 27 Apr||Exercices|
The exercices will be taught by Jouni Puuronen and the computer projects by Jukka-Pekka Kauppi.
- Statistics majors: Bachelor's degree recommended.
- Mathematics majors: Bachelor's degree recommended. It should include basic courses in analysis (including vector analysis), linear algebra I&II, introduction to probability, introduction to statistical inference. (Preferably also some more statistics courses.)
- Computer science majors: Bachelor's degree recommended. It should include the mathematics courses listed above for mathematic majors. Preferably you should also have done both the courses "Introduction to machine learning" and "Probabilistic models".
- supervised vs. unsupervised learning
- applications of unsupervised learning
- probabilistic formulation: generative models or latent variable models
- overview of the topics below
- Review of some basic mathematics (linear algebra, probability)
- Numerical optimization
- gradient method, Newton's method, stochastic gradient, alternating variables
- Principal component analysis and factor analysis
- formulation as minimization of reconstruction error or maximization of component variance
- computation using covariance matrix and its eigen-value decomposition
- factor analysis and interpretation of PCA as estimation of gaussian generative model
- factor rotations
- Independent component analysis
- problem of blind source separation, why non-gaussianity is needed for identifiability
- correlation vs. independence
- ICA as maximization of non-gaussianity, measurement of non-Gaussianity by cumulants
- likelihood of the model and maximum likelihood estimation
- implementation by gradient methods and FastICA
- k-means algorithm
- formulation as mixture of gaussians
- maximization of likelihood, EM algorithm
- Nonlinear dimension reduction
- non-metric multi-dimensional scaling and related methods: kernel PCA, Laplacian eigenmaps, IsoMap
- Kohonen's self-organizing map
Literature and material
Here are the complete lecture notes for this year's course. Just to keep search engines away, you need the login uml and password uml. There is no book for the course.
Here are the exercices considered in the sessions. Note that they are slightly modified from the ones in the lecture notes, and a subset of them.