Unsupervised Machine Learning

Exam

Year Semester Date Period Language In charge

Lectures

Time Room Lecturer Date
Tue 14-16 C222 Aapo Hyvärinen 15.01.2013-22.02.2013
Thu 14-16 C222 Aapo Hyvärinen 15.01.2013-22.02.2013
Fri 14-16 C222 Aapo Hyvärinen 15.01.2013-22.02.2013

Information for international students

The course will be completely in English.

General

 

Please note that the course is in Period III this year.

 

Target audience

Master's students in computer science (specialization in algorithms & machine learning, or bioinformatics), applied mathematics (specialization statistical machine learning or e.g. stochastics), or statistics.

Description

Unsupervised learning is one of the main streams of machine learning, and closely related to exploratory data analysis and data mining. This course describes some of the main methods in unsupervised learning.

In recent years, machine learning has become heavily dependent on statistical theory which is why this course is somewhere on the borderline between statistics and computer science. Emphasis is put both on the statistical formulation of the methods as well as on their computational implementation. The goal is not only to introduce the methods on a theoretical level but also to show how they can be implemented in scientific computing environments such as Matlab or R.

Computer projects are an important part of the course, but they are given separate credits, see Projects for Unsupervised Machine Learning. The projects will be given in the exercice session marked below in the schedule.

One of the weekly sessions (usually Friday) will be an exercice session, the timetable will be as follows: 

Tue 15 Jan Lecture * Thu 17 Jan Lecture * Fri 18 Jan

Exercices

Intro to computer assignments

Tue 22 Jan Lecture * Thu 24 Jan Lecture * Fri 25 Jan Exercices
Tue 29 Jan Lecture * Thu 31 Jan Lecture * Fri 1 Feb Exercices
Tue 5 Feb Lecture * Thu 7 Feb Lecture * Fri 8 Feb Exercices
Tue 12 Feb Lecture * Thu 15 Feb Lecture * Fri 15 Feb Exercices
Tue 19 Feb Lecture * Thu 21 Feb Lecture * Fri 22 Feb Exercices

The exercices will be taught by Jouni Puuronen and the computer projects by Jukka-Pekka Kauppi.

Prerequisites

  • Statistics majors: Bachelor's degree recommended.
  • Mathematics majors: Bachelor's degree recommended. It should include basic courses in calculus (including vector calculus), linear algebra I&II, introduction to probability, introduction to statistical inference.
  • Computer science majors: Bachelor's degree recommended. It should include the following mathematics courses: calculus (including vector calculus), linear algebra I&II, introduction to probability. You should also have done "Introduction to machine learning" in Period II.

Contents:

  • Introduction. Supervised vs. unsupervised learning. Applications of unsupervised learning. Probabilistic formulation. Review of some basic mathematics (linear algebra, probability)
  • Numerical optimization. Gradient method, Newton's method, stochastic gradient, projected gradient methods
  • Principal component analysis and factor analysis. Formulation as minimization of reconstruction error or maximization of component variance. Computation using covariance matrix and its eigen-value decomposition. Factor analysis and interpretation of PCA as estimation of gaussian generative model. Factor rotations.
  • Independent component analysis. Problem of blind source separation, why non-gaussianity is needed for identifiability. Correlation vs. independence. ICA as maximization of non-gaussianity, measurement of non-Gaussianity by cumulants. Likelihood of the model and maximum likelihood estimation. Implementation by gradient methods and FastICA. Applications of component analysis.
  • Clustering. K-means algorithm.Gaussian mixture model: Maximization of likelihood, EM algorithm.
  • Nonlinear dimension reduction. Non-metric multi-dimensional scaling and related methods: kernel PCA, Laplacian eigenmaps, IsoMap. Kohonen's self-organizing map

Completing the course

There will be a single exam at the end of the course. Check the exact timetable and place on the CS dept exam page.

Active participation in the exercise sessions will give you points for the exam. See here for details.

 

Literature and material

Here are the complete lecture notes for this year's course. Just to keep search engines away, you need the login uml and password uml. There is no book for the course.

Here are the exercices to be considered in the Friday sessions.