# Supervised Machine Learning

## Koe

Vuosi | Lukukausi | Päivämäärä | Periodi | Kieli | Vastuuhenkilö |
---|---|---|---|---|---|

2011 | kevät | 18.01-24.02. | 3-3 | Englanti | Jyrki Kivinen |

## Luennot

Aika | Huone | Luennoija | Päivämäärä |
---|---|---|---|

Ti 10-12 | C222 | Jyrki Kivinen | 18.01.2011-24.02.2011 |

To 10-12 | C222 | Jyrki Kivinen | 18.01.2011-24.02.2011 |

## Harjoitusryhmät

Aika | Huone | Ohjaaja | Päivämäärä | Huomioitavaa |
---|---|---|---|---|

To 14-16 | C222 | Panu Luosto | 24.01.2011—25.02.2011 |

## Yleistä

**The course has been graded.** See below for details.

## Prerequisites

The course Introduction to Machine Learning is not strictly required. However it includes a lot of very useful motivation, context and other background which we will here cover only very briefly.

The students are expected to have basic knowledge of linear algebra, probability theory and calculus. Some multivariate calculus will be needed, but we will briefly cover the necessary tools for those not familiar.

Although there are not many specific prerequisites from mathematics, the approach taken on the course is mainly mathematical, and familiarity with mathematical manipulations will help a lot.

Part of the homework will require some programming. Basic programming skills are essential, and familiarity with tools such as Matlab, Octave or R will be very helpful.

## Contents

The course covers a selection of topics mainly related to (binary) classification. This is a large research area, and the choice of topics is somewhat based on the personal preferences of the lecturer (whose research area is computational learning theory, in particular online learning).

Emphasis will be on provable performance guarantees we can provide for learning algorithms.

Table of contents (preliminary):

- Introduction
- basic concepts
- mathematical frameworks for supervised learning: online and statistical

- Online learning
- combining expert advice
- linear classification and the perceptron algorithm
- relative loss bounds (also known as regret bounds)

- Statistical learning
- basic statistical model, connection to online learning
- complexity measures: VC-dimension and Rademacher complexity
- Support Vector Machine

## Kurssin suorittaminen

The exam has now been graded. Sorry for the delay,

- Results of the course
- Exam problems and some solutions
- Exam scores for each problem

More detailed comments about the exam will appear here soon. For now, I'll just notice that the exam turned out to be more difficult than intended, which has been taken into account in grading, so you may have received more points than you expected.

The grade consists of homework (20%) and exam (80%).

Read also the course policies.

Please give also some feedback for the course. You may wait until after the exam if you wish, but please do not forget!

## Kirjallisuus ja materiaali

The material consists of lecture notes, exercise problems and their solutions, and possibly some additional original articles.

Lecture notes and homework material will appear here. The whole set of lecture notes (pages 1–201) is now available.

- A new page was added after page 103 to fix a mistake. The page numbers from 104 are now off by one compared to earlier versions.
- Some additional steps were added to clarify the proof on page 175.

Homework solutions need to be turned out to the course assistant Panu Luosto in advance, on paper. The deadline is Tuesday before the exercise session at 15:00. There will be an envelope at the door to Panu's office B233 for turning in your homework. If you have problems with the procedure, contact directly Panu or the lecturer Jyrki Kivinen.

- Homework 1 (due Tue 25 January at 15:00): problems, solutions
- Homework 2 (due Tue 1 February at 15:00): problems, solutions
- Homework 3 (due Tue 8 February at 15:00): problems, solutions
- Homework 4 (due Tue 15 February at 15:00): problems, solutions
- Homework 5 (due Tue 22 February at 15:00): problems (notice that there is a correction to problems 2 and 3), data, LIBSVM, solutions
- Extra homework (no credit, just for practice): problems, solutions

The course does not follow any single textbook. Some recommended textbooks and articles on the course topic are listed under the tab Additional references.