3 Concepts: Information, Course at the Department of Computer ScienceUniversity of Helsinki, Spring 2005.

3 Concepts: Information

This course belongs to the "Three concepts" series, and provides an introduction to information and coding theory for computer science students. In fact much of the course can be viewed as applications of Shannon's central result known as the source coding theorem. The theoretical results will be illustrated by various descriptions of practical data compression systems from Huffman coding to Rissanen's arithmetic coding. In order to demonstrate the wide applicability of information-theoretic concepts, the role of Bayesian inference in data compression is discussed, and we end the course by describing application of information-theoretic principles to (statistical) modeling, i.e., the Minimum Description Length Principle (MDL).

Instructor: Professor Petri Myllymäki, Head of the Intelligent Systems Specialization Area and the Complex Systems Computation (CoSCo) research group
Course assistant: M.Sc. Tomi Päiväniemi
Language: Although all the material in the course will be in English, the lectures will be given this year in Finnish.
Lectures: 20.01.-10.03. Thursdays 16-19 in B222.
Posters and projects: In addition to regular lectures, the course involves project work and poster presentations. A special session will be organized for presenting the results of the projects and the posters. The time of the session will be announced later, it will take place after the final set of lectures.

These pages will be updated during the course and the current schedule and topics are only tentative. Follow this page for updates!

Course description
Term paper

Course Schedule (tentative):

Thu. 20.01., 16-19 Järjestäytymistilaisuus, kurssin esittely.
Lecture: ''What is Information Theory?''
Thu. 27.01., 16-19Projektin I esittely, ryhmien muodostaminen
Lecture: ''How much can we compress? - Shannon's Source Coding Theorem''
Thu. 03.02., 16-19 Projektin II esittely
Lecture: ''Revenge of a student: Symbol codes''
Mon. 14.02., 12:00 Project I deadline.
Thu. 17.02., 16-19 ''The great Lempel-Ziv vs. Arithmetic coding debate'' at HTC (Participation obligatory.)
Mon. 21.02., 12:00 Project II, 1st deadline.
Thu. 24.02., 16-19 Lecture: ''On Minimum Description Length Modeling''
Thu. 10.03., 16-19 Lecture: ''Year 2020 - Topics in Information Theory for Further Studies''
Mon. 07.03., 12:00 Project II, 2nd deadline.
Mon. 14.03., 12:00 Project II, 3rd deadline.
Thu. 17.03., 16-21 (!) Poster session at HTC (Participation obligatory.)
Thu. 24.03., 12:00 Project II, final deadline.
Mon. 04.04., 12:00 Term paper deadline.


 3 Concepts: Information
Last updated January 26, 2005