Three Concepts: Information, Course at the Department of Computer ScienceUniversity of Helsinki, Fall 2006.

Three Concepts: Information

This course belongs to the "Three concepts" series, and provides an introduction to information and coding theory for computer science students. In fact much of the course can be viewed as applications of Shannon's central result known as the source coding theorem. The theoretical results will be illustrated by various descriptions of practical data compression systems from Huffman coding to Rissanen's arithmetic coding. In order to demonstrate the wide applicability of information-theoretic concepts, the role of Bayesian inference in data compression is discussed, and we end the course by describing application of information-theoretic principles to (statistical) modeling, i.e., the Minimum Description Length Principle (MDL).

Instructor: Professor Petri Myllymäki, Head of the Intelligent Systems Specialization Area and the Complex Systems Computation (CoSCo) research group
Course assistant: M.Sc. Tomi Silander
Language: Although all the material in the course will be in English, the lectures will be given this year in Finnish.
Lectures: 07.09.-12.10. Thursdays 16-19 in B222.
Posters and projects: In addition to regular lectures, the course involves project work and poster presentations (see the schedule below).

These pages will be updated during the course and the current schedule and topics are only tentative. Follow this page for updates!

Course description
Term paper
Results (Intranet)

Course Schedule (tentative):

Thu. 07.09. Ei opetusta, kurssi alkaa vasta 14. päivä.
Thu. 14.09., 16-19, B222 Järjestäytymistilaisuus, kurssin esittely.
Lecture: ''What is Information Theory?''
Thu. 21.09., 16-19, B222 Lecture: ''How much can we compress? - Shannon's Source Coding Theorem''
Thu. 28.09., 16-19, B222 Lecture: ''Revenge of a student: Symbol codes''
Projektien I ja II esittely, ryhmien muodostaminen
Thu. 05.10., 16-19, B222 Lecture: ''On Minimum Description Length Modeling''
Thu. 12.10., 16-19, B222 Lecture: ''Year 2020 - Topics in Information Theory for Further Studies''
Thu. 19.10. No session.
Thu. 26.10. No session.
Wed. 01.11., 12:00 Project I deadline.
Thu. 02.11., 16-17, C221Project II Q&A
Wed 08.11., 12:00 Project II, 1st deadline.
Thu. 09.11., 16-18, B222Review of project results.
''The great Lempel-Ziv vs. Arithmetic coding debate'' (Participation obligatory.)
Wed. 15.11., 12:00 Project II, 2nd deadline.
Thu. 16.11., 16-17, C221Review of project results.
Wed. 22.11., 12:00 Project II, 3rd deadline.
Thu. 23.11., 16-19, B222Review of project results.
Poster session (Participation obligatory.)
Wed. 29.11., 12:00 Project II, final deadline.
Thu. 30.11., 16-18, B222 Final review of project results: presentations from each of the project teams.
Thu. 07.12., 08:00 Deadline of the poster reports and Project II reports.
Fri. 15.12., 12:00 Term paper deadline.


 3 Concepts: Information
Last updated August 25, 2006