Department of Computer Science

This is a new course belonging to the new Algorithms and Machine Learning sub-pogramme in the Master's programme of the department, and together with 582637 Project in probability models (2 cr), it forms one of the three optional courses of the sub-programme.

This course provides an introduction to probabilistic modeling with emphasis on graphical models and their applications in Artificial Intelligence, Computational Intelligence and Data Mining. The first part of the course introduces basic concepts of graphical models and Bayesian inference. Topics will include Markov models, Markov random fields, and simple Bayesian networks, with illustrating examples from machine learning or data mining. The second part of the course introduces further theoretical and algorithmic topics on graphical models. The focus will be on Bayesian networks with discrete variables, and the topics will include conditional independence and Markov properties, efficient inference algorithms, and their connection with graph theory.

There will be exercise groups starting from the second week of the class. Exercise times are Fri 15-17.

The course is an introductory course, and only elementary knowledge of probability theory is required. Different parts of the course, however, have different requirements with respect to the mathematical machinery needed to apply the concepts in question. Typically some analysis and elementary mathematical statistics is required. We assume that the participants are familiar with topics covered in the courses 582630 Design and analysis of algorithms (4 cr) and 582631 Introduction to machine learning (4 cr).

The primary material will be lecture slides and chapters from certain books. Lecture slides in pdf are given below, with the 4-slides-per-page versions for printing given inside the parentheses. (You may also check out the materials of the previous Three concepts: probability course in 2009, 2008 and 2007)

• Lecture 1. (4-on-1 version)
• Lecture 2. (4-on-1 version)
• Lecture 3. (4-on-1 version)
• Lecture 4. (4-on-1 version)
• Lecture 5. (4-on-1 version)
• Lecture 6. (4-on-1 version)
• Lecture 7. (4-on-1 version)
• Lecture 8. (4-on-1 version)
• Lecture 9. (4-on-1 version)
• Lecture 10. (4-on-1 version)
• Lecture 11. (4-on-1 version) (updated on Feb 25)
• Lecture 12. (4-on-1 version)

 

Additional recommended reading materials (for subjects discussed in the classes or to be discussed soon; more will be added):

• Alan Hàjek. Interpretations of Probability, from Stanford Encyclopedia of Philosophy.
• Frank P. Ramsey. Truth and Probability, 1926.
• Judea Pearl. Probabilistic Reasoning in Intelligent Systems, Morgan Kaufmann, 1988. Chap. 1; Chap. 4, 5.
• Robert G. Cowell, A. Philip Dawid, Steffen L. Lauritzen, and David J. Spiegelhalter. Probabilistic Networks and Expert Systems: Exact Computational Methods for Bayesian Networks, Springer, 2007. Chap. 2; Chap. 5.1, 5.2, 5.3; Chap. 6.
• A. C. Davison. Statistical Models, Cambridge Univ. Press, 2003. Chap. 6.1, 6.2; Chap. 4.1, 4.7.
• Finn V. Jensen. An Introduction to Bayesian Networks. UCL Press, 1996. Chap. 3.; Chap. 4.
• A. Philip Dawid. Conditional Independence, in Encyclopedia of Statistical Sciences, pp. 146-55, Wiley-Interscience, 1998.