Probabilistic Models
Exam
Year | Semester | Date | Period | Language | In charge |
---|---|---|---|---|---|
2015 | spring | 13.01-26.02. | 3-3 | English | Sotiris Tasoulis |
Lectures
Time | Room | Lecturer | Date |
---|---|---|---|
Tue 16-18 | D122 | Sotiris Tasoulis | 13.01.2015-26.02.2015 |
Thu 16-18 | D122 | Sotiris Tasoulis | 13.01.2015-26.02.2015 |
Exercise groups
Time | Room | Instructor | Date | Observe |
---|---|---|---|---|
Wed 16-18 | B222 | Quan Nguyen | 19.01.2015—27.02.2015 |
General
IMPORTANT NOTICE FOR THE FINAL EXAM!
You may use one A4 sheet of handwritten notes.
You may use a calculator, but you may not use a cell phone, laptop, etc.
Please see the course syllabus.
The course learning objectives are available here.
A (tentative) detailed description of the lectures is available here.
A (tentative) set of objectives for each of the exercise sets is available here.
Enable the HTML5 player in YouTube here: https://www.youtube.com/html5
Literature and material
Materials for Class
Tuesday, January 13: Introduction to Probabilistic Models, Slides
Thursday, January 15: Refresher on Probability Theory, Slides
- A Tutorial on Probability Theory. Particularly Sections 1 - 7.
- (Optional) Combinations and Permutations: https://www.youtube.com/embed/C0p7kHOfMq8
- Introduction to Probability: http://www.youtube.com/embed/-8eSOmTPUbk
- Conditional Probability 1: http://www.youtube.com/embed/cwADSMeiIoE
- (Optional) Conditional Probability 2: http://www.youtube.com/embed/XYCCWrON7gQ
- Independent Events: http://www.youtube.com/embed/JV3_xdgC6yQ
- Bayes' Rule 1: http://www.youtube.com/embed/E2pOJwSwWDk
- (Optional) Bayes' Rule 2: http://www.youtube.com/embed/12fAdvRxqHI
- Joint Distributions: https://class.coursera.org/pgm/lecture/preview (Week 1, Distributions)
Note: Kevin deLaplante (http://www.youtube.com/user/PhilosophyFreak/videos) has many more introductory videos that may be helpful.
Tuesday, January 20: Bayesian Networks, Slides, Handout
- David Barber: Bayesian Reasoning and Machine Learning. Sections 3.1, 3.2, 3.3 (You will have to follow links to download the pdf.)
- (Optional) Graphical Models. Particularly the first 40 slides.
- (Optional) Independence and Conditional Independence: https://class.coursera.org/pgm/lecture/preview (Week 1, Conditional Independence)
- (Optional, but may be necessary to understand the next video) Factors: https://class.coursera.org/pgm/lecture/preview (Week 1, Factors)
- Introduction to Bayesian Networks: https://class.coursera.org/pgm/lecture/preview (Week 1, Semantics and Factorization)
- Intuitions of Reasoning with Bayesian Networks: https://class.coursera.org/pgm/lecture/preview (Week 1, Reasoning Patterns)
- d-separation 1: https://class.coursera.org/pgm/lecture/preview (Week 1, Flow of Probabilistic Influence)
- d-separation 2: https://class.coursera.org/pgm/lecture/preview (Week 1, Independencies in Bayesian Networks)
Thursday, January 22: Special Models (Naive Bayes), Slides
- Bayesian Classifiers: http://youtube.com/embed/ivBSZZyaRHY (You can ignore the bit about continuous variables at the end.)
- Naive Bayes Classifier: http://youtube.com/embed/54wfthrhwLQ (Again, you can ignore the part about continuous variables.)
Tuesday, January 27 - Thursday, January 29 : Special Models (Hidden Markov Models), Slides
- Hidden Markov Models, Introduction: http://youtube.com/embed/TPRoLreU9lA
- Hidden Markov Models, Detailed Example: http://youtube.com/embed/M_IIW0VYMEA
- Forward-Backward Algorithm, Introduction: http://youtube.com/embed/7zDARfKVm7s
- Forward Algorithm: http://youtube.com/embed/M7afek1nEKM
- (Optional) Forward Algorithm, Detailed Discussion: http://youtube.com/embed/MPmrFu4jFk4
- Backward Algorithm: http://youtube.com/embed/jwYuki9GgJo
- (Optional) Template Models: https://class.coursera.org/pgm/lecture/preview (Week 1, Overview of Template Models)
- (Optional) Hidden Markov Models: https://class.coursera.org/pgm/lecture/preview (Week 1, Temporal Models - HMMs)
- Bishop: Pattern Recognition and Machine Learning, Chapter 8. Sections 8.4.1 - 8.4.4
- (Optional) David Barber: Bayesian Reasoning and Machine Learning. Section 5.1 (You will have to follow links to download the pdf if you have not already.)
- Factors: https://class.coursera.org/pgm/lecture/preview (Week 1, Factors) (You may have already watched this one)
- Message Passing: https://class.coursera.org/pgm/lecture/preview (Week 3, Belief Propagation)
- Cluster Graphs: https://class.coursera.org/pgm/lecture/preview (Week 3, Properties of Cluster Graphs)
- (Optional) Belief Propagation in Trees: https://class.coursera.org/pgm/lecture/preview (Week 3, Clique Tree Algorithm - Computation)
- (Optional) Cluster Trees
- (Optional) HMM Algorithms as Sum-Product
Tuesday, February 10: Inference in General Bayesian Networks (Jointree Algorithm, also called the junction tree algorithm), Slides
Thursday, February 12: Multinomial Parameter Estimation, Slides
- Maximum likelihood estimates: https://class.coursera.org/pgm/lecture/preview (Week 7, Maximum Likelihood Estimation)
- Maximum likelihood estimates for Bayesian networks: https://class.coursera.org/pgm/lecture/preview (Week 7, Maximum Likelihood Estimation for Bayesian Networks)
- Bayesian estimates: https://class.coursera.org/pgm/lecture/preview (Week 7, Bayesian Estimation)
- Bayesian estimates for Bayesian networks: https://class.coursera.org/pgm/lecture/preview (Week 7, Bayesian Estimation for Bayesian Networks)
Tuesday, February 17: Learning with incomplete data, Slides
- Learning with incomplete data https://class.coursera.org/pgm/lecture/preview (Week 9, Learning with incomplete data )
- Expectation Maximization https://class.coursera.org/pgm/lecture/preview (Week 9, Expectation Maximization)
Thursday, February 19 - Tuesday, February 24: Scoring Functions for Structure Learning, Lecture Notes, Slides
- Likelihood scores: https://class.coursera.org/pgm/lecture/preview (Week 8, Likelihood Scores)
- Bayesian scores: https://class.coursera.org/pgm/lecture/preview (Week 8, Bayesian Scores)
- Optimal structure learning (Section 2)
- (Optional) BIC and asymptotic consistency: https://class.coursera.org/pgm/lecture/preview (Week 8, BIC and Asymptotic Consistency)
Lab Exercises
Exercise grades
Wednesday, January 21: Foundations of Probability
- Exercise set 1 (not graded)
Wednesday, January 28: Bayesian Networks
Wednesday, February 4: Naive Bayes Classifiers and Hidden Markov Models
Wednesday, February 11: Inference by Factors Elimination
LaTex References
The Wikibook on LaTex is quite thorough and well organized. In particular, the Mathematics section may be helpful.
Stack Overflow and the relevant section in Stack Exchange are also good resources, although it is often easiest to search using Google.
Additional Material
- Material of the year 2014 course
- Material of the year 2013 course
- Material of the year 2012 course
- Material of the year 2011 course
- Material of the year 2010 course.
- David Barber: Bayesian Reasoning and Machine Learning, parts I and II.
- Richard E. Neapolitan: Learning Bayesian Networks.
- Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach. Prentice Hall, 2003. Chapters 13, 14, (15, optional), 20.1-20.3
- Judea Pearl: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988.
- Daphne Koller and Nir Friedman: Probabilistic graphical models. MIT Press, 2009.
- Daphne Koller's online course on Probabilstic Graphical Models (recorded lectures can be found here).
- Myllymäki & Tirri, Bayes-verkkojen mahdollisuudet.
- B-Course: Bayesian software and on-line tutorial.
- Bishop: Pattern Recognition and Machine Learning, Chapter 8.
- Loredo, From Laplace to Supernova SN 1987A: Bayesian Inference in Astrophysics.
- Mackay, Information Theory, Inference, and Learning Algorithms: Chapter IV
- Heckerman, A Tutorial on Learning Bayesian Networks.
- Buntine, Operations for Learning with Graphical Models.
- Jeffrey, Probabilistic thinking
- Jaynes, Probability Theory: The Logic of Science.
- Howson & Urbach: Bayesian reasoning in science