5811267-5 Elements of Uncertain Reasoning (4cu)
Course at the Department of Computer Science, University of Helsinki
Instructor: Henry Tirri, Complex Systems Computation Group
Contents
General information
Schedule
Online course material
Course projects
Uncertainty on the WEB
Student viewpoints
Results
General Information
Many of the research issues in Artificial Intelligence, Computational Intelligence and Data Mining can be actually viewed as topics in the "science of uncertainty," which addresses the problem of optimal processing of incomplete information. In many cases the various different approaches in neural networks, fuzzy logic, non-monotonic logics and Bayesian networks all address the same fundamental question: what is the appropriate language and inference mechanism for plausible inference, i.e., inference with incomplete information. Plausible inference techniques can then be used to implement such generic tasks as prediction, planning and decision making. This course aims at introducing many of the fundamental concepts, methods and techniques used in plausible inference. The course is divided into several modules each addressing topics found in many different fields, including mathematical statistics, information theory and computer science.
Who should attend?
Anyone interested in building models with incomplete information: if you are interested in such topics as machine learning, neural networks, Bayesian inference, data mining or data compression, the course might give you new ideas, viewpoint or tools for your problems.
What prerequisites are required?
The course is an introductory course, and no previous knowledge of the topics is assumed. Different modules of the course, however, have different requirements with respect to the mathematical machinery needed to apply the concepts in question. Typically only elementary analysis and probability theory are needed. Some of the projects also involve programming.
What is needed to get the credit (4cu) for the course?
In addition to the classes, the students are expected to
finish several projects during the course (40% of the grade)
prepare a poster presentation for the joint poster session at the end of the course (co-located with the poster session of the Quantum Computing seminar) (30% of the grade)
pass a "home exam" at the end of the course (30% of the grade)
Participation to the classes is not enforced, but strongly encouraged, as many of the project issues will be discussed during the classes also.
Back to top
Schedule
Class hours: Monday 10-12 Room A414, Wednesday 10-12 Room 320
Topics discussed at the course:
Intuitions behind Bayesian modeling
Elements of Bayesian inference
Bayesian networks and their construction from data
Bayesian Neural networks
Minimum encoding modeling
Applications of Bayesian modeling
Back to top
Online course material
HTML-versions of the latest class notes 
Postscript versions of the class notes
Home exam (Deadline 22.12.97 9.00AM)
Literature
Back to top
Postscript versions of the Class notes
Obs! It seems that in many cases ghostview has a problem of showing the transparencies, but when down-loaded, they print properly.
MI: Bayesian modeling, inference and decisions
MI.1. Appetizer
MI.2. Bayesian inference- intuitive description
Some additional references:
D.V.Lindley, Making Decisions. 2nd Edition, John Wiley & Sons, 1985.
T.J.Loredo. From Laplace to supernova SN 1987A: Bayesian inference in astrophysics. In P.Fougere, editor, Maximum Entropy and Bayesian Methods. Kluwer Academic Publishers, 1990, 81-142.
E.T.Jaynes, Papers on probability, statistics and statistical physics. Edited by R.D.Rosenkrantz. D. Reidel Publishing Company, 1983.
MII: Elements of Bayesian inference
MII.1. Concepts for Bayesian inference
MII.2. Models for proportions
Material to read for this topic:
Chapters 1 and 2 in: A.Gelman, J. Carlin, H. Stern, and D. Rubin. Bayesian Data Analysis. Chapman & Hall, 1995.
Chapters 6,7 and 8 in: D.Berry. Statistics. A Bayesian perspective. Duxbury Press, 1996.
MII.3. The General Case
Additional material:
Chapters 3 and 5 in: A.Gelman, J. Carlin, H. Stern, and D. Rubin. Bayesian Data Analysis. Chapman & Hall, 1995.
MIII: Bayesian networks
MIII.1. Introduction to Bayesian networks
Additional material:
F.Jensen, An Introduction to Bayesian Networks (ISBN 0-387-91502-8, UCL Press/Springer-Verlag, NY 1996).
Thomas Richardson course notes from the graduate course "An Introduction to Bayesian Networks" (Spring 1997).
MIII.2. Learning Bayesian Networks
Additional material:
D.Heckerman, D.Geiger and D.Chickering, Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20(3):197-243, 1995.
D.Heckerman, Bayesian networks for data mining. Data Mining and Knowledge Discovery 1(1), 79-119, 1997.
MIV: Assessing the performance of a model construction algorithm
MIV. Assessing (classification) model's predictive performance
Additional material:
P.Cohen, Empirical Methods for Artificial Intelligence.The MIT Press, 1995.
MV: Minimum encoding modeling
MV.1. Intuitive description
MV.2. Coding and information
MV.3. Stochastic complexity
Additional material:
J. Rissanen. Stochastic Complexity in Statistical Inquiry. World Scientific Publishing Company, New Jersey, 1989.
T. Cover and J.A. Thomas, Elements of Information Theory. John Wiley & Sons, 1991.
Back to top
HTML-versions of some of the latest class notes
MIV: Assessing the performance of a model construction algorithm
MIV. Assessing (classification) model's predictive performance
MV: Minimum encoding modeling
MV.1. Intuitive description
MV.2. Coding and information
MV.3. Stochastic complexity
Back to top
Literature
Some good general information sources for various parts of the course:
J.O. Berger. Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York, 1985.
J.M. Bernardo and A.F.M Smith. Bayesian Theory. John Wiley, 1994.
C.M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995.
P. Cheeseman. Probabilistic versus fuzzy reasoning. In L.N. Kanal and J.F. Lemmer, editors, Uncertainty in Artificial Intelligence 1, pages 85--102. Elsevier Science Publishers B.V. (North-Holland), Amsterdam, 1986.
T. Cover and J.A. Thomas, Elements of Information Theory. John Wiley & Sons, 1991.
A.Gelman, J. Carlin, H. Stern, and D. Rubin. Bayesian Data Analysis. Chapman & Hall, 1995.
W. R. Gilks, S. Richardson, and Spiegelhalter, D.J. Markov chain Monte Carlo in practice. Chapman & Hall, London, GB, 1996.
D. Mackay. Bayesian Methods for Adaptive Models. PhD thesis, California Institute of Technology, 1992.
J. Rissanen. Stochastic Complexity in Statistical Inquiry. World Scientific Publishing Company, New Jersey, 1989.
D.M. Titterington, A.F.M. Smith, and U.E. Makov. Statistical Analysis of Finite Mixture Distributions. John Wiley & Sons, New York, 1985.
Back to top
Uncertainty on the WEB
Association for Uncertainty in Artificial Intelligence
Data Understanding Group
Microsoft Research
Institute of Statistics & Decision Sciences
Russ Almond's Bayesian Belief Net Page
Air Force Institute of Technology Bayesian Network Page
Network Demos at the NRC Institute for Information Technology (Ottawa)
Back to top
"Student viewpoints"
Question: "In your own opinion, give the three most important issues in Bayesian modeling"
Selected answers after the "Bayesian inference" introductory section:
Allows the calculation of model probabilities
One needs an initial view of the model
Examples modify the model to reflect the reality (or data) better
Well-suited for computers
Close to human intuition
Observed data affects the view of the world
Finding model with data
Allows inference with incomplete information
Self-improving estimations (with data)
Subjectivity
Belief-update rule
Model averaging
Inference with very little data
Missing values do not harm the analysis
No particular model is selected, the data determines the model
Adjustable priors can be included in the analysis
Question whether it has anything to do with human rationality
Prediction with beliefs
Good choice of model family
Back to top
Last Revised: Tuesday, 13 January 1998