3 Concepts: Information
Course Material
Lectures
» Set 1:
What is Information Theory?
» Set 2:
How much can we compress? - Shannon's Source Coding Theorem
» Set 3:
Revenge of a Student - Symbol Codes
» Set 4:
On Minimum Description Length Modeling
» Set 5:
`Year 2020' - Topics in Information Theory for Further Studies
Literature, further reading
Some good general information sources for various parts of the course
-
Bell T. C., Cleary J. G., Witten I. H.,
Text Compression.
Prentice Hall, Englewood Cliffs, New Jersey, 1990.
-
Berlekamp E. R.,
Algebraic Coding Theory.
McGraw-Hill, New York, 1968.
-
Blahut R. E.,
Principles and Practice of Information Theory.
Addison-Wesley, New York, 1987.
-
Cover T. M., Thomas J. A.,
Elements of Information Theory.
Wiley, New York, 1991.
-
Deco G., Obradovic D.,
An Information-Theoretic Approach to Neural Computing.
Springer-Verlag, New York, 1996.
-
Frey B. J.,
Graphical Models for Machine Learning and Digital Communication.
MIT Press, Cambridge, Massachusetts.
-
Grünwald P.,
The Minimum Description Length Principle and Reasoning under Uncertainty
ILLC Dissertation Series DS 1998-03.
-
Lin S., Costello, Jr., D. J.,
Error control coding: fundamentals and applications.
Prentice-Hall, Englewood Cliffs, N.J., 1983.
-
MacWilliams R. J., Sloane N. J. A.,
The theory of error-correcting codes.
North-Holland, Amsterdam, 1977.
-
McEliece R. J.,
The theory of information and coding: a mathematical framework for
communication.
Reading, Mass.: Addison-Wesley, 1977. Reprinted 1984 by C.U.P. in
Encyclopedia of Mathematics.
-
Peterson W. W., Weldon, Jr., E. J.,
Error-Correcting Codes.
MIT Press, 2nd Edition, Cambridge, Massachusetts, 1972.
-
Rao T. R. N., Fujiwara E.,
Error-control Coding for Computer Systems.
Prentice-Hall, 1989.
-
Williams R. N.,
Adaptive Data Compression.
Kluwer Academic Publishers, USA, 1991.
-
Witten I. H., Neal R. M., Cleary J. G.,
Arithmetic coding for data compression.
Communications of the ACM, 1987, Vol. 30, No. 6, p. 520-540.
Links
»
DataCompression.info
»
Compression Pointers
»
MIT Course on Information and Entropy
»
Stanford Information Theory Class
»
Entropy on the World Wide Web
»
G.J. Chaitin Home Page
»
David Dowe's Minimum Encoding Length Inference Page
»
Bayesians Worldwide
»
Quantum Information
»
Information about gzip