# Assignments

Return assignments by the following lecture by e-mail (both Teemu and Jukka) or in writing (during the lecture).

### Assignment for Lecture 2: ''Mathematical Preliminaries''

Read Chapter 2 of Cover & Thomas (to be found in the course folder), and do at least four out of the following five exercises:

1. Show that ln is strictly concave.
Hint: Derivatives.
2. Show that Gibbs' inequality holds as an equality if and only if p(x) = q(x) for all x.
Hint: When does ln(x)=x-1 hold? Alternative hint: Next exercise.
3. Prove Gibbs' inequality using Jensen's inequality.
Hint: Gibbs' inequality is also known as the "information inequality". Note that -ln is (strictly) convex.
4. Exercise 2.2 in Cover & Thomas (1st edition).
Hint: Look at Exercise 2.5 ("Entropy of functions of random variable") in Cover & Thomas (1st edition). If Y=2X or Y=cos X, in which case(s) is Y a function of X and/or X a function of Y?
5. Exercise 2.16 (item (f) optional) in Cover & Thomas (1st edition).   Three Concepts: Information 2007