**Three Concepts: Information**

# Assignments

Return assignments by the following lecture by e-mail (both Teemu
and Jukka) or in writing (during the lecture).

### Assignment for Lecture 2: ''Mathematical Preliminaries''

Read **Chapter 2** of Cover & Thomas (to be found in the course folder),
and do **at least four** out of the following five exercises:

- Show that
*ln* is strictly concave.

*Hint:* Derivatives.

- Show that Gibbs' inequality holds as an equality if and only if
*p(x) = q(x)* for all *x*.

*Hint:* When does
*ln(x)=x-1* hold? *Alternative hint:* Next exercise.

- Prove Gibbs' inequality using Jensen's
inequality.

*Hint:* Gibbs' inequality is also known as the
"information inequality". Note that *-ln* is (strictly)
convex.

- Exercise 2.2 in Cover & Thomas (1st
edition).

*Hint:* Look at Exercise 2.5 ("Entropy of functions
of random variable") in Cover & Thomas (1st edition). If
Y=2^{X} or Y=cos X, in which case(s) is Y a function of X and/or X
a function of Y?

- Exercise 2.16 (item
*(f)* optional) in
Cover & Thomas (1st edition).