Comments on the course examination and its grading for
INFORMATION-THEORETIC MODELING
(Exam dated 22 Oct 2014, examiner: Teemu Roos)

Jussi Maatta and Teemu Roos
Department of Computer Science
University of Helsinki

----------
QUESTION 1
----------

For question 1, half-points were given in some "subquestions".
The total score for question 1 was then rounded up.

(a)

General guideline for grading:
* 1 point: a reasonably correct and complete answer
* 1/2 points: answer is not wrong but does not demonstrate solid understanding
* 0 points: wrong or trivial answer

Comments:
(i)   It's not enough to say that I(X;Y)>=0 means that the mutual information
      of X and Y is nonnegative. That's just restating the question. A good
      answer somehow explains what mutual information and its non-negativity
      means.
(ii)  Either a verbal or formal description of AEP gave full points.
(iii) 1/2 points for describing what is a universal source code.
      1/2 points for arguing why NML is such.
(iv)  1/2 points if you said something that demonstrates that you understood
      what the statement means. Full points if you gave an argument for why
      the statement is true.

(b)

Grading:
For both the minimal and maximal MI cases:
* 1 point for a correct answer
* 1 point for explaining why the answer is correct or a reasonable attempt
  to find such an explanation

Comments:
* The mutual information is minimal (zero) when we have independence.
  This was supposed to be easy.
* The maximal MI case was more difficult than was intended. Trying to solve
  it analytically is messy. Grading was very liberal here.

(c)

Same grading guideline as for 1(a).

Comments:
(i)--(iii): Straightforward if you read the slides.
(iv):       Many missed the answer we were looking for:
            AEP gives the optimal expected per-symbol code-length.
            That is, it is a kind of a lower bound for data compression.

----------
QUESTION 2
----------

(a)

Most people had no trouble with this. Some score deductions had to be made
when you did not state all the steps explicitly enough: we cannot assume
that you know the algorithm, you must demonstrate it yourself.

6 points for a correct answer.
Deductions:
-1: Did not explain how to assign zeros and ones to the edges of the
    Huffman tree.
-1: Did not encode the sequence abcde. (Read the problem statement!)

(b)

6 points: A correct and thorough answer.
5 points: A correct answer with a brief or incomplete written or drawn
          explanation.
3 points: A correct answer but with little explanation of what's going on.
1 point:  Some knowledge of arithmetic coding apparent but did not arrive
          at any sensible result.

A friendly reminder:
You should realize that your answers should primarily demonstrate an
understanding of what you are doing, not your capability of simulating
a computer.

----------
QUESTION 3
----------

Grading for the first bullet point:
* 1 point for correctly defining the model class selection problem.
  (We have a family M_i of model classes, each of them with a parameter
  set \Theta_i. We want to pick the one that best suits your data.)
* 1 point for giving an example that fits the above definition.

For the rest of the bullets:
* 2 points for a solid answer.
* 1 point for saying something that makes some sense both in itself
  and in the context of the question.