¹⁷Here we focused on the uncertainty in obtained reward. In an orthodox Bayesian interpretation, it may in fact not be possible to say that there is any uncertainty about the expected reward, since the expected reward is a subjective quantity, something purely defined by what the agent believes and expects. In contrast, in a “frequentist” intepretation, the expected reward is an objective quality in the outer world (how much the agent would get on average if it repeated the same action many times) so it can be misestimated, thus adding to the uncertainty of reward loss. Notwithstanding such theoretical arguments, I think it is clear that for biological organisms, understanding the real evolutionary value of, say, a piece of food may actually be a highly complex process involving a lot of learning and computation, so it can surely go wrong, as in the case of sugary food, which means there is uncertainty. I should also mention that the lowering of expectations discussed earlier in this chapter is different from the appreciation of uncertainty considered here since, in probabilistic terminology, mathematical expectation is completely different from variance.