²It may seem very abstract to consider the world in terms of states. More insight might be obtained by taking an object-oriented viewpoint and considering the world a collection of objects. However, such a theory is still in its infancy (Diuk et al., 2008; Guestrin et al., 2003), so we have to use the approach based on world states. Another question is whether the states should really be considered discrete-valued, such as indexed by integers. In fact, most current AI systems do not use such a categorical representation, but rather some kind of continuous-valued perceptual representation in a neural network, for example given by the outputs of certain neurons. However, the approach using discrete states is widely used in theory and textbooks because of its conceptual simplicity. This is discussed in more detail in Chapters 4 and 7.