¹⁰Chapter 9 will explain the idea of replay whose application to this case would be as follows. Maybe your brain does actually hear the sentence “Helsinki is the capital of Finland” many times. One of the learning systems in the brain is based on storing events, or short episodes, in an area called the hippocampus. It uses special mechanisms, presumably quite different from stochastic gradient methods, to store the sentence after hearing it just once. Then, the hippocampus feeds the sentence to the other parts of the brain many times, and that allows Hebbian learning and something similar to stochastic gradient learning to happen.