17. 11. 2020

Neocortex Model paper

by Jeff Hawkins


al., 2003; Polsky et al., 2004). Yet, despite the many advances in understanding the active properties of dendrites, it remains a mystery why neurons have so many synapses and what their precise role is in memory and cortical processing. Lacking a theory of why neurons have active dendrites, almost all artificial neural networks, such as those used in dendrites and with unrealistically few synapses, strongly suggesting they are missing key functional aspects of real neural tissue. If we want to understand how the neocortex works and build systems that work on the same principles as the neocortex, we need an understanding of how biological neurons use their thousands of synapses and active dendrites. Of course, neurons cannot be understood in isolation. We also need a complementary theory of how networks of neurons, each with thousands of synapses, work together towards a common purpose. In this paper we introduce such a theory. First, we show how a typical pyramidal neuron with active dendrites and thousands of synapses can recognize hundreds of unique patterns of cellular activity. We show that a neuron can recognize hundreds of patterns even in the presence of large amounts of noise and variability as long as overall neural activity is sparse. Next we introduce a neuron model where the inputs to different parts of the dendritic tree serve different purposes. In this model the patterns recognized by a neuron’s distal synapses are used for prediction. Each neuron learns to recognize hundreds of patterns that often precede the cell becoming active. The recognition of any one of these learned patterns acts as a prediction by depolarizing the cell without directly causing an action potential. Finally, we show how a network of neurons with this property will learn and recall sequences of patterns. The network model relies on depolarized neurons firing quickly and inhibiting other nearby neurons, thus biasing the network’s activation towards its predictions. Through simulation we illustrate that the sequence memory network exhibits numerous desirable properties such as on-line learning, multiple simultaneous predictions, and robustness. Given the similarity of neurons throughout the neocortex and the importance of sequence memory for inference and behavior, we propose that sequence memory is a property of neural tissue throughout the neocortex and thus represents a new and important unifying principle for understanding how the neocortex works. (Sunday, November 22, 2015, 12:06 PM, page 2)