Understanding How Reinforcement Works

The concept of "reinforcement" has a long history in psychology. Pavlov used the term reinforcement to explain the strengthening of the association between the sound of a bell and the production of saliva in dogs during respondent conditioning, also known as classical conditioning. Skinner used the term "reinforcement" to explain the increased frequency or probability of responding during operant conditioning, also known as instrumental conditioning. Skinner's work gave rise to the branch of psychology known as the experimental analysis of behavior.

The Journal of the Experimental Analysis of Behavior publishes results of basic science research concerning the necessary and sufficient conditions required to predictably modify behavior. The field of Applied Behavior Analysis has developed these findings into a technology of behavior change (see Martin and Pear, 2014). The Journal of Applied Behavior Analysis reports research concerning the therapeutic applications of this behavior change technology. The evidence of the effectiveness of this behavior change technology is so strong that it currently characterizes most school psychology interventions.

Both the basic and applied fields of the experimental analysis of behavior are rooted in behaviorism. Unfortunately, behaviorism cannot explain how and why reinforcement works. Behaviorism has supported the identification of necessary and sufficient conditions for reinforcement to work but cannot explain how and why those conditions are necessary and sufficient for reinforcement to work.

In short, behaviorism cannot explain how and why reinforcers are reinforcing. That reinforcement has a cognitive basis has been empirically established since the pioneering work of Rescorla (1987, 1988), but cognitive psychologists still cannot explain how and why reinforcement works. On the contrary, the cognitive revolution in psychology marginalized the relevance of reinforcement learning regarding explanations of behaviors that involve higher cognitive processes rather than helping to explain them.

What is actually going on during reinforcement? How can we explain how reinforcement works? Pennington (2014) asked and answered the question: "What does it mean to explain something from a natural science perspective? Basically, it means that we identify the cause of that thing in terms of relevant mechanisms" (p. 3). Mechanisms answer both the "how" and "why" questions that philosophers typically separate. Mechanism information provides step-by-step details that explain how something works the way it does. The necessary connections among these steps explain why something works the way it does. What is the mechanism that explains how reinforcement works?

Cognitive psychologists understand reinforcement from a mental perspective. They prefer mental mechanisms, such as storing copies of experiences as memories and learning rules. While such explanations make intuitive sense, they do not contain actual mechanism information, because they do not specify what is physically occurring during reinforcement.

Skinner (1953, 1977) objected to storage metaphors on the basis that these "explanations" do not actually explain anything because, as metaphors, they do not provide mechanism information. Skinner creatively concluded that the rat survived reinforcement as a changed rat without specifying the nature of that change because he was writing long ago when neuroscience did not yet have relevant mechanism information.

Neuroscience research over more than the past half-century has revealed much relevant mechanism information regarding how reinforcement works. Our neurons are connected to one another with synapses. These synapses are collectively known as our connectome. It is responsible for all that we think and do, including our personalities (Seung, 2012). Reinforcement modifies the excitatory/inhibitory properties of relevant synapses via biological experience-dependent plasticity mechanisms (Bear, Connors, and Paradiso, 2015). Connectionist neural network models implement essential elements of these biological mechanisms. These models provide a more dynamic and psychologically relevant explanation of learning and memory formation. For example, Read and Miller (1998) published a connectionist model of social reasoning and social behavior. Read and others (2010) published a connectionist model of personality.

I, Tryon (2014), have discussed the relevance of connectionist models to clinical psychology. My previous Psychology Today post entitled "Belated Happy Birthday AlphaZero" distinguished connectionist brain-based models from standard psychological mind-based models. That post presented a simple, three-layer neural network model and illustrated how it works with a figure. If you look at this figure, you can see that altering the values of the simulated synapses connecting the simulated neurons will clearly change how the network processes information and thereby will necessarily change how the network responds. This is what reinforcement does. Reinforcement modifies synapses, and these changes alter how the network behaves.

Psychology Essential Reads

The Time Cure: Taking Our Leave

Is a Texas Bitcoin Miner Making People Sick?

How effective are these connectionist models? Why should we prefer their explanations of cognitive processing? The evidence is in their remarkable accomplishments in the field of artificial intelligence. I discuss several of these accomplishments in my post "Belated Happy Birthday AlphaZero."

But didn't psychologists reject Skinner's reinforcement explanation of how we learn a long time ago? Yes, they did, because reinforcement learning makes little sense from the perspective of mind-based models because we rarely learn anything when someone gives us a reinforcer during every step of the learning process. But reinforcement learning makes a great deal of sense from the perspective of brain-based connectionist models, where synaptic properties gradually and continuously change during learning. This distinction reveals that reinforcement learning means two different things.

Reinforcement as the practice of giving or doing something to a person while they are learning is mostly what cognitive psychologists rejected as characteristic of how we learn. They also rejected the view that learning mainly involved strengthening associations. But the term reinforcement now has an artificial intelligence meaning that refers to the synaptic change process that is central to connectionist neural network models of psychology and behavior.

It is this perspective that supports Skinner's view of learning. Reinforcement refers to the gradual modification of synaptic properties that occurs during learning. These synaptic modifications shape our behavior in predictable ways. They enable us to learn by forming memories. These synaptic modifications allow for experimental and applied behavior analysis to predictably modify behavior. Reinforcement physically modifies our connectome.