Verified by 
There is a lot of excitement in the science-focused blogosphere over a failure to replicate a classic social psychology study. Social psychology has come under fire recently because of the Diederik Stapel phony data debacle so there seems to be a bit of a hair trigger over social psychological data. This trigger was recently pulled again when StéphaneDoyen and colleagues failed to replicate John Bargh’s “elderly-slow” priming effect.
In Bargh’s original 1996 study, people primed with words associated with the stereotype of the elderly (e.g. retirement) actually walked more slowly than people who were shown other words unrelated to the elderly. Thanks to the work of Bargh and others in the years before the elderly study, we had learned that priming someone with a trait word like stubborn would make that person more likely to evaluate another’s behavior as stubborn without realizing this priming-effect had occurred. The elderly-slow finding was stunning because it demonstrated that activating a concept in someone’s mind not only made us more likely to use that concept when judging others – it made us act more like the concept or stereotype ourselves. Hundreds of studies followed showing that people primed with the a stereotype embodied it themselves. Prime someone with the concept of professor and they will do better on general knowledge test. Prime them with supermodel and they will do worse. Reviews of these studies suggest that the general category of priming-induced automatic behavior was quite robust (here & here).
A few weeks ago, PLoS One published Doyen’s failure to replicate the original elderly-slow effect. In study 1, using a slightly different methodology than Bargh, Belgian undergraduates did not walk more slowly when primed with the elderly stereotype: a simple failure to replicate. Because they had measured walking speed with an automated electronic system rather than with handheld stopwatches the authors suspected that the difference between their study and Bargh’s was the subjective timing of the experimenters in the original. Their thought was that experimenter expectancies about the study hypotheses led the original experimenters to bias the walking speeds to confirm the hypothesis. In order to assess this, Doyen et al. ran a second study in which they explicitly gave two sets of experimenters different hypotheses. Lo and behold, experimenters who expected the elderly stereotype to slow people down recorded slower speeds than those who expected the elderly stereotype to speed people up.
Many in the blogosphere took this new publication as a dire sign for Bargh’s research. Before we jump to this conclusion, let’s consider two key issues: (a) Do expectancy effects explain away Bargh’s original findings? and (b) Are prime-induced automatic behavior effects real?
Do expectancy effects explain the elderly-slow effects?
I won’t say its impossible that expectancy effects were involved in the original finding from Bargh’s lab, but I think it's extremely unlikely and nothing about the Doyen et al. findings changes that. Here’s the thing, expectancy effects can alter just about any behavior. This is nothing new. One of my first advisors in graduate school, Robert Rosenthal, ran studies like this with rats back in the 1960s. Experimenters who expected their rats to perform better in a maze did perform better (objectively better). Regardless of whether Bargh’s original finding represented a real effect or not, Doyen et al. would probably have been able to produce the expectancy effects they saw. It is essentially unrelated to the reality of Bargh’s findings.
There was also an essential difference between the Bargh and Doyen findings. Doyen went out of his way to explicitly induce expectations in his experimenters. In contrast, Bargh went out of his way to avoid giving his experimenters any expectations about the study hypotheses. Thus, if there were any expectancy effects in the Bargh findings (and I doubt there were), they likely bear no resemblance to those produced by Doyen. I think it was fine for Doyen to run and report on those two studies, but his conclusion that this in any way might explain Bargh’s original findings is unfounded.
Are priming-induced automatic behavior effects real?
The dominant response in the blogosphere now is to question whether priming the elderly stereotype really makes people walk more slowly. It is true that I don’t know of published replications of the precise effect reported by Bargh on the elderly. But there have been several conceptually-related effects of priming the elderly stereotype including one effect that is mediated by how much contact a person has had with the elderly in the past. It would require some tortured logic to explain how that could have resulted from experimenter expectancies. There have been multiple unpublished non-replications of the elderly-walking (including one in my lab that I may discuss in a future blog), but it is hard to know what to make of them given that they haven’t been peer-reviewed. More than a few times, reviewers of peer-reviewed manuscripts have pointed out flaws in studies that authors had not considered. Its impossible to know how often these non-replications were unbiased high quality replication attempts. It is interesting how uncritically these unpublished non-replications have been received in the blogosphere, in contrast to the original published effect that went through serious peer-review.
At a certain level it does not matter whether the exact primes Bargh used produce a change in walking speed over the exact distance he measured. Some have said 'We need to replicate this exactly. Conceptual replications aren't good enough'. But I'm not sure why we care about this specific manipulation unless we are about to start using it as an intervention to treat patients. What we care about is whether priming-induced automatic behavior in general is a real phenomenon. Does priming a concept verbally cause us to act as if we embody the concept within ourselves? The answer to this question is a resounding yes. This was a shocking finding when Bargh first discovered it in 1996.
Ever since the 1950s when movie theaters tried and failed to use subliminal priming to get us to buy more Coca-Cola, popcorn, and Jujubes, scientists have assumed that primes couldn't change our behavior. Following Bargh's initial findings, hundreds of studies focusing on a wide range of behaviors, stereotypes, and contexts have shown this general class of phenomena to be real. It has also been extended to show that goals and motivation can be primed too. I don't think it is fully agreed upon why these effects occur, but I think the existence proof is complete. The priming effects are real.
Coming attractions…
Of course, file drawer effects are real too (unpublished studies that need to be accounted for when we consider whether an effect is real and how big the effect is). I’ll suggest how this issue can be address in a systematic fashion in my next blog. Before too long I’ll get back to the social brain (i.e. something I actually know something about).
Social: Why Our Brains Are Wired to Connect - now available from Amazon http://amzn.to/ZlYMP3
Follow me on Twitter: @social_brains
