Verified by 
Environment
What Constitutes Real Science?
Science and Nature question psychology and neuroscience research
Posted August 29, 2015
A team of researchers set the scientific community on its ear recently when it published findings that showed 60 out of 100 psychological research study findings could not be replicated.[1] Both Science and Nature published the findings delivering a serious blow to the scientific credibility of psychological research. The team showed that replication – the basis for being admissible science – was not possible or netted watered down answers in more than half of the psychology studies. Without reproducibility, these studies become information and opinion, not scientific fact. Why did this happen? What does it mean?
In academia, where the lion’s share of research occurs, it is publish or perish. Researchers who prove something compelling are more likely to be published in Tier 1 journals. However, even at Tier 1 institutions, many scientists who are not well-trained in constructing research, create hypothesis-driven research that is only compelling if the hypothesis is proven.[2]
I had the good fortune/rude awakening of studying with Mark S. Cohen PhD, Tony Buffington, DVM, PhD, Emeran Mayer MD, PhD, and the late great Candace Pert, PhD (who discovered opiate receptors on the brain). They were all superb scientists who would not even consider a design paradigm that would not produce compelling findings, whether the hypothesis was proven or disproven. Creating such a research paradigm is very difficult, but it’s the difference between good science and collecting possibly misleading information that cannot be replicated.
When you are emotionally invested in proving or disproving your hypothesis, you become vulnerable to confirmation bias, which pays attention to facts that support your expectations and ignores those that do not.[3, 4] Creating and conducting compelling research is difficult – the scientists who trained me were champions. Just like in sports, not everyone is a champion because not everyone is willing or capable of doing the work it takes to perform at a championship level. I get that because I am not an A-list researcher, but my colleagues are. I bring other unique skills to our table that interdigitate with the expertise of other team members. Most scientists and labs do not have this luxury. Scientists unwilling or incapable of doing the extra work necessary for verifiable results become too comfortable with increasing sample size to achieve desirable statistical outcomes, or cherry-picking from the data report to support their hypotheses, which can produce misleading results or findings that cannot be replicated.
The Darker Side
These are difficult economic times. National Institutes of Health (NIH) funding is increasingly more difficult to secure. Researchers are forced to turn to private funding sources, such as the pharmaceutical companies. The drug companies are interested in studies that support the efficacy of their products, or disprove or minimize their harmful effects. Scientists are people. Some people are more ethical than others. I am most certainly not defending the scientists who bias research to achieve an end that pleases the hand that feeds them. However, I understand financial desperation and its ability to dissolve moral fiber. We are humans. No realm is sacrosanct in our world because humans walk where angels won’t. This is not a science thing happening to humans - it’s a human thing happening to science. Likewise, I understand that humility turns humans into angels and ego turns people into demons. Again, scientists are people, and some people, who feel unappreciated, do off-centered things to shore up sagging spirits, which in research can lead to data that is not replicable. I don’t condone this either, but I understand quiet desperation and curious actions to survive feelings of inadequacy and under appreciation. We have all been there, we all know this to some degree – again it’s a human thing.
The Brighter Side
Humans want definitive answers and certainty; and Detroit wants palm trees. No single scientific study is ever going to produce a definitive answer one way or another even in the most meticulous and ethical research - that’s just how science is. One of my mentors, Professor Sandra Cole, Ph.D. (University of Michigan School of Medicine) taught me that to be a good consumer of the research, one must understand who the researcher was, who the subjects were, and the context in which the research occurred. That is the best we can hope for - to be good consumers of the research.
How is this the brighter side? Well, it’s like when I was a small child: my mother, and three other neighborhood mothers, decided they needed to build a fence to keep the children safe. What they lacked in skills and tools they made up for in determination and attitude. Yes, they used nails where they should have used screws, and sometimes one pounded in nails with a brick, because another was using the hammer. But at the end of the day, they had built a fence. A few years back, I revisited the place where my childhood home once stood: the houses were gone, and the lots were grown over with weeds, but that fence was still standing. Moreover, no child ever wandered into the street – it had served its purpose.
The purpose of science is to increase human knowledge. Doing that presents difficulties that are specific to each scientific discipline, and psychology and neuroscience face more complexities than most – perhaps. Like that fence, the neighborhood women built, the posts are not always perfectly aligned, and the integrity varies, and some nails are pounded in perfectly, whereas others halfway and bent. However, at the end of the day, those women built that fence and they knew what the fence could and could not do, so the fence served its purpose. Likewise, psychology and neuroscience, like all science, builds knowledge that promotes human safety. We rely on it like my mother and those women relied on that fence. That safety has to be built and reinforced in a collaborative effort, as best we can, with what we have to work with, and accepted for what it is, or what is not. Onward! Remain Fabulous and Phenomenal.
Join my email list to receive notifications of new posts
Or visit me at:
UCLA Center for the Neurobiology of Stress at the David Geffen School of Medicine
REFERENCES
1. B. A. Nosek, G.A., G. C. Banks, D. Borsboom, S. D. Bowman, et al Estimating the reproducibility of psychological science. Science, 2015. Vol. 349 no. 6251
2. Baker, M., Over half of psychology studies fail reproducibility test. Nature, 2015.
3. Fugelsang, J.A., et al., Theory and data interactions of the scientific mind: evidence from the molecular and the cognitive laboratory. Can J Exp Psychol, 2004. 58(2): p. 86-95.
4. Mercier, H. and D. Sperber, Why do humans reason? Arguments for an argumentative theory. Behav Brain Sci, 2011. 34(2): p. 57-74; discussion 74-111.
