Variety Is the Spice of Life (In Research Too)

Psychology has been heavily criticized lately for its research practices. Most commonly discussed is the so-called “replication crisis,” whereby efforts to replicate classic and non-classic studies alike have often failed. Psychology has become transfixed on this issue, and at times even paralyzed. But there are clear signs that things are changing for the better.

Researchers, journals, and organizations are pushing for greater transparency, cooperation, data sharing, and pre-registration of research hypotheses, methods, and data analytic strategies. This must all surely be good for the science of psychology.

It is clear that psychology bears the brunt of the criticism about replication. Truth be told, however, many (if not most) fields of science have difficulties replicating their findings. John Ioannidis wrote a very highly read and cited paper on this problem called "Why Most Published Research Findings Are False" that has shaken science deeply. Most people concentrate on psychology, and psychologists certainly like pointing their fingers at themselves, but the deeper we dig, the more we find that this is not a problem limited to psychology, but rather is common in other sciences, including medicine and chemistry.

Psychology certainly has its faults. But psychology is also at the forefront of addressing concerns about research methods, inferential statistics, and best practice recommendations.

Replication failures happen for many reasons, including low statistical power and small samples (that generally fail to generalize to the wider world). One of the problems with trying to reproduce psychological findings concerns the sheer complexity of the human mind. (As a psychologist, and especially as a social psychologist, I’m often envious of researchers in other disciplines who study more simple phenomena, particularly those that don’t react to being studied!).

It is also important to keep in mind that psychology generally attempts to explain relatively universal truths about humanity, but in reality, we only look at a very narrow slice of humanity, what researchers call WEIRD (Western, Educated, Industrialized, Rich, Democratic) participants. As Henrich et al. note, “Within the field of psychology, 95 percent of psychological samples come from countries with only 12 percent of the world’s population (Arnett 2008).”

Is psychology the only offender in this regard? Sadly, no. Apparently, studies that map the human genome are similarly focused on a very narrow slice of humanity. As of 2009, about 96 percent of genomic data was derived from people of (White) European background.

And things have not improved much since. Devaney (2019) reports that our present understanding of the human genome has been largely derived from samples who are White European (78 percent), with very little from those with African (2 percent) or Hispanic or Latin American (1 percent) ancestry. This is staggering—one would reasonably expect that researchers attempting to map humanity’s genomes would sample a broad swathe of humanity.

So what is the problem with understanding the psychology of humanity from observing predominantly White, European people in educated and enriched environments? And what is the problem with only using such people to map the human genome?

The question almost answers itself, doesn’t it?

We should be invested in examining the full diversity of humanity if we want to understand the full diversity of humanity. One wouldn’t study only penguins if one wished to learn about birds as a general category. The same applies to the study of humans, whether their psychological makeup or their genetics.

Failure to examine the diversity of a species is fraught with problems and possible dangers when trying to make claims about the nature of that species. After all, we’d risk concluding that birds can’t fly if we spent our research energies only in the Antarctic.