A Progress Report on the WEIRDness of Psychological Samples

Remember the term "WEIRD"? If you're not familiar with this quirky acronym, it stands for Western, Educated, Industrialized, Rich, and Democratic. In the realm of psychology, this descriptor has been disproportionately over-represented, with most research relying heavily on this relatively narrow demographic from Western countries as participants. How far have we come in making psychological samples less WEIRD?

A decade ago, we were inspired by the pioneering work of Henrich, Heine, and Norenzayan (2010), who illuminated the stark reality that WEIRD subjects often significantly differ from the broader human population in a variety of measurable traits. We aimed to examine how well psychological samples represented the global population, and now, we're revisiting the issue to assess any progress.

A Look Back: The Diversity of Psychological Samples

To assess the state of psychological samples, we examined empirical articles published in Psychological Science in 2014. Our original study analyzed 223 research articles, comprising 428 individual studies and 450 samples, examining sample characteristics such as national location, age, online/offline participation, compensation, gender, race, socioeconomic status, and employment status.

The Global Perspective

Our findings showed that American participants dominated psychological research, accounting for approximately half of all samples published in Psychological Science. However, this was a slight decrease from the 68% documented by Arnett (2008). Venturing further into the global landscape, we discovered that 12% of samples hailed from English-speaking countries (excluding the United States), while 16% had roots in Europe. Asian representation was a mere 4%, with Africa, Latin America, and the Middle East garnering just 1% each. Alarmingly, 11% of authors kept their sample location under wraps, a worrying statistic.

Homogeneity Across Borders

When we initially compared sample characteristics across regions, we found relatively homogeneous samples. Most samples were still collected offline, mostly used young adults, and offered fixed compensation.

Under-Reported Demographic Details

While gender diversity was relatively well-documented (71% reporting), racial diversity remained a challenge, with only 20% of samples providing such information. Among those that did report racial demographics, most samples consisted predominantly of individuals from white backgrounds.

Ever-Present Undergraduates

The reliance on undergraduate students as research participants persisted, although at a reduced rate. In 2014, 36% of samples in Psychological Science involved undergraduates, compared to 67% in JPSP in 2007. This decline may be attributed to the increased use of online samples. However, it is challenging to draw definitive conclusions.

From Recommendations to Action: Progress in the Past Decade

Based on our original research, we made several recommendations to foster change and enhance the diversity of psychological science. Ten years on, it's time to review how these recommendations have been put into practice and what progress has been made.

1. Reporting Sample Characteristics

We recommended that authors report various characteristics of their samples. This, we noted, would enable a more comprehensive understanding of the sample composition and enhance transparency in research reporting. It is encouraging to see that the American Psychological Association (APA) has adopted similar requirements in its journals since 2018.

2. Contextualizing Abstracts and Conclusions

We suggested that abstracts and conclusions should be written to explicitly link the findings to the populations sampled and consider the potential influence of cultural and contextual factors. Since our call for this practice, the APA has introduced "Constraints on Generality" statements in its journal guidelines.

Cross-Cultural Psychology Essential Reads

6 Psychological Universals

Partitioning the Rainbow: The Influence of Culture on Color

3. Emphasizing Sample Justification

We called for researchers to provide a justification for their choice of population sample. While this suggestion hasn't been widely adopted, it remains crucial to encourage thoughtful and purposeful sampling strategies.

4. Leveraging Sample Diversity

We encouraged researchers to explore the impact of cultural diversity within their samples, examining the effects of different demographic variables. While formal guidelines for this are yet to be established, occasional requests for such analyses by reviewers suggest growing awareness.

5. Recognizing the Significance of Non-WEIRDness

We urged journal editors to instruct reviewers to consider non-WEIRDness as a point of interest in research manuscripts. Although formal changes in review criteria are yet to be reported, the idea of including diversity statements in evaluation processes has been making the rounds in scientific communities, even leading to controversy in some areas.

6. Incentivizing Diversity

We proposed offering recognition or rewards for studies that sample more diverse populations. The Global Environmental Psychology journal has taken a step in this direction by considering diversity as a criterion for manuscript evaluations.

7. Setting Diversity Targets

We recommended setting a goal for at least 50% of published manuscripts to include studies sampling non-WEIRD populations in at least one dimension. While no leading journals have yet adopted this target, it remains an important discussion point.

The Road Ahead: Overcoming Challenges and Future Directions

We have made some progress over the past decade in addressing the limitations of WEIRD samples in psychological science, but several challenges remain. The need for diversity extends beyond just sample characteristics — it also applies to the researchers conducting these studies. We must foster a diverse pool of scientists asking different and non-WEIRD research questions.

There are also concerns about the potential for diversity targets leading to well-funded Western institutions simply paying for access to non-WEIRD samples, and the need to avoid ethnocentric measurements and uphold ethical practices in cross-cultural research.

In closing, the crucial question remains: How can we further enhance the diversity of our science to better understand the psychology of Homo sapiens? This complex challenge demands ongoing dialogue, collaboration, and concerted efforts from researchers, publishers, and scientific societies. By valuing diversity and recognizing its importance, we can create a more comprehensive and inclusive psychological science for the benefit of all humanity.