Opening Up the File Drawer in Cross-Cultural Psychology

Imagine spending five years thinking of creative ways to solve a problem, and then shelving your solution deep into your cabinet so that no one finds out what you did. Does this seem strange to you? And yet, this is a common occurrence in science. When scientists do not find results that are in line with what they expect, they often do not publish the results, and this is called the file drawer problem. It is rampant across all scientific disciplines: A shiny new drug with promising results is lionized, and the ones that fail are hidden away.

As a defense mechanism, maybe this seems okay—we do want to hide away our “failures"—but what does this look like in science as an institution? Millions of dollars and time wasted: the time and money of scientists and their associates, recruiting and paying participants, and the like, in an attempt to hide how their assumption about the world was probably misguided. As such, scientists owe it to each other, their participants, and the general public to explain how and why certain assumptions were wrong. This also means that the literature regarding a certain topic is biased: We only know what is true and not what is non-confirmatory, or what is false.

However, the file drawer problem has now been at the center of discussion in meta-science, what with improved attention to replication and reproduction of scientific findings. This means that independent scientist teams are now working toward corroborating published findings, so that we can trust the findings better. After all, if multiple teams have found similar results, we can ascertain that it was not a fluke. Of course, this is not a function of a single scientist or lab, but a systemic problem: Novel findings are much easier to publish than replications or reproductions.

The file drawer problem has strong implications for cross-cultural psychology.

Source: Clker-Free-Vector-Images/Pixabay

Now, think of cross-cultural psychology, a subfield of psychology aimed at understanding how cultures are similar or differ, or how we are similar or different as a product of our respective cultures. Consider this: If we do not publish null or negative results, what happens to results showing that countries or cultures are similar? Given the reluctance of journals to publish null results, it is likely that cross-cultural similarities are not known, and that we exaggerate differences across cultures. This is especially concerning considering that even within cross-cultural psychology journals, more than 85% of the samples were from the U.S. This means that a large proportion of our knowledge about cultures, especially within psychology, is how they compare to the U.S.

The problem of file drawers, especially when it comes to cross-cultural psychology, takes entirely different forms. For instance, if we found that Indians and U.S. Americans are similar with respect to honesty, this seems to be less flashy than the findings that “Indians are less honest than Americans.” This, coupled with the fact that the latter is more likely to get published, means that now scientific scholarship has acknowledged and canonized a difference. If a second study by an independent group finds that there is no difference between the two countries, this replication is less likely to get published, which means that we will never know how the two countries are similar.

Emphasizing how cultures are different from the U.S. also likely has important implications beyond publishing. Research has shown that emphasizing differences versus similarities affects how we perceive groups. For instance, when similarities are highlighted, instead of differences, interpersonal and intergroup attitudes improve. This means that highlighting how Indians and US Americans are similar in honesty levels would improve how each group perceives the other.

Ignoring null results might exaggerate cross-cultural differences and downplay similarities

Source: მარიამ იაკობაძე/Wikimedia Commons

Circling back to how an extraordinarily large number of samples in cross-cultural psychology compares Americans with other countries, this implies that because of the file drawer, we exaggerate how Americans are different from people in other countries. This also means that there might be an unnecessarily increased “exotic”-ization of cultures thought of as different from the U.S. For example, we have seen how prejudice against Asian Americans has increased during the COVID-19 pandemic, especially with respect to the foods that they eat. China as a country holds about 18% of the world’s total population, while the U.S. holds only about 4%, and yet China is highly exoticized in psychological literature, wherein the minority country—the U.S.—is thought of as “normal” or even “moral.”

Though the strengths of conducting replications and other aspects of open science are being increasingly recognized, these replications should also include samples from other cultures. In case there are differences in what was originally found versus a different cultural context, it is imperative that we not automatically dismiss the difference because of the “exotic” nature of culture, but examine the result in itself. That is, if, for example, we find that Indians are more aggressive than U.S. Americans, then instead of claiming the “backward” or “less evolved” culture of India, it might do well to consider whether the tools measuring aggression are culture-fair; whether Indians and Americans interpret “aggression” in similar ways; and whether the definition of aggression is all-encompassing. This is likely only if Indian researchers are involved in every step of research decision-making. More importantly, it is also necessary to focus on cultural similarities, of which there might be many that are unacknowledged, rather than only differences.

This post was written by Arathy Puthillam, a research psychologist at Monk Prayogshala, India. Her research focuses on social, moral, and political psychology. She tweets @WallflowerBlack