Do Discrimination Self-Reports Follow Principles of Logic?

There are many different ways to assess discrimination. My last post reviewed three recent papers, which use a range of methods, and found discrimination to be a relatively rare exception rather than the rule. One method is to survey people about their experiences of discrimination. There are, of course, many limitations to such self-reports; one probably should not take them at face value absent extensive validation work. This post, however, is not a general evaluation of such measures.

Instead, I focus on one potential flaw in such assessments: They may produce logically incoherent responses.

What Is Logical Incoherence?

Source: Painting: Domenichino. Source: Wikimedia Commons

Any belief is logically incoherent if it makes mutually exclusive predictions and or requires mutually exclusive assumptions. For example, “all beliefs about groups are inaccurate” is logically incoherent, because it would mean that believing two groups differ is inaccurate, and believing they do not differ is inaccurate.

Here is a famous psychology example. It is logically incoherent to believe that:

It is more likely that:

1. Linda is a bank teller and active in the feminist movement

than

2. Linda is a bank teller.

This is because all Lindas who are feminist bank tellers are bank tellers; if any Lindas who are bank tellers are not feminists then there are more Lindas who are bank tellers than Lindas who are feminist bank tellers. Even if all Lindas who are bank tellers are also feminists, 1=2. Under no conditions can statement 1 be more likely than statement 2.

This is the “conjunction fallacy” and was first discovered by Tversky and Kahneman (1983) who found that 85 percent of their respondents incoherently rated 1 as more probable than 2. A conjunction (both of two things being simultaneously true) cannot possibly be more likely than either one being true. It cannot possibly be more likely that tomorrow will be hot and humid than that it will be hot.

Some Conjunctions Are Subtle

It cannot be more likely that you will stay home tomorrow because you are ill than that you will stay home tomorrow. Don’t be distracted by the “because.” It is still a conjunction (stay home and ill most both occur). Which gets us to discrimination. It cannot be more likely that someone will treat you disrespectfully tomorrow because of discrimination than that someone will treat you disrespectfully tomorrow.

Discrimination Survey Responses Display the Conjunction Fallacy

But that is how some people respond to such survey questions – as if being treated disrespectfully because of discrimination is more common than being treated disrespectfully.

Gomez & Trierweiler (2001) examined self-reported experiences of mistreatment by others among 135 women and 170 African Americans. They asked many questions, such as “How often do your coworkers (or classmates) act as if they are better than you?” under either of two conditions. In one, the questionnaire was labeled Everyday Experiences. In the other, for women, the questionnaire led off with instructions saying “we are interested in everyday experiences of sexism and gender discrimination;” for African Americans, it read “everyday experiences of racism and race discrimination.”

Participants in Gomez and Trierweiler’s research reported results consistent with the conjunction fallacy. They reported experiencing more mistreatment due to discrimination than mistreatment:

Source: Data: Based on Gomez & Trierweiler, 2001. Image: Lee Jussim

This finding was been unintentionally replicated in 2019 by Lee et al, in a study of whites’ and non-whites (mostly people described in the article as Black, Asian, or Hispanic) perceptions of being disrespected. I say “unintentionally” because Lee et al did not cite the Gomez and Trierweiler study and because their report makes no mention of logical coherence or the conjunction fallacy.

One group was asked, “How often do you feel you have been treated with less respect or courtesy than other people?” A second group was asked, “How often do you feel you have been treated with less respect or courtesy than other people because of your race, ethnicity, gender, disabilities, sexual orientation, or age?” (Yet a third group was asked exclusively about race-based discrimination, which produced results much like the second group and, for simplicity, is not presented here.) Response options were 0 (never), 1 (rarely), 2 (sometimes), and 3 (often).

Source: Data based on Lee et al, 2019. Image: Lee Jussim

There is some good news here. The % Often responses are logically coherent; more people say they have been often treated with less respect than say they are treated with less respect because of discrimination.

But the % Sometimes results are incoherent. More people claimed to be sometimes treated with disrespect because of discrimination than claim they are sometimes treated with disrespect, which is not logically possible.

Bias Essential Reads

The Impact of Puberty on Racial Bias Against Black Children

Navigating Cognitive Minefields

Some Cautions

These were not particularly large studies (there were only about 30 non-white participants in each condition of the Lee et al study) or had representative samples. That means we do not know whether this sort of logical incoherence is frequent. Regardless of how generalizable the findings are, both studies found logical incoherence in their respondents’ self-reports of experiences of discrimination.

Something Is Wrong Somewhere

Proof that I have supernatural powers.

Source: Lee Jussim

We now know that something is wrong somewhere. If the questions are interpreted as intended, responses are incoherent; either the self-reported estimates of being victimized are underestimated, the self-reported estimates of being victimized by discrimination are overestimated, or both. Because the studies do not permit us to conclude which beliefs are incorrect, no one, including researchers, should presume either is correct, pending better validity evidence.

It is also possible that the questions are not interpreted as intended. Psychologically, perhaps people think about completely different things when they think about disrespect versus disrespect due to discrimination. If so, then respondents might not be logically incoherent. If, when asked about Linda the bank teller people think about bank tellers named Linda, but if, when asked about Linda the feminist bank teller, people think about feminists (and forget the bank teller part) then it is not logically incoherent for them to think there are more Linda the Feminists than Linda the Bank Tellers.

Although this seems far-fetched, it is not impossible, and would mean that respondents are not responding to the question the researchers meant to ask. In this scenario, respondents are interpreting the questions very differently than the researchers, who therefore should not presume they know how to interpret them.