Verified by 
Race and Ethnicity
Hard Truths and Half-Truths about Race on Campus, Part II
Stereotype threat is (still) overblown vs standardized test validity denialism
Posted June 12, 2016
This is the second in a three part series addressing the recent controversies on campus. It is a response to a critique of my WSJ article titled Hard Truths about Race on Campus, by Evelyn Carter (a postdoctoral psychologist at Purdue) and Lisel Murdock-Perriera, a graduate student in Stanford’s Psychology program (hence, C & M-P). You can find a synopsis of and links to both our article and the critique here.
One of the common demands made in the recent round of student protests is to raise black admissions to be proportional to the black population (about 13% in the U.S., but the demands sometimes vary with respect to the reference population). In our article, we pointed out that, because of an existing large black/white academic qualifications and achievement gap on most campuses, this would: 1. Increase race differences on campus; 2. increase the informativeness of race with respect to academic competence on campuses; and 3. thereby further divide students and increase tensions.
C&M-P characterized this as a "half-truth" and further criticized us for ignoring stereotype threat research. According to C&M-P, stereotype threat research shows that race "...differences in test scores have little to do with ability or qualifications, and more to do with a climate that systematically favors majority applicants."
It's argument deconstruction time once again...
DID WE IGNORE STEREOTYPE THREAT RESEARCH?
They wrote:
“…Haidt and Jussim ignore equally well-documented relevant research on stereotype threat.”
Their claim that we ignored stereotype threat is completely true. This is because stereotype threat is: 1. Not only not well-documented, it is widely misrepresented and misinterpreted; 3. Irrelevant even if it was well-documented.
This will be obvious once what was actually found -- as opposed to the manner in which it is widely oversold, misrepresented and misinterpreted -- is understood,
STEREOTYPE THREAT HAS NEVER SHOWED THAT "REMOVE THREAT AND B=W TEST SCORES"
C&M-P wrote:
In settings that communicate that stereotypes are valid bases for evaluation, we see that Black students underperform relative to their White counterparts (Steele & Aronson, 1995; Steele, 1997). Notably, messages that communicate the irrelevance of stereotypes on judgments of performance minimize these achievement gaps (Steele, Spencer, & Aronson, 2002). Universities concerned about underperforming minority students should consider how they can create identity-safe environments that diminish the relevance of stereotypes in judging performance, not conclude that Black students have “weaker qualifications” than White students (Purdie-Vaughns & Walton, 2011).
I have addressed this issue elsewhere (see also Jussim et al, in press; Sackett et al, 2004). The research they cited (Steele & Aronson, 1995; Steele, 1997) did not provide a shred of evidence that “identity-safe” environments even reduce pre-existing racial gaps, let alone eliminate them. Steele & Aronson (1995) found that, when threat was removed, black/white statistically adjusted means were equal only after controlling for prior differences. This is like saying, "there were no differences, after we removed the differences." Declaring "blacks and whites had the same test scores, controlling for previous test scores" is just as vacuous as declaring "Tampa and Nome have the same average daily high temperatures, controlling for previous temperatures” (we actually statistically demonstrated this with real temperature data using the Steele & Aronson method of comparing adjusted means, see my earlier blog entry, Is Stereotype Threat Oversold, for the gory details).
STEREOTYPE THREAT IS DUBIOUS, NOT “WELL DOCUMENTED”
And that’s the good news. Stereotype threat proponents have now backed off from their once common claims that stereotype threat fully accounts for Black/White achievement differences. For example, summarizing their meta-analysis of stereotype threat, Walton, Spencer, & Erman (2013) wrote: “The estimates suggest that psychological threat accounts for … 17–28% of the White/Black gap on the SAT.”
But whether stereotype threat is even a 17-28% piece in the puzzle is not clear to many scientists outside of an inner circle of advocates and enthusiasts. Skeptical reviews of the stereotype threat literature question whether the phenomenon actually even exists, and suggest that, if it does, its overall effects are quite small (Flore & Wichert, 2012; Stoets & Geary, 2012). Furthermore, some large scale failures to replicate have also been published. Stereotype threat is "well-established" among its most evangelical advocates, but is controversial in the wider scientific community.
Truth rating of “stereotype threat” as being a “well-documented” phenomenon providing a powerful explanation of racial achievement differences? Mostly untrue.
Truth rating of any claim that if one removes stereotype threat, there are no black/white achievement differences? Completely false. (for the record, although C&M-P did not make this strong claim explicitly, they concluded that "... universities ... should ... not conclude that black students have 'weaker qualifications' than white students" in large part on the basis of stereotype threat research -- so I see no daylight between their claim and the false interpretation of stereotype threat research).
DO STANDARDIZED TESTS "HAVE LITTLE TO DO WITH ABILITY"? A CASE OF SCIENCE DENIAL ... BY SOCIAL SCIENTISTS
C&M-P also make the stunning claim that: “differences in test scores have little to do with ability or qualifications…”
This is out of touch with the science. The predictive validity of standardized tests is one of the most well-established findings in all of psychology. (Who said conservatives are the only ones engaging in “science denial”?). It constitutes wishful thinking and “making stuff up” or, possibly, willful blindness to overwhelming data for the validity of achievement and cognitive ability tests, which predict not only educational performance, but all sorts of life outcomes (Kunzel & Hezlett 2010; Neisser et al, 1996). To be sure, intelligence and achievement test scores are not the only factors influencing or predicting future accomplishment. Grit, motivation, and commitment also count a lot, and sometimes more than test scores.
But that is a far cry from claiming "test scores have little to do with ability or qualifications." Can C&M-P really be saying that people who score in the bottom 10% of SATs and GREs – roughly comparable to those with IQs of about 80 – are generally just as capable as the people with scores in the top 10%, roughly comparable to IQ scores of 120? If they meant something other than this, then perhaps they will articulate it.
Truth rating of the claim that “test scores have little to do with ability or qualifications” – completely untrue.
STEREOTYPE THREAT IS NOT RELEVANT EVEN IF IT IS VALID
If stereotype threat is valid, it means standardized tests underestimate blacks' academic competencies and might explain about 50 points of the 200 point SAT gap. This is one argument that can be made for preferential selection procedures that are more modest than the 120 point preferential selection at many colleges.
But even if stereotype threat is 100% true (which I doubt), it is irrelevant to our point that one cannot go deeper into any applicant pool without reducing the quality of those admitted. Such a claim reflects either an extraordinary level of innumeracy for scientists with a Phd and training in statistics, or a willful refusal to acknowledge the obvious. Most academic qualifications (e.g., grades, standardized test scores) are approximately normally distributed. One can see this here, and, although this is data from 2000, things have not changed much. This means that, the deeper you go into any college applicant pool, the less qualified the applicants become.
But one does not need statistics to see this, one just needs common sense. Even street sports-playing 10 year olds “get” this. When dividing players into two teams, the two players doing the dividing alternate – because the best players get chosen first. The average mileage of the two most gas efficient cars is better than the average of the ten most gas efficient. The top 10% of any college applicant pool has a stronger record of achievement (on average) than does the top 50%.
Truth rating of their claim that going deeper into the pool won't require lowering standards even further: Completely false. Race differences in qualifications on many campuses are already large; do we really think we are solving anything by making them even larger?
----
The first entry in this series, which includes summaries of our WSJ editorial, and the C&M-P critique, and our response to their false claim that we adopted a colorblind ideology, can be found here.
The third entry, on the informativeness of race and the nature of microaggressions (training, reporting, etc.) can be found here.
P.S. I invited Carter to write a guest blog response to my debunking of her critique, but she declined.
