Verified by 
By Blair Cox and Emily Balcetis
The Graduate Record Exam died this year.
The GRE, for short, has been required for admission to graduate programs for decades. But this year, because of a confluence of factors, far fewer universities required its scores from their applicants. In fact, over 370 natural sciences programs dropped it as a required component [see the list here]. Why? In large part because of unfairness. But will dropping the GRE reduce the bias that so many universities were trying to address? New data suggests it may not.
The GRE has a sordid history with respect to disparities in test scores that track with minority racial and ethnic status. Even the company that designed the test acknowledges that fact.
Disparities in test scores might be made worse as a result of COVID. Because of quarantine, the GRE became an at-home experience in 2020. The company plans to keep the at-home testing option available moving forward. However, not everyone has equal access to computers with webcams, reliable internet access, and quiet testing environments needed to perform at their best or even take the test at all.
For this and other reasons, many universities including ours—New York University—dropped the GRE as a requirement for graduate school applications this year.
The weaker reliance on standardized scores is a step to recognizing and working to reduce inequality in the graduate application process. But, there is still a problem.
Removing the GRE might just be shifting the source of bias to another element of the application—and that is research experience.
Beyond GRE scores, applicants to graduate STEM programs also compile prior research experiences. They detail their work in developing theories, designing studies, analyzing data, and disseminating their results. This seems like a fair and just way of assessing applicants’ readiness for graduate-level research and training and understanding of what that commitment entails. To a reviewer, it might feel like an indicator of effort, preparation, and investment in the field. But as new data we analyzed from our own program suggests, access to research training opportunities may be systematically unequal.
Here's what we found.
My lab ran a summer research internship for students interested in gaining skills in social psychology. Over an 8-week period, in New York City, interns created behavioral science experiments, learned data analytic techniques, received mentorship for graduate school applications, met with guest speakers who presented cutting edge research, participated in conferences, and published their work.
In 2011, the internship was in its infancy. For three years, it had no funding but a lot of committed mentors who were volunteering their time to train these students. Anyone from any school anywhere in the world could apply. But this lack of funding meant that the interns who accepted the invitation to join would have to foot the cost of living in the city, commuting, and compensate for lost summer wages they could have received by working other part-time jobs.
In 2014, the program received a seed award that partially subsidized expenses for students. It paid interns a travel stipend to get to and return from New York City. The award also covered the cost of their lunches several times a week. It offset the costs to interns…a little.
But the next three years were dramatically different. From 2015-2017, the National Science Foundation funded the program fully. As the program director, the grant we received felt like hitting the jackpot. The dollar amount for each year was eight times larger than the seed funding, and 75% of the award went directly to covering the interns’ expenses. The majority of the remaining funds supported Ph.D. candidates who helped train the interns. This meant that the funding could make an experience that some interns might find impossible to afford, now realizable. The award fully covered interns’ housing costs, a university meal plan, conference fees for their travel after the internship ended, all lunches, and research grant awards. It also provided them with a weekly salary at a living wage for their 8 weeks on campus.
Because the funding source required it, we could only accept applications from individuals who were American citizens or permanent residents. But they could be living or attending university anywhere. This likely reduced the total number of applications our program would have received if we were allowed to maintain the all-access policy we had first implemented.
Of course, funding upped the total number of applications submitted. We received over 11 times as many applications overall when the program was fully funded compared to the final year it was unfunded. That is, in 2013 when the program had no funding, we saw 126 applications. The next year, when it was partially funded, we received 322. And in the next three years when it was fully funded, we received an average of over 1400 applications each year.
Having that grant made a big impact on the number of people who could now afford and wanted to apply for the opportunity to gain experience in the field.
But even more importantly, the rate of increase was markedly different for people from different racial and ethnic groups. We investigated the impact of funding interns on the increase in applications we received from White and Asian people. We also looked at applications from Black, Hispanic, Latin, and native Hawaiian and Pacific Islander people. We divided our applicant pool into these two groups: well-represented and underrepresented minority.
White and Asian individuals, in general, are well represented in the sciences and math—the areas our program focused training in—but the others are not. According to a 2018 report from the Pew Research Center, 65% of all employed people are White, but 69% of people employed in STEM careers are White; 6% of employed people are Asian, but they constitute 13% of people working in STEM. On the other hand, though 11% of employed people are Black, they are only 9% of the STEM workforce; 16% of employed people are Hispanic, but only 7% of Hispanic people work in STEM.
This way of grouping individuals aligned with the National Science Foundation’s definition of what it means to be underrepresented in the sciences [read here]. But, we recognize some problems with this system. Asian-identifying individuals too are people of color. There are sizable differences within the Asian American identity. In fact, income inequality is rising at the fastest rates within individuals that identify as Asian because of these differences, new research finds. And we also know that grouping Asian-identifying individuals with majority racial groups perpetuates the model minority stereotype, erasing the very real experiences of discrimination that Asian individuals do face. We acknowledge that we can't speak to these nuances.
That said, in our data, we found that the number of applications from White and Asian people that we received increased by a factor of nearly 10 (9.9 to be precise) when comparing the final unfunded year to the fully funded yearly average. Specifically, the number of applications rose to 919 from 93.
What caught our eye though was that the number of applications from underrepresented minority individuals that we received increased by a factor of almost 15 (again, to be precise 14.8). The number rose to 488 on average from 33. (You can check out the data yourself in OSF). Offering funding meant that the rate of increase in applicants from underrepresented minorities skyrocketed.
Of course, this analysis does not consider underrepresentation in many other forms. The impact on members of LGBTQ+, economically disadvantaged, and disabled groups aren’t considered here. We didn’t look at gender or the effects on those who are the first in their family to earn an undergraduate degree. We recognize our analysis does nothing to speak to the experiences members of these groups had when considering whether to apply.
But here are the things we can say. Internships that require interns to cover their own cost of living and lost wages are not affordable for all students in training. Members of underrepresented racial and ethnic groups may not be able to finance their own participation in these programs. This creates a systemic inequality in access to learning opportunities, professional development, and mentorship. If academic program administrators fail to recognize the influence of funding on access to learning opportunities, racial disparities will continue to exist in scientific training.
Another thing to consider is how information about opportunities spreads. Yes, we changed the funding structure of our program over the years. But it is possible that the advertising and direct outreach to historically black and Hispanic-serving institutions we did spread widely and to more diverse audiences through channels we weren't aware of when the program offered compensation. Access to information about opportunities to gain multiple and varied research experiences may very well be unequally distributed.
The implications are great, as internships beget more opportunities. Students with more research experience are more likely to be selected for inclusion in other labs. They are the ones offered advanced training. They have access to letters of recommendation from more faculty who can attest to their skills. They are the students who are accepted at higher rates into graduate training programs and are offered the fellowship packages to fund their studies. If we take a critical look at the racial makeup of our STEM workforce and find it lacking with respect to diversity, we should turn to these early years of training and the disparities that exist in our research training pipeline programs.
The GRE died this year. It might be resurrected in the future. We don’t know. But what we are certain of is that if we commit to providing substantive funding for our training programs, we can reduce one systemic barrier to the involvement of students of color in academia.
Blair Cox is a Ph.D. candidate at the New York University, Steinhardt School’s Psychology and Social Intervention Program.
