Verified by 
[continued from Part 1 and Part 2]
In my previous blog post, I discussed some of the problems related to ABA, an intensive training regimen consisting of an elaborate system of rewards to make children comply with external directives and to memorize and engage in very specific behaviors.
To any of these concerns—and all others, for that matter—proponents have a single prepared response, which could be paraphrased as follows: "ABA, and ABA alone, is evidence-based. You may find it distasteful, but it works. The only alternative for a low-functioning autistic child is institutionalization or a terrible life."[1] This is not only what parents are told; it is the comeback to all concerns that might be raised by a trainer:
[I asked,] "Why does he have to have quiet hands? He’s not hurting anyone." "Why can’t we just find out what’s bothering him and help him find a solution?" "Why do we need to track that he knows 1000 words when he obviously knows way more than that?" Every time I would question their methods or their reasoning, my questions would be answered with some variation of, “This is the only evidence-based treatment for autism. It’s the only way they can learn.”
The uncomfortable irony is that we are apparently supposed to accept such appeals to "evidence" on faith. I have written elsewhere about how research cited in the field of education sometimes doesn't stand up to close examination. This is particularly true of traditional practices rooted in behaviorism—not only ABA and similar interventions for children with special needs but also highly scripted direct instruction of discrete facts and skills in early childhood (and beyond) and explicit phonics-based strategies for teaching reading.
You might assume that those who use the phrase "evidence-based practice" (EBP) are offering a testable claim, asserting that the practices in question are supported by good data. In reality, the phrase is more of an all-purpose honorific, wielded to silence dissent, intimidate critics, and imply that anyone who criticizes what they're doing is rejecting science itself.[2] It's reminiscent of the way a religious leader might declare that what we've been told to do is "God's will": End of discussion.
Moreover—and it took me a while to catch on to this—behaviorists often use "EBP" just as a shorthand for the practices they like, in contrast to the (progressive or humanistic) approaches they may revile. It may not matter if the evidence is actually weak or ambiguous or even if it points in the other direction. They'll often come up with some reason to dismiss those inconvenient findings because their method is "evidence-based" by definition.
Which brings us back to that new research review. The work of eleven authors—including, interestingly, an ABA therapist—representing the University of Texas, Boston College, Vanderbilt, and Mount Holyoke, it was published in January 2020 in Psychological Bulletin (PB), a prestigious social science journal that specializes in lengthy integrative research reviews. The article is not a polemic; it confines itself to describing peer-reviewed research. The authors cast a wide net, looking for every English-language study in the last half-century that compared an intervention group with a control group in treating children up to age 8 who had been diagnosed with Autism Spectrum Disorder. This yielded 1,615 separate results from 150 reports representing 6,240 participants.
The most striking finding in this research review is how few high-quality assessments of ABA have ever been conducted. In fact, the great majority of ABA studies were so poorly designed that they didn't merit inclusion in this review. Rather than comparing the results of different treatments for groups of children, behaviorist journals commonly publish single-subject studies, in which one child is assessed before and after treatment. (This method was invented by behaviorists back when their behavior-shaping efforts were limited to lab rats.) You don't have to be a trained data analyst to see the serious limitations of this method in terms of the results' lack of generalizability. For the authors of the PB review, these limitations were so glaring that it didn't even make sense for them to bother with the results of single-subject studies. Yet those dubious results are the primary basis for behaviorists' claims that ABA is "evidence-based."
Have there been any group-design studies of ABA? Yes, and the reviewers looked at them carefully. Three findings stand out. First, the best way to conclude with any confidence that different outcomes are due to an intervention and not to pre-existing differences between the members of the groups is to randomly assign subjects to either the treatment condition or the control group. But so few ABA studies did this that it was impossible for the reviewers to calculate an effect size for any outcome.
Second, more than three out of five of the ABA studies judged its effectiveness based on what was reported by a parent or teacher—another serious methodological flaw. "Parents and teachers are virtually always aware of the extent and nature of a child's participation in an intervention study," the reviewers explained. "Moreover, they are likely to be personally invested in the outcome." This experimental design routinely makes treatments appear more effective than they really are—to the point that, according to earlier research, some caregivers will happily report that the training was successful even when the child didn't even get the training.
Third, the reviewers pointed out that many autism studies rely on very specific outcome measures. It's a lot easier to claim success at producing "context-bound" behaviors—those that show up only in a particular setting—and those "that reflect what was directly addressed in the intervention" ("proximal outcomes"). This is related to the generalization problem I mentioned earlier: Even when ABA seems to work, it often succeeds only at getting a kid to do one specific thing or do it in one specific environment.
The PB review also evaluated several other types of intervention. It found that solid evidence supports other approaches, notably those that are not only different from, but virtually the polar opposite of, ABA. One with a particularly strong theoretical base and clinical track record is DIR/Floortime, developed by the late child psychiatrist Stanley Greenspan, whose priority was "to avoid focusing only on changing the behavior." The DIR model emphasizes emotional development, individual differences, and trusting relationships. Whereas ABA is about doing things to autistic children, DIR is about working with them. (For more details, see this book, this article, and this website.[3])
The comprehensive new PB review concludes by pronouncing two types of intervention "promising"—and ABA was not one of them. (In fact, the reviewers urged insurance companies to consider covering these two data-backed models.) One is a hybrid called Naturalistic Developmental Behavioral Interventions and the other is DIR/Floortime. Incidentally, almost four out of every five studies of the latter used a random-assignment methodology, so not only was the evidence in its favor impressive, but the quality of that evidence was high.
It appears, then, that these other approaches have a claim to being "evidence-based" that is stronger than ABA's. But there's a larger issue here to keep in mind, particularly when considering behavioral interventions. Whenever a strategy (in any domain) is said to "work" or be "effective," it's always worth asking how those words are being used. Many criteria for measuring effectiveness in autism studies—even rigorously conducted studies—are based on changing surface behaviors. Consider the anguish of a writer named Maxfield Sparrow:
What looks like progress is happening at the expense of the child’s sense of self, comfort, feelings of safety, ability to love who they are, stress levels, and more. The outward appearance is of improvement, but with classic ABA therapy, that outward improvement is married to a dramatic increase in internal anxiety and suffering ... I was once an Autistic child and I ... still carry emotional scars decades later. It doesn’t matter whether the perpetrator is a therapist, a teacher, a parent, or an age-peer: bullying is bullying.
And now we know that it isn't even evidence-based bullying.
NOTES
1. Whenever a practice or school of thought is criticized, supporters frequently respond not by considering the merits of the criticism but by asserting that the critic must have encountered an inferior version, the implication being that there's no reason to question the approach itself. It is true that multiple versions of ABA exist, and presumably individual practitioners with varying levels of expertise advertise their services using the same (insurance-approved) label. Nevertheless, as far as I can tell, all of the concerns I have raised here apply to the core principles of ABA itself as it is supposed to be practiced.
2. There are indeed people who reject science, of course—from climate-change deniers to anti-vaxxers. But behaviorism represents a reductive, experience-denying caricature of science that is still trapped in the century-old ideology of logical positivism. It gives real science a bad name.
3. Of course, there are a number of other models, schools, and programs for children with autism that I haven't mentioned here because of space limitations and/or the fact that I haven't investigated them. Also, one can find plenty of humanistic, practical advice online that steers clear of behavior control, such as this discussion of how parents might deal with meltdowns and aggression.
