Verified by 
This is one of a series of blog posts about a controversial review in a high-profile journal claiming serious negative health consequences for women if they obtained an abortion. This one evaluates the evidence mustered in the review for the strongly worded conclusion that abortion puts women's mental health at risk.
The most downloaded article from the October 2011 British Journal of Psychiatry is a review of the negative mental health effects of abortion by American Priscilla Coleman. The article has prompted an extraordinary number of e-letters, almost all of them highly critical. Anti-choice websites, apparently alerted in advance of the article's publication, created an immediate buzz about it across the US. The article was quickly sent to the Florida state legislature in support of a bill restricting abortion and the article was introduced as evidence in the appeal of a Wisconsin court decision ruling against necessity of warning women seeking abortion of the risk of suicide they face.
Professor Coleman unwittingly provides a clue to a standard we can apply in evaluating her review and conclusions.
"In this highly politicised area of research it is imperative for researchers to apply scientifically based evaluation standards in a systematic, unbiased manner when synthesising and critiquing research findings. If not, authors open themselves up to accusations of shifting standards based on conclusions aligned with a particular political viewpoint. Moreover, the results may be dangerously misleading and result in misinformation guiding the practice of abortion."
Thank you, Professor Coleman. Having been so alerted, will we find your results... "dangerously misleading" and that they "result in misinformation guiding the practice of abortion"? In this blog post we will consider whether Professor Coleman meets her own proposed standard of providing "scientifically based evaluation standards in a systematic, unbiased manner when synthesizing and critiquing research findings."
A close read of the article immediately suggests a strong bias including a preference for her own extremely weak studies to the exclusion of others, nonsensical comparisons, and misinterpretation of basis statistics.
Although there is a vast literature concerning mental health effects of abortion, Coleman selects only 22 studies, 11 of them her own. She indicated that she has excluded other studies as being too poorly designed, but she fails to identify which studies were excluded and specifically why.
Although randomized clinical trials of the gold standard for evaluating the effects of an intervention on mental and physical health, it is not surprising that no randomized trials are available. We cannot imagine a situation in which large numbers of women with unwanted pregnancies and considering an abortion would agree to commit to a choice of action based on a flip of a coin, to either obtain the abortion or continue the unwanted pregnancy. So, we are forced to rely on comparisons of women with unwanted pregnancies who either did or did not get abortion. Such studies do not allow us to definitively establish that any mental health differences between these two group of women are entirely due whether they obtained an abortion, rather than some other aspects of their backgrounds or circumstances. But hopefully we can assemble sufficient numbers of large enough groups of such women with enough background information on them so that we can take it into account in deciding whether abortion per se is related to their mental health afterwards.
In this context, it is useful to see how a group convened by the Royal College of Psychiatrists views the evidence must by Coleman.
Coleman's article came out shortly before the expected release of the full report on the mental health consequences of abortions by the Royal College of Psychiatrists. Fortunately, anyone interested can access a draft copy of that report from the Internet. The long document is quite clear and excruciatingly detailed in describing the methods leading up to its conclusions, with a precise description of the search strategy used to locate studies so that anyone could replicate it; identification of which studies were included and which were excluded; how studies were evaluated; and the strength of the evidence for the final conclusions. In striking contrast, the Coleman article has no such transparency and we are left either to accept her conclusions or puzzle how she reads them.
The Steering Group for the Royal College report consisted of 11 members, mostly physicians, and all were required to declare before each meeting any conflicts of interest, not only financial, but also whether they had made public statements for or against abortion or held any membership in organizations that had public views or direct interest in abortion.
Back to Coleman's review, the Royal College of Psychiatrists report indicates that in their evaluation of the literature, they considered a number of studies conducted by Coleman herself and included in her review. They considered all of the Coleman studies that they reviewed to be methodologically poor. One of her own him him papers that she included in her own review was rejected as having no usable data. A number of her papers were rejected because they had inappropriate control/comparison groups and still other papers were rejected because they used inappropriate measures of mental health after the abortion. In addition to its other limitations, still another Coleman paper was rejected because of a lack of statistical control for mental health prior to the abortion. The Royal College of Psychiatrists report did include one of Coleman's papers, but with serious criticism. You can check all this out for yourself by examining Appendix 6 of the Royal College of Psychiatrists report.
What is the issue with Coleman's choice of comparisons/control groups? She relied heavily on comparisons between women with received an abortion for unwanted pregnancy and women who had completed a pregnancy that was planned or wanted. Of course, this strategy cannot get at the effects of abortion because presumably the circumstances of having an unwanted or unplanned pregnancy are different from the circumstances of completing a wanted pregnancy. It is not surprising that women having a wanted pregnancy have better mental health than women who did not want to be pregnant. Do you think the Coleman knew what she was doing and proceeded anyway? Even before the Royal College of Psychiatrists report, Coleman's work had received serious criticism and summary rejection, so that she had to be aware of criticism before including the studies in her review.
Another criticism of Coleman's review is that she often did not control for prior mental health. So, if a woman smoked marijuana or consumed alcohol after an abortion, she was compared to women who dealing with a newborn child, and any differences were attributed to the first group of women having had an abortion, even in situations where ascertainment of drug or alcohol use occurred before the abortion.
Coleman either deliberately did reckless things with her statistical calculations or did not understand basic statistics. For instance, near the end of the results section of her review, she states that the population attributable risk (PAR) associated with abortion is 10% of all mental health problems and 34.9% of all suicides in women of reproductive age. PAR is intended to represent the proportion of such problems that can be causally attributed to abortion. Coleman's interpretations are ridiculed in the E-letter responses to her article, but I think you can see that they are patently absurd. If you need assistance in interpreting PAR, please consult the the E-letter responses for excellent, succinct explanations of what is wrong with what Coleman did.
If you examine Figures 1 and 2 in Coleman's review, you can see that she counts each of her own studies multiple times in her calculation of the effects attributable to abortion. This practice was also roundly criticized in the E-letter responses to her article because each study should only be entered once, if the conditions are met for integrating results of studies in a meta-analysis and providing a test of the statistical significance of the resulting effect size. This may sound like a technical point, but it is something quite basic and taught in any Meta-Analysis 101.
Coleman's calculation of overall effect sizes for the negative mental health effects of abortion involve integrating multiple effects obtained from the same flawed studies into a single effect size that cannot accurately characterize any of the individual effects - anxiety, depression, substance abuse, and suicide - that went into it. Again we are encountering a nonsensical statistic. In meta-analysis workshops and basic courses, it has become standard practice to include a slide allegedly of a road sign announcing that you are entering New Cuyama, California on which a population of 562, an elevation of 2150 feet, and a date of establishment of 1951 are summed up to a total of a total of 4663. This is meant to be a spoof of the meaninglessness of combining diverse outcomes into a single summary effect size.
The draft Royal College of Psychiatrists report contradicts Coleman by concluding that mental health outcomes are likely to be the same, whether women with unwanted pregnancies offered abortion or birth. This is the same conclusion reached by two previous reviews, one an American Psychological Association Task Force report and one an article in Contraception that were similarly harsh in their evaluation of Coleman's work.
Just what was Coleman trying to accomplish in publishing a flawed review that is fundamentally an exercise in pseudoscience? Anyone capable of using the software that Coleman applied in conducting her meta analysis should be aware of the multiple errors she made in entering data and interpreting her results. Near the end of her article, Coleman reveals her intention to affect public policy by burdening women interested in terminating an unwanted pregnancy with the necessity of first being warned about the dangers of abortion, albeit with false information generated by Coleman's review:
Until sound evidence documenting mental health benefits of abortion is available, clinicians should convey the current state of uncertaintyrelated to benefits of abortion in addition to sharing the most accurate information pertaining to statistically validated risks.
This is call for a required warning is consistent with what anti-choice activists are attempting to require in states where they lack the political strength for an outright ban on abortions. And as soon as Coleman's review was posted on the British Journal of Psychiatry website, it was sent to state legislatures.
In my next blog post, I will discuss some of the background of Priscilla Coleman and one of her co-authors and speculate about how her article ever got published in the prestigious British Journal of Psychiatry. Stay tuned and follow me on Twitter @CoyneoftheRealm in the meantime.
