Verified by 
*First author is Mary Phillips
A proliferation of “Sleep Clinics” are charging large fees to teach parents how to train infants and young children to sleep. “Sleep therapists” advocate for the use of various interventions despite the absence of credible longitudinal studies and gold-standard empirical research. We review a summary of the “Best Literature” provided by the American Academy of Sleep Medicine (AASM).
A researcher who was consulted about sleep training research indicated that the following article was the “best review of the literature.” The article was published in the Journal of Pediatric Sleep (volume 29, number 10, 2006). The title of the review was: “Behavioral Treatment of Bedtime Problems and Night Wakings in Infants and Young Children: An Academy of Sleep Medicine Review.” It includes a summary review of 52 articles supporting the intervention of “crying it out” for infants and young children. Let’s check it out.
Of the 52 articles reviewed only 11 studies were considered to have good data—they met Levels I & II as established by the AASM Classification for Evidence. (Table 3 in Research Summary) Note that the remaining 41 studies were considered to have poor data—they were rated as Levels III, IV, and V with recommended grades of “C” by the AASM. These studies are designated as Case studies/series: poor case control (Level III), review of case controlled (Level 1V) and Opinions (Level V). These 41 studies of the 52 are not included in this discussion (though they are included in the conclusion, see below).
For the purpose of this discussion, the focus is on the 11 studies that met Level I Classification for Evidence (9 studies) and Level II Classification for Evidence (2 studies).
The review stated, “The research provided strong empirical evidence.” Yet, empirical evidence requires objective evidence that can be measured, tested, verified and replicated. Contrary to their conclusion, all 11 studies used subjective measurements (parental reports/diaries), an unreliable source of data. (See their chart at end of the their Reference List)
Furthermore, their chart notes (last column) that 6 of the 11 studies had at least one data problem:
1) no fidelity check (to determine if interventions were followed) and/or
2) no objective measurements and/or
3) no inter-reliability check for diagnosis.
These factors (fidelity issues, the many sources of possible bias in parental reports, and the lack of inter-reliability checks) for any study make it impossible to reach accurate conclusions about the effectiveness of the research, let alone generalize it to any other group. The multiple flaws in these studies noted by the authors, compromise the reliability and validity of the research and should prevent any conclusion that an intervention is effective.
But that did not stop the reviewers from concluding that “The research provided strong empirical evidence” for cry-it-out or total extinction (the term they use is “unmodified extinction”).
But there are more issues that the reviewers did not address:
- The age range of the children studied varied widely from 3 months to 59 months.
- Developmental issues and differences were not addressed.
- Two of the 11 studies had restrictive populations – one with breast-fed infants only, and one with pre-school children only and cannot be generalized to other populations. (66 & 105 in reference list)
- No standardized or operational definition of “sleep problems” was used in the studies. All definitions were defined by parents as subjective complaints. These definitions will be influenced by parent educational level, psychopathology, parenting styles and family dynamics.
- There was no consensus on diagnostic criteria.
- Durability (maintenance of sleep training outcome) was reported as 5% at one year later. The researchers stated, “the durability of the treatment effects should be interpreted with caution”. Average follow up was less than 6 months.
- No longitudinal studies have been done to determine outcomes of interventions, including effects on child wellbeing.
With all these problems in study design and data collection, the strong conclusions reached by the reviewers must be called out. They are unethical. Despite stating earlier that only 11 of the studies met the AASM classification levels I & II (Table 3), the reviewers included all 52 studies to make the statement, “94% report improvements due to behavioral interventions.”
Summary
The 11 studies presented in this best review contained multiple research flaws which compromised the reliability, validity and integrity of the studies. Effectiveness of the interventions is questionable due to the many possible sources of bias in subjective measurements. Six of the 11 studies did not have treatment fidelity checks. Four of the studies did not have checks for inter-reliability of diagnosis. Due to the unreliability of treatment interventions, the possibility of many sources of bias, and absence of inter-reliability of diagnosis we cannot make accurate conclusions about the research.
Despite the multiple flaws which compromised the reliability, validity and integrity of the studies, the reviewers concluded that the research provided “strong empirical evidence to support the intervention”. The fidelity issues and many sources of possible bias in parental reports/diaries make it impossible to reach accurate conclusions about the effectiveness of the intervention. Yet, the reviewers endorsed the intervention in spite of the flawed research.
Any research that concludes that it is safe and good for babies to be treated in these manners must be longitudinal and attendant to all possible negative outcomes for the child into adulthood. Because the research conducted is far from this standard and even with low standards is multiply flawed, it is unethical to recommend cry-it-out/unmodified extinction/total extinction sleep training to address sleep problems and night wakings.
NOTE: This review was guided by these notions: (1) The burden of proof is always on the person making the claim; (2) Here, the reviewers made the claim that “strong empirical research supported the use of behavioral sleep interventions in infants and children”. (3) The data provided was not empirical. In addition the integrity, reliability and validity of the studies was compromised. We could not make accurate conclusions about the effectiveness of the interventions. (4) Therefore, the burden of proof was not met and we cannot ethically recommend the interventions.
*Mary Phillips is an Associate Professor of Developmental Psychology in Virginia, a Licensed Professional Counselor (LPC), a National Board Certified Counselor (NCC), and a Certified Cognitive Therapist.
Links to other posts on sleep training and young children's sleep:
6 Hidden Myths Behind Baby Sleep Training Advocacy
Child Sleep Training’s “Best Review of Research”
Parents Misled by Cry-It-Out Sleep Training Reports
REBUTTAL to critique of "Parents Misled by...Sleep Training Reports"
Baby Sleep Training: Mistakes “Experts” and Parents Make
'Let Crying Babes Lie'? So Wrong
Simple Ways to Calm a Crying Baby
Normal, Human Infant Sleep: Feeding Method and Development
Normal Infant Sleep: Changing Patterns
Normal Parent Behaviors and Why They Won’t Hurt Your Child
Normal Infant Sleep: Night Nursing's Importance
More Normal Parenting for Sleep
Understanding and Helping Toddler Sleep
Understanding and Helping Toddler Sleep-Tiredness?
Understanding and Helping Toddler Sleep--Preparing Success
