Verified by 
Filed Under: Writing Wrongs
Statistical Problems in Scientific Papers
Science News editor Tom Siegfried writes a fascinating feature on how research papers are often plagued by "the shortcomings of statistics." As he states up top, some science is "more like a crapshoot" than like, well, science:
Even when performed correctly, statistical tests are widely misunderstood and frequently misinterpreted. As a result, countless conclusions in the scientific literature are erroneous, and tests of medical dangers or treatments are often contradictory and confusing.
Turns out there's a history of scientific literature about poor scientific literature. The problem gets complex, but part of it stems from papers that confuse their "statistically significant" results with practical significance. A research finding becomes "statistically significant" when its p-value falls below .05, which basically means there was less than a 5 percent chance the result was an "improbably fluke," as Siegfried puts it.
But there's a big difference between reporting statistical significance and using the word "significant" in its everyday sense. Siegfried points to work done by Stephen Ziliak and Deirdre N. McCloskey, authors of The Cult of Statistical Significance, who studied journals from many fields and found that psychological researchers, in particular, failed to make this critical distinction. They write:
In medicine and epidemiology and especially psychology the concluding sections [of papers] are often sizeless summaries of significant tests reported earlier in the article. Significance this, significant that.
For more technical analyses, check out posts at Marginal Revolution and Andrew Gelman.
Filed Under: Solving Religion
Randomness Is Godliness
While we're on the topic of statistical significance, a recent study published in Psychological Science found scientific evidence of God. Well not quite. But a group of psychologists from the University of Waterloo did report that people may turn to God as a way to cope with their fear of a random existence.
Study Breakdown:
- 37 subjects took a pill they were told was an herbal supplement
- Subjects were split into two pools: 1. Those told there was no side effects to the pill. 2. Those told they might experience "mild arousal or anxiety"
- The subjects then performed a word scramble; some words were related to randomness, some to negativity
- Researchers then assessed belief in the supernatural
In the end, subjects in the "no side effect" pool who unscrambled words related to randomness showed stronger beliefs in supernatural forces than those in the same pool given words about negativity. In short, subjects turned to supernatural forces when unable to attribute their heightened sense of "randomness" to something else—in this case anxiety caused by the pill. The researchers conclude that:
... belief in supernatural sources of control, such as God and karma, may function, in part, to defend against distress associated with randomness ...
Jonah Lehrer, properly skeptical of one study's ability to "solve" religion "by a clever study involving 37 teenagers," offers a nice broader take, and even finds a way to quote Bob Dylan. To which I can only add that the last time Bob played "God Knows" was April 8, 2006, in Sun City, Arizona.
Filed Under: Joining the Fray
Mad Blogging Skills
Speaking of mild anxiety, there's a new blog dedicated to the history of psychiatry, called H-Madness. Its mission, should you choose to accept it:
H-Madness is intended as a resource for scholars interested in the history of madness, mental illness and their treatment (including the history of psychiatry, psychotherapy, and clinical psychology and social work).
A quick survey finds a nice mixture of posts ranging from critiques of the DMS-V to a movie review of "Shutter Island" ("a hodge-podge of hastily drawn psychoanalytic symbols") to those, uh, written in French. C'est la psych.
(HT Neuroworld)
Filed Under: Ambi-turners
You (Kinda) Have Value
A few bites of psychiatric wisdom propel the plot of "Greenberg," the new Noah Baumbach movie starring Ben Stiller as an Honorary Headcase. Greenberg, recently released from a stint in a mental hospital, struggles to control his neurotic tendencies in the real world. In one scene he meets up with an old flame, who tells him she was once comforted by a shrink who said, "You have value." (Apparently she has subconscious urges to be an item at Wal-Mart.)
I saw the movie this weekend and can't give it the same therapeutic reassurance, despite raves by David Denby and A.O. Scott. Though I will say, in fairness, that another line passed down from a shrink works its way to a more humorous conclusion.
Filed Under: We're All Greenbergs
The Relativity of Happiness
Elizabeth Kolbert of the New Yorker uses the release of a new book on happiness research to revisit a classic psych study from a 1978 called "Lottery Winners and Accident Victims: Is Happiness Relative?"
I won't give a full study breakdown; Kolbert handles the basics sufficiently. But simply put, the researchers studied the well-being of lotto winners, accident victims, and a control group, and found the winners were no happier than the others and even took "significantly less pleasure from a series of mundane events," they reported in the Journal of Personality and Social Psychology. (That would be statistically significant less pleasure, for those keeping score at home.)
I spoke about the topic with Daniel Gilbert of Harvard, author of Stumbling on Happiness, a couple years ago, and he interpreted the relativity of happiness as a sign of human resilience:
Human resilience is really quite astonishing. ... People who suffer real tragedy and trauma typically recover more quickly than they expect to and often return to their original level of happiness, or something close to it. That's the good news ... . The bad news is that the good things that happen to us don't feel as good or last as long as we think they will. We're resilient in both directions. We rebound from distress but we also rebound from joy.
So cheer up, Greenberg, because when you do, you'll still have plenty to be upset about.
