Verified by 
In graduate school, as part of my never-ending-dissertation-writing-effort, I ran a simple experiment in the laboratory. My aim was to search for ways to increase task performance so I was constantly recruiting participants and giving them assignments and measuring their timely task completion or performance (how many times did they spot a grammatical error? How many times were they able to solve the simple algebra question?)
In one particular study, I hypothesized that people would be procrastinating less in groups. I had people working on tasks alone; and I had people working on similar tasks in groups. I gave them essays to proofread, I waited, I corrected their essays and paid them from my very small experimental funding. As it turns out, people did not work more in groups (procrastinators will remain so no matter how many people watch them) and my precious funding was out the door. Of course this made me mad, both on professional and personal levels. No one publishes a negative result; editors are interested in seeing what works rather than things that don't work. More importantly, on a personal level, when an experiment didn't work, it delayed my graduation date by a year.
Even though this particular hypothesis was without merit, I experienced something noteworthy during that study. I had about 60 participants, and their task was to proofread 30-page essays over a three week period. The essays were computer generated and they were grammatically correct but inherently meaningless and extremely boring. Here is a sample text:
"In the works of Joyce, a predominant concept is the distinction between ground and figure. Marx uses the term ‘feminism' to denote not, in fact, narrative, but postnarrative. Therefore, Scuglia[4] suggests that we have to choose between subtextual discourse and precultural discourse. Foucault suggests the use of Sartreist existentialism to challenge sexual identity. It could be said that if subtextual discourse holds, we have to choose between Sartreist existentialism and the materialist paradigm of context."
I agree that this is a bit tricky, since the sentences look like they might mean something. The reason for giving the subjects a boring and meaningless task was to measure the time and effort they provided for each task. Each essay involved a number of artificially inserted spelling errors (everything about this study is artificial, I know) and I was paying participants based on the number of errors they caught.
Experimental protocol requires that if withholding information is necessary during the study, participants need to be briefed once the study is over. Their informed consent gives them the right to withhold the data collected from their behavior. So if and when a subject learns that the procedure involved meaningless tasks and therefore asks to be confidentially removed from data collection, he has to be granted the right. The debriefing session also gives the experimenter the opportunity to talk to participants and collect exit surveys if necessary.
At the end of the above study, I met with the subjects personally, made their payment, explained the nature of the study and reminded them their rights. I asked them what they thought about the essays. How interesting was it? How much time did they spend on the task? Would they do it again?
All subjects but one told me that they got bored with these essays, and they didn't understand anything (naturally). However, one of my participants said she learned "a lot." I was surprised, of course. Especially since the essays were absolutely empty mingling of words. This participant was a graduate student and she was carrying books when she came to the debriefing session. I had no doubt that she read a lot. I asked her what she was studying. She said she was a PhD student in comparative literature.
I know that it is not possible to make any conclusions with one data point, because science needs generalizations. But luckily stories and anecdotes and blogs do not. (Like Philip Roth said, Politics generalizes, art particularizes.) So why did that literature student feel the need to claim that she learned something?
Before I deliberate, I would like to state that the computer did not randomly generate something that made sense. In case you are a cynic, this wasn't the case of monkeys typing up Shakespeare.
What was going on was an inability on the student's part to admit any ignorance on her area of expertise. Experts, it seems, are sometimes hurt by their familiarity with their subject because they have illusions of knowledge. Two researchers, Son and Kornell (2010) wrote about the perils of too much information and how expertise might lead to overconfidence.
One of their studies asks mathematics and history professors to categorize famous names in their fields. For example, a mathematics professor is given the following statement: "Mathematician-Johannes de Groot." The professor is asked to assess whether de Groot is a mathematician. They could choose from one of the three responses: Yes, No, Don't Know. A mathematics professor is also given the names of historians and athletes. There are questions from all three categories, but the professor is an expert in only one category. So the mathematics professor also faces the quesion "Athlete- Lance Armstrong." Then again choose one of the responses. Yes, No, Don't Know.
The catch in the study is that some of the names are made up. The question looks like this: "Mathematician- Benoit Thoron." Since there is no such person in the world of mathematics, the correct answer is "No." The cautious answer is "Don't know." However, experts said "I don't know" fewer times when the question was in their area of expertise. And instead of admitting they didn't know, they answered "Yes" more often to made-up names. For example, mathematicians said "Yes" 19 times to made-up mathematicians and 7 times to made-up historians. Son and Kornell state that "experts were fooled into endorsing falsehoods because they failed to admit that they did not know."
Isn't this unfortunate? The number of PhDs and experts are increasing every year. We hear affirmative statements from experts all the time, possibly underestimating their overconfidence.
This is unfortunate on another level: Benoit Thoron could very well have been a mathematician's name. It not only sounds serious, it is French. It rhymes with énumération.
Mais bon. I can't write anything more because I'm not an expert in French mathematician names.
