Are you smarter than Aristotle?: On the Flynn Effect and the Aristotle Paradox

IQ Test Examiner: All bears are white where there is always snow; in Novaya Zemlya there is always snow; what color are the bears there?

Soviet Union Peasant: I have seen only black bears and I do not talk of what I have not seen.

IQ Test Examiner: But what do my words imply?

Soviet Union Peasant: If a person has not been there he can not say anything on the basis of words. If a man was 60 or 80 and had seen a white bear there and told me about it, he could be believed.

---

This peasant, hailing from a remote area of the Soviet Union in the early part of the 20th century, was interviewed by the psychologist Alexander Luria. It turns out that this peasant's way of thinking was quite common back then. This type of thinking involves reasoning based on personal experience and reference to the concrete, functional use of objects.

For some IQ subtests, this type of thinking won't get you a very good score. Indeed, this is built into the very scoring instructions of some of the most widely administered IQ tests. Take the scoring instructions for the WISC-R Similarities IQ subtest (italics are mine):

2 points are given for "pertinent general categorizations"

1 point is given for "the naming of one or more common properties orfunctions of a member of a pair (a more concrete problem-solving approach)"

0 points are given to answers that may be functional but are phrased in a more concrete way

To make this idea concrete (which would probably score me low on an IQ test!), I will use a real example. This item was previously found on an actual WISC-R:

What do liberty and justice have in common?

According to the scoring instructions, 2 points are given to those who answer that both are ideals or that both are moral rights, 1 point is given to those who say that both are "freedoms", but 0 points are given to the person who says "free things", because it's the more concrete response.

As James R. Flynn notes in his most recent book, "You are just not supposed to be preoccupied with how we use something or how much good it does you to possess it."

Let's put this in context for a moment. In my previous post, I discuss the "Flynn Effect"-the finding that IQ rose quite a bit during the 20th century. The type of IQ-test content that requires abstract generalizations showed some of the largest increases.

This poses a major paradox. If one were to work backwards, this would mean Aristotle's IQ can be estimated to have been -1000 (that's right, negative 1000). How can this be? Aristotle clearly came up with some pretty darn good ideas.

There are various ways to respond to this paradox. Some are more satisfying than others. One way is to say that there really is no paradox: On average, the people of our grandparents generation really were that less intelligent than we are today! This one isn't satisfying to me. If we took their IQ estimation at face value, we would expect that the average person from 1900 wouldn't understand a word coming out of their grandchildren's mouths. Personal experience (again this wouldn't score me a point on an IQ test) reveals that this isn't the case.

Another way to respond is to say: See, this proves that IQ tests aren't measuring anything meaningful! Again, this doesn't satisfy me. Research shows that within each generation, IQ tests predict an awful lot of things that any reasonable person would consider meaningful. They even predict some practical outcomes, such as job performance. So it's probably not fair to argue that IQ tests have no practical significance in today's world.

We could also argue that we aren't actually smarter, we are just more sophisticated test takers: We have simply become better at taking tests, thanks to better schooling. This would be all fine and dandy, if it weren't for the fact that the IQ subtests that displayed the smallest gains (such as vocabulary and general knowledge) are the ones that are most capable of being educated!

Now let's explore James R. Flynn's way out of the paradox.

According to Flynn, our ancestors thought very differently about the world. The industrial revolution brought to prominence a particular type of thought--scientific operational thought.

Peasant spectacles were probably the least scientific (e.g., most tied to experience) than others living at the same time, but on average, people back then did not all receive the same scientific instruction we do today. Once the industrial revolution brought with it a different set of demands, scientific thought flooded the classroom curriculum, and the average person became much more comfortable at hypothetical thinking and making abstract generalizations.

None of this negates the IQ test or the value of scientific thought. Today, in an era where all children are handed post-scientific spectacles, the smarter one will probably be the one who uses such spectacles. Indeed, if one views the definition of intelligence as "adaptation to the environment", people back then were adapting to the demands of their environment. The demands today are different. To be sure, such change is a good thing. The IQ increases witnessed in the 20th century certainly "represent nothing less than a liberation of the human mind" from the concrete and have "paved the way for mass education on the university level and the emergence of an intellectual cadre without whom our present civilization would be inconceivable."

Flynn's resolution of the paradox only offers an explanation as to how it can be that the average performance on subtests requiring abstract generalizations was so much lower than it is today. Today, with more opportunities for educating the scientific mind, more people hold a ticket that gives them the chance to properly display their true IQ. According to Flynn, many people may have been smart back in the day, and may have been capable of answering the IQ items requiring abstract generalizations, but may have found the items so foreign and absurd to even take the test seriously. Or they may have answered the test questions with a certain habit of mind that wouldn't have earned them a high score.

Flynn goes further by proposing a mechanism; Piaget's distinction between concrete operational and formal operational thinking. Flynn cogently shows in his book how being on the concrete level in Piaget terms can hinder your performance on an IQ test but not necessarily make you mentally handicapped. Indeed, a large amount of teenagers today haven't attained Piaget's formal level of thinking, but would we want to say that they are all mentally handicapped? Probably not. Indeed, for most of these teenagers and for our ancestors, "they were quite capable of on-the-spot problem solving in the concrete situations that dominated their lives."

Flynn also argues that most of our pre-scientific spectacle wearing ancestors in 1900 were indeed on the concrete level and that "people lacking a scientific perspective are much more likely to have their intelligence grounded on the concrete level." Flynn presents evidence in his book that item difficulty on the test that has displayed the largest gains -- The Ravens Progressive Matrices -- correlates quite well with Piagetian competence. Performance on this test requires one to look at a matrix of (presumably) never seen before pictures and determine the common pattern. Research shows that twenty items on this test in particular require the participant to be "either on the threshold of the formal level or operating on that level."

So according to Flynn, no, our grandparents weren't stupid. And no, we all aren't smarter than Aristotle (although you personally may be). In fact, Aristotle's once said that, "we are what we frequently do." Which, interestingly, is Flynn's entire point.

Keep in mind that IQ test scores are calculated relative to others taking a test. In Aristotle's day, he was the cat's pajamas. And if he took a modern day IQ test, he'd probably still do quite well. But if they had IQ tests back in Ancient Greece, the average score of the population would probably have been much lower than it is today (at least for tests involving operational scientific thought).

According to Flynn's resolution, we have to face a different set of problems today that were unheard of to our ancestors. Today, we take such thought for granted.

Are you satisfied with Flynn's resolution to the paradox? Please weigh in.

Personally, I am most satisfied with Flynn's resolution in comparison to the alternatives, but I have a few qualms.

Sure, it's easy to see how providing an answer on the verbal Similarities sub-test of the WISC-R IQ test in terms of the function or value of the two entities (such as in the liberty and justice example) can get you a lower score on that subtest. The scoring instructions do indeed damn "the concrete in favor of the abstract." It's harder for me to imagine though how this would operate on the nonverbal Ravens Progressive Matrices test which is not open ended and clearly requires one to make abstract generalizations.

Indeed when it comes to performance on the Ravens test, one can't be penalized for providing a functional response, because it's impossible to give a functional response on this test (this test is multiple choice only)! Of course, Flynn may still be right that the cause of lower performance on Ravens may be due to the general lack of education and familiarity with the form of thinking that is required to do well on the test. But "the scoring instructions are biased against that habit of mind" argument can't be applied in the case of Ravens (which is the particular test that has displayed the largest increases-- gains much larger than seen on the verbal Similarities test).

My second qualm is that people were capable of formal operational thought back then. Piaget started his research back in 1929 and found that after about age 11, people really could handle formal operational thinking. So to propose that the mechanism behind the Flynn effect is a shift from concrete operational to formal operational thought is to say that the average person back then used the same form of thinking as the average 10 year old today. I feel as though, if true, the implications of this would be interesting in its own right and may raise just as many issues as the Aristotle paradox raised!

Therefore, let's not be too quick to accept Flynn's explanation in its entirety as fact. In nearly every popular media discussion of the Flynn effect, the writer summarizes Flynn's argument and treats it as the truth. Not even Flynn is that dogmatic!

In my opinion, Flynn's major contribution is not solving the paradox definitely, but specifying the parameters of the paradox. This limits the number of reasonable alternatives. This is why I admire Flynn so much: he possesses a unique combination of philosophical and scientific thinking.

In Flynn's words, here are the parameters: "Our ancestors were not mentally retarded; yet they could not cope with a huge number of Raven's items; nor could they, as recently as those born in the 1930s, cope with a large number of Similarities items - and that we must seek an explanation in new habits of mind, rather than talk about test sophistication."

We're not done though. So far, I've only discussed one paradox raised by the Flynn Effect. But there are more.

Chew on this one for a little while:

The rise in IQ from generation to generation suggests that the average IQ score of a particular population can change through the influence of environmental factors. And quite a bit.

If this is so, how can IQ tests be meaningfully capturing anything of importance, if those things that are captured are so dependent on environmental influence?

Indeed, twin studies demonstrate the strong influence of genes in determining IQ score. And IQ tests do show pretty good stability and predict various practical outcomes within a generation.

What's going on here? In my next few posts, I'll try to work ourselves out of this mess.

© 2008 Scott Barry Kaufman, All Rights Reserved

--

Notes:

The dialogue at the beginning of this post as well as all quotes from James Flynn are taken from Flynn's recent book, What is Intelligence? Beyond the Flynn Effect. I thank James Flynn and the syndicates of Cambridge University Press for allowing me permission to reproduce the dialogue. Also thanks to Elliot Paul for his insightful comments on an earlier draft of this post.

I have been quite impressed by some of the responses to the first paradox. A few readers of my blog hit the nail right on the head. In fact, one individual personally emailed me because they were too shy to post their solution on my blog, and this person was in near perfect agreement with Flynn's hypothesis! I look forward to reading potential solutions to this next paradox. There is usually more than one interpretation of data, and some of you may hit upon something none of us intelligence researchers have ever thought of. So stay involved!