Study: Is Complex Problem Solving Distinct From IQ?

A screenshot from the downloadable game, Genetics Lab, designed to test complex problem solving

The idea that skills other than traditional IQ might matter more than testable intelligence in the real world of messy, ever-changing problems isn't a new one. But what is new is the developing alternative of testing complex problem solving. An article just published in the journal Intelligence tries to pick apart the difference: instead of answering questions about trains leaving stations, vocabulary and block stacking patterns to measure IQ, the researchers propose “computer-based problem-solving scenarios called microworlds” to assess higher-order thinking skills. Designing these microworlds requires looking inside what makes higher-order thinking and, in fact, what goes on inside complex problem solving itself.

See, questions on IQ tests have answers: donkey and horse are to mule as lion and tiger are to liger. (Okay, I kind of love the liger, but that’s beside the point.) But in the real world, problems can consist of many, interconnected variables whose relationships are obscured and change over time, and goals can be unclear or even competing and contradictory – factors the authors of the Intelligence study cite as central to complex problem solving.

But pulling apart these ideas – traditional definitions of intelligence from complex problem solving – is tricky. As you’d expect, there’s significant overlap: people who are intelligent are generally good at CPS and vice a versa. But in what ways do these skills diverge? How is the intelligence measured by IQ and trained in school different from the skills of complex problem solving needed in the real world?

To answer that question researchers stuck 563 Luxembourgian high-schoolers in a genetics lab – or, at least in front of a free, downloadable microworld called Genetics Lab, which you should definitely check out if you have time. In GL, test-takers turn on and off “genes” and then click “next day” to discover how the combinations affect a fictional creature’s traits like IQ, Inventions and Ideas. It turns out to be a bit of a mish-mash and sorting out which genes create which abilities is a strange and complex soup.

Like many problems in the real world, GL not only requires manipulating information on the way to a goal, but sallying forth into the microworld to discover this information in the first place and using the information to create a mental model of how the system works. So Genetics Lab is scored three ways: Do students manipulate Genetics Lab in a way that creates unambiguous rules? Can they understand the relationships between genes and characteristics? And only finally, can they manipulate genes to get target characteristics?

Students were studious. Researchers researched. And then the researchers compared students’ three GL scores to their IQs, grades and whole bunch of other measures. What they found is cool, but only when you look at it with a bit of nuance. See, they found that Genetics Lab measures the same thing as the reasoning section of the standard IQ test. D’oh!

But, “Although they might not measure something different from reasoning scales, they measure it differently,” the authors write. And by measuring this reasoning or problem solving or whatever you want to call it differently, the researchers were able to see not only the endpoints but the waypoints that led there – they could measure problem solving process and not simply its product, and so answer not only the question of who is and is not good at reasoning/problem-solving, but what got them there.

Gathering information, making a mental model of how information drives results, and finally putting this model to work to get the results you want: that’s the process of complex problem solving. At this point, it ‘aint yet been pulled as a skill distinct from IQ-like reasoning. But a peek inside this process shows how results happen -- and as a parent or teacher know this process may allow you to help your kids learn this complex and important skill.