Is Screen Time Bad for Teens?

A few years ago, two Oxford researchers made headlines by declaring that screen time was just as bad for teen mental health as … potatoes. The paper is routinely cited as evidence that social media doesn’t matter for mental health, and even to argue that connecting depression to social media use amounts to a moral panic.

Except: That paper didn’t examine social media use specifically; it looked at screen time overall, including watching TV and simply owning a computer. It even counted talking on the phone as screen time.

The picture is very different when you zoom in on social media. In one of the datasets used by the Oxford researchers, for example, others have found that heavy users of social media are twice as likely to be depressed as non-users.

The Oxford researchers argued that combining all screen time and all mental health measures was the most objective approach. In some fields, that might have been correct. But with the diversity of different types of screen time, lumping everything together muddied the waters.

In addition, the Oxford paper includes some analytical choices that are extremely strange, yet escaped the notice of the many researchers citing their paper and all of the journalists that covered it. My colleagues and I didn’t discover them until we ran the statistical code in an attempt to replicate their analyses in a new paper. But these decisions change everything.

One scale, 73% of the data

When psychologists want to assess mental health and well-being, they usually use questionnaires that ask about happiness, life satisfaction, symptoms of depression, and so on. Some of these scales ask people to report on their own feelings (known as self-report), while others ask someone else to report on another person’s feelings.

The Millennium Cohort Study, the highest quality of the datasets the Oxford researchers analyzed, includes 4 scales assessing mental health. One involves a parent reporting their view of their teen child’s mental health, and the other 3 involve the teen self-reporting on his/her own mental health.

If a researcher wanted to know how social media use was associated with mental health across all 4 measures, it would make sense to report the average or the median correlation by treating all 4 scales equally. After all, that’s how averages work: All datapoints are treated the same.

But that’s not what the Oxford researchers did. Instead, they entered the results for the parent-report scale in 8 different ways: the total scale, 5 subscales, and 2 combinations of subscales. In contrast, they included each of the 3 teen-report scales only once. That meant 8 out of 11 scales (73% of their data) were from a single parent-report scale.

Even most experts wouldn’t catch this odd choice on a first or second reading of the paper; it was included in a table in the supplemental materials. I couldn’t find a single researcher who cited the Oxford paper who made note of this odd choice. It also wasn’t mentioned in any of the popular press articles.

This decision had a big impact. Two Spanish researchers have found in a new paper that the Oxford study’s conclusions were “severely misleading,” partially because of this choice to make the parent scale into 8 scales instead of one. That’s because teens’ heavy social media use was strongly linked to worse mental health when teens self-reported, but was linked to slightly better mental health when parents reported on teens’ mental health. Averaged together, it looked like social media use was not linked to mental health, because the results canceled each other out – especially because the parent-report scale constituted 73% of the Oxford researchers’ analyses.

What happens when you count each of the 4 scales equally? The link between social media use and mental health is suddenly much larger.

This also solved a mystery. Back in 2020, we theorized that the Oxford researchers’ results were different because they included single items from the scales and random combinations of those items. Because the parent report scale arbitrarily has more items on it (25, vs. 13 for the self-report depression scale, 5 for self-esteem, and 6 for life satisfaction), that might have skewed the results. But they reported in their reply that the results were the same when only scales were included. Now we know why: 8 out of the 11 scales were the parent-report measure, which produces opposite results than the teen-report scales.

Internet Addiction Essential Reads

Does Excessive Screen Time Change Adolescents' Brains?

How to Set Screen Time Rules for Your Family

Lumping all screen time together

The Oxford authors also averaged across different screen activities, which included TV as well as more recent inventions like electronic devices and social media. That also obscured significant effects. For example, the Spanish researchers found that hours spent watching TV was not associated with suicidal thoughts, but electronic device use was – in fact, nearly twice as many heavy users had suicidal thoughts than light users in the Youth Risk Behavior Surveillance dataset. When these results were combined, they looked like a very small effect.

“Averaging over technologies ... gives [results] that may appear practically irrelevant,” the Spanish researchers wrote, which “misled Orben et al. (2019) to their conclusion that technology has no relevant association with well-being, whereas we argue that 1.88 higher odds of thinking about suicide are definitely practically relevant.”

In the Millennium Cohort Study, the Spanish researchers found that social media and internet use were strongly linked to mental health issues, while TV and gaming were only weakly linked. Using a related but different approach, my co-authors and I found the same: Social media and internet use, especially among girls, had the biggest effects. A 2019 paper by Yvonne Kelly using this dataset had also come to this conclusion, finding that twice as many boys, and three times as many girls, were depressed at high levels of social media use compared to no use.

We also made a few other analytic decisions different from the Oxford paper. For example, they controlled for variables (like feeling distant from parents) that might actually explain why heavy social media use is linked to depression -- and experts on these types of analyses strongly advise against controlling for variables like this. They also controlled for other variables that overlapped significantly with mental health issues (like a measure of negative attitudes toward school that included the item “How often do you feel unhappy at school?”) Not surprisingly, if you control for a measure of unhappiness when examining depression, links appear weaker.

What does this mean in practical terms?

Thus, three independent research groups — the Spanish researchers, my colleagues and I, and Yvonne Kelly and her co-authors — all found substantial links between certain types of screen time and mental health issues among teens in the same datasets that the Oxford researchers concluded had no meaningful effects.

So what is a meaningful effect? Since the Oxford researchers decided to compare the correlations between screen time and mental health to other teen experiences measured in the same datasets like wearing glasses, riding a bicycle, and eating potatoes, we thought we should do the same.

As it turns out, the association between poor mental health and social media use among girls is larger than the association between mental health and binge drinking, hard drug use, marijuana use, lack of exercise, early sexual activity, being suspended from school, being stopped by police, and carrying a weapon. That does not mean these unwise activities are safer than social media use, or that social media use is worse than these activities. It means that if we’re going to dismiss social media’s links to poor mental health for small statistical associations, we’ll also have to dismiss a long list of activities routinely targeted for public health interventions.

So, if you’re trying to figure out why the research on screen time and mental health has been so confusing, here is one reason: One of the most highly cited papers treats a single scale as 3/4 of the data and averages different types of screen time together when very meaningful differences exist. As the Spanish researchers put it, Specification Curve Analysis (the averaging technique used by the Oxford researchers) “is a tool intended to prevent selective reporting. Alas, when used for inference it can create severe biases and false positives.” Although the idea behind SCA was to present unbiased analyses, it still relies on making analytic decisions, and some of those analytic decisions were questionable.

It’s tempting to conclude that because researchers disagree, we don’t know anything about screen time and mental health. Actually, we now know more. Science advances as researchers debate. It’s also reassuring that the Spanish researchers independently came to similar conclusions as we did, even though our statistical and theoretical approaches were different.

Not only that, but we are closer to agreeing. One of the Oxford researchers, Amy Orben, published a review paper in 2020 noting that correlations with mental health were larger for social media, often between .10 to .15 – and that was for males and females combined. We found it to be between .10 to .20 for girls, depending on the quality of the mental health measure. That's pretty close. Where we disagree is whether that's large enough to be meaningful. We think it is. For context, the correlation between childhood lead exposure and adult IQ is -.11.

It’s time to lay to rest the idea that screen time doesn’t matter for mental health. It does. It’s just that some screen time matters more than others. For mental health, social media use, especially among girls, may be the most problematic.

Practically speaking, that is good news. Parents, policymakers, and teens themselves don’t need to consider giving up technology; instead, they can focus on cutting back on social media time (and perhaps general internet use). Teens can find other ways to stay in touch with friends virtually, like FaceTime or Skype, texting, or gaming together.

In sum: Looking at all screen time obscures the substantial link between social media use and mental health among teens. If that’s like potatoes, it’s high time to lay off the French fries.