Whose Expert Guidance on Risk and Reopenings Should You Follow?

Earlier this summer the New York Times published a story “When 511 Epidemiologists Expect to Fly, Hug and Do 18 Other Everyday Activities Again.” I was not one of them. Perhaps my invitation ended up in my spam folder…it's hard to tell when the methodology of the survey was not described in sufficient detail. But I digress.

The article states:

“Their answers are not guidelines for the public, and incorporate respondents’ individual life circumstances, risk tolerance and expectations about when there will be widespread testing, contact tracing, treatment and vaccination for Covid-19. They said it’s these things that will determine their actions, because the virus sets the timeline....

Still, as policymakers lift restrictions and protests break out nationwide over police brutality, epidemiologists must make their own decisions about what they will do, despite the uncertainty -- just like everyone else. They are more likely, though, to be immersed in the data about Covid-19 and have training on the dynamics of infectious disease and how to think about risk."

While there’s a lot of language in this quote that resonates with me, if you’ve read my prior post about making decisions in the face of uncertainty you know that the way I make decisions is through a risk-benefit framework. And perhaps that is what the Times was hoping to accomplish with this piece, but the problem is that at no point do they actually ask these experts the specific trade-offs they are willing to make to stay "safe."

If you say that you are “Never” willing to hug or shake hands with a friend (as 6% of epidemiologists apparently say), then you are also saying you will never again comfort a grieving friend. Or greet a stranger during religious services as a sign of faith. Presumably, they will never again high-five someone in celebration, as these are just advanced forms of handshakes. What are the specific trade-offs that these behaviors will entail?

In my opinion (irony intended), the Times did not conduct a survey of expert opinion. It conducted a litmus test of whether you are the type of person who worries about shark attacks when you go to the beach or worries about a hijacking when you get on a plane. You don’t need any expertise in epidemiology to give a "valid" response to these questions because the responses have little, if anything, to do with the expertise these people have.

Now, a survey of this type — not of any particular occupation, but of attitudes and perceptions around the safety of resuming activities — does have the potential to provide meaningful information. Similar to metrics like "consumer confidence," these attitude surveys can help monitor changes over time and how those changes are correlated with actual risk or behaviors. (There's a nice piece on 538 related to this idea.)

If I were designing such a survey, in addition to asking the questions the Times asked about behaviors, I would have asked the following:

1. Have you been tested for COVID?
2. Have you tested positive for COVID?
3. Has someone you know been tested for COVID?
4. Has someone you know tested positive for COVID?
5. To what degree do you agree with the statement: Whether I get exposed to COVID is completely within my control.
6. On a scale of 0 to 100, where 0 is that it definitely will not happen, and 100 is that it will certainly happen, how likely are you to test positive for COVID in the next month?
7. Overall, I expect more good things to happen to me than bad.

I’d want to know the answers to questions 1 through 4 because researchers have long known that the beliefs are shaped by experience. If the people who happen to be in this survey were much more closely affected by the epidemic than public health professionals in general, their opinions on these behaviors would likely be biased towards being more conservative. Moreover, as articulated in Daniel Kanheman in Thinking Fast and Slow, the recall of said experience is influenced by the context in which it is assessed.

Health Essential Reads

The Patient Safety Risk Model

Is Vitamin D Supplementation an Evidence-Based Practice?

I’d want to know the answer to question 5 to get a sense of how much they think their exposure risk is a function of their own (controllable) behaviors versus the ecological nature of epidemics. Presumably if you think your behavior is the major determinant of your risk, and risk is to be avoided, you will be less willing to engage in behaviors that put you at risk as compared to whether you think the behavior of other people (i.e., whether they wear masks, whether they wash hands, etc.) or some other externality (i.e., God, state officials) is the major determinant of your risk.

I’d want to know the answer to question 6 to get a sense of whether they think their personal risk of exposure is higher or lower than what is in reality, based on what we know about the epidemic so far and the probability of exposure in their local area. To estimate the latter I would compare their answer to the current prevalence of exposure based on testing data.

And finally, I’d want to know the answer to question 7 as a rough way to figure out if they are more optimistic or pessimistic — not just about COVID, but about life in general. This is an item from the Life Orientation Test - Revised which you can access here. Numerous studies have shown that dispositional factors like optimism are correlated with a wide range of health-related behaviors, from awareness of health risks to seeking health care.

And my hypothesis would be that the responses to this final item (item 7) would be the strongest predictor of under what conditions (if any) people will send their children to school for in-person instruction, or get on a plane, or shake hands, or visit their aging loved ones. While data can help us estimate some of the values in our internal risk-benefit calculator, there is simply too much dynamic uncertainty about this pandemic to rely on data alone. And the "weights" that apply in my risk-benefit calculator may well be different than the ones that operate in yours, but I honestly doubt that is because I am an epidemiologist.

The takeaway here is that data is cheap, but information is priceless. What is the difference? Information empowers your understanding and decision-making. Data, especially data without sufficient context, risks impairing it.