The Power of Nuance: Lessons for Public Health (with Emily Oster)

Emily Oster: Thanks for having me. I always love to talk to you.

Emily Oster: Yeah, I mean, when you look at the landscape of information about vaccines, there’s a tremendous amount of information suggesting safety and efficacy of childhood vaccinations, which is what we’re talking about with measles. The measles vaccine in particular has been around for many, many decades, has saved literally millions of lives. I think it’s easy to forget that things like measles and pertussis and the basic stuff we vaccinate kids for, they kill a lot of people. They still kill a lot of people, and they certainly killed a lot of people before vaccines. And, this is a space where it’s both we have a lot of evidence of safety. There’s a lot of reason to think that high vaccination rates are important.

So, measles is actually something that lives on surfaces, and so it is really, really contagious. You need a very high vaccination rate to generate herd immunity; but the vaccine itself is incredibly protective. So, it’s really a very good vaccine.

And then, we have this sort of basic misinformation about links between, say, the measles vaccine and autism, which it’s not just, like: Well, different people could see things differently. It’s, like, the evidence on which that idea is based was literally made up. It’s made up. Somebody made it up in the service of money, basically. And, we can talk more about that if you want. But the study that initially linked those was just not correct. And, there’s tons of data after that from large scale data sets that show that there’s no link there.

So, this is a place where I think that the evidence on both benefits and the evidence on risks is really clear that vaccination for measles and for pertussis and some of these early childhood vaccines are a really good idea. Full stop.

Emily Oster: Yeah, absolutely. I think it’s, again: if we’re in the space of nuance, I think it’s important that we not say, no one has ever [?] through a vaccine or that your child isn’t going to get a fever. Many kids do get a fever about 10 days after the measles vaccine. That is a very common side effect. So, I think we don’t want to pretend that literally there could never be any–that there are no possible side effects of something. Because, that’s not true, and because then people don’t trust you. But, at the same time, I think we need to give the message that those side effects are small. They are limited. The things that you’re worried about, like the big picture, that’s not supported in the data. And, there’s a lot of reason why you should want your kid to have this vaccine so they don’t get the measles. Which you may well have had in 1955. The measles vaccine is from the 1960s.

Emily Oster: So, most of the milk that people in the United States consume is pasteurized, and pasteurization kills pathogens. It’s a procedure that was developed by Louis Pasteur. It’s also actually a miracle of modern science because it allows us to transport and store milk safely.

So, in the pre-pasteurization era, milk would be transported fairly long distances without pasteurization, and it caused a lot of diseases, like large tuberculosis outbreaks associated with milk. And so, I think it is not an exaggeration to say that this also saved many millions of lives as a result of having this pasteurization.

Some people don’t like pasteurized milk. They prefer raw milk. Raw milk has an interesting regulatory space in America. So, you cannot transport raw milk, which is literally just unpasteurized milk. So, think about it comes out of the cow, you put it in a bottle: that’s raw milk. It cannot be transported across state lines. So, you can’t milk your cow in Rhode Island and sell it in Massachusetts. But, different states have different rules about whether you can sell raw milk at the farm, not sell it at all, sell it in stores. There’s a variety of–the regulatory landscape is very complicated.

When you look at the data, it is definitely true that raw milk is more likely to cause disease than pasteurized milk. So, if you look at foodborne outbreaks of illness from dairy, they are disproportionately–relative to the consumption amounts, they are disproportionately in raw milk. And, it’s sort of clear why that would be: When you milk the cow, there might be poop around because there’s poop, and that can get in the milk, that can cause E. coli. If you pasteurize the milk, that deals with that. If you don’t pasteurize the milk, you may have that problem. So, it’s not surprising to learn that raw milk would cause more disease.

Having said that–and this is the sort of nuance of the point–it’s not that likely that raw milk will cause disease. So, the risk, if you count up the chance and number of cases and how much foodborne illness we see from it, it’s not a spectacularly large number. It’s within the kinds of risk numbers that people would take in other aspects of their lives–some people. So, if you said, ‘Is it a crazy thing? Would only a crazy person drink raw milk?’ I don’t think so. Is it the safest form of milk to drink from an illness perspective? No. The safest form of milk to drink is pasteurized milk. And, that’s why that’s most of what we sell.

But, I think there’s a sort of–this is the point I was trying to make in the Times essay–I think to tell people that ‘choosing to drink raw milk from a local farm is just as problematic as choosing not to vaccinate your kid for measles,’ I don’t think that those things are comparable in the quality of the evidence that you’d be basing that on.

Emily Oster: I don’t think so.

Emily Oster: This is a little–people tell you all kinds of stuff about raw milk: it has this feature and that feature. I don’t think those things are real well-supported in the data. Either people, some people–I mean maybe this is what you’re saying–some people like the way it tastes more, and so I think it does taste different.

Emily Oster: Yes, there are many places where people sell raw–I mean, most of Europe, you can get raw milk.

Emily Oster: Yeah. And, I think that there’s something that you gain by having some nuance here, which is this ability to make it clearer to people. What are the circumstances that would make–if you said, ‘This isn’t the, quote, “safest choice” from an illness standpoint’–if we totally dismiss the concept and say, ‘Well, this is incredibly dangerous and no one should do it,’ we sort of miss the opportunity to explain to people: ‘You know, here are some things you want to be thoughtful about.’ Like, ‘You want to get this from a trusted source. You want to go to a farm where you know that they’re washing things correctly before they put the milk in.’ And, those kind of lessons are not really possible if we’re not communicating in a nuanced way.

Emily Oster: Yeah. And, I mean, that is absolutely right. And I think it’s something that I struggle with because my whole, like, brand and feeling is, like, we should be more nuanced. And, I do very much take the point that some people would just like to be told what–would be like to be told what to do.

I continue to think we have too many–we spend too much time with the view that everyone wants to be told what to do. In a world in which people are doing their own research. Which they are. Not all of them, but some people–

Emily Oster: a good chunk more than used to are doing their own research, whatever that means. And, in the world in which people are doing their own research, you need to give them information to help them do that smarter. Right?

And I think that’s just the reality. We have to–as public health communication, we have to respect the fact that people are looking into what we’re saying and they’re trying to figure out what is the right thing. And, giving them better information to make that choice is part of the job of public health.

Emily Oster: It’s complicated–

Emily Oster: No, I don’t think that. I mean, I guess the way I would see it is I think that if we’re going to encourage nuance and thoughtfulness in decisions, we also have to help people think about priorities and sort of what decisions are really worth. Like, where there’s a real trade-off or there’s a real, kind of, either a very complicated upside/downside, or a very big/small, maybe a small probability, a very big possible downside. Like, what are the choices where we want–we actually really want–people to be able to engage with them where it matters a lot for either themselves or for public health, what they do?

And then, what are the choices where, like, it’s not that important?

And so it’s not–like, we could tell people, ‘Here’s what we recommend, here’s what we don’t recommend,’ but we don’t want to have them that be the decision they invest a lot in.

Let me give you an example of this; and so, I do a lot of parenting stuff and I talk to, sort of, pediatricians about how we are communicating to parents. And, one of the things about being a new parent is you get, like a–parents are looking to do the right thing. That’s, like, their main thing. And, there’s a tremendous amount of advice and things you’re told where you’re told what to do.

And some of those things, it’s actually pretty important to make–or at least there’s a lot of evidence–that one choice would be medically a better idea than another. Like, for example, vaccines.

And then, there are many things where the pediatrician may say, ‘Here’s what I recommend, but actually it really didn’t matter.’ So, starting solids. Like, should you start with purees or baby-led weaning? This is a thing that comes up. Should you give your kid purees? Should you give them solids? It’s really not important. It’s, like, whatever. Both are fine.

And so, often pediatricians would be like, ‘This is what you should do.’ Because they think people are looking for an answer.

But, actually, that’s a place where it’s not so much that you want nuance, you just want to say, ‘Here are the two choices. Either is fine, just pick one. If you want me to tell you which one to pick, I’ll tell you which one to pick, but it doesn’t really matter.’

And then, get them focused on, like, if you really want to dig into something where you want to make a nuanced decision, here are the actual important things to think about. Here’s a bunch of stuff you really just, kind of, anything’s fine.

Emily Oster: I think of these three examples, fluoride is by far the most complicated on a bunch of dimensions. Partly the data is hard to understand for people, and partly just the whole conversation is very complicated. But, let’s–we can back up and parcel it out a bit.

So, in general, fluoride is good for protecting your teeth. So, if you ask, ‘What do we know about fluoride in general?’ It’s, like, these fluoride rinses, fluoride toothpaste–there’s a lot of evidence that that prevents cavities in kids and in adults.

And, cavities turn out to be bad for you. They’re bad because your teeth hurt and you need–but actually they’re kind of broadly bad. It’s not good for you to have a lot of untreated cavities.

So, in order to address the so inequity of access to fluoride, there has, for a long time been in many places–in Israel, also most places in the United States–there’s municipal water fluoride systems. Fluoride is added to municipal water supplies so people get access to fluoride with the hope that that will improve dental health.

The evidence supporting the improvement of dental health there I think is kind of reasonably good. Actually, some of the best evidence comes from Israel, where you stopped putting fluoride in the water and then a lot more kids had bad cavities. So, we see some, I think reasonable quality evidence, especially with the existing knowledge about fluoride that actually this approach improves teeth, improves dental health.

The concern that people have is that fluoride in water can cause neurodevelopmental problems when consumed by pregnant women, and to some extent by kids. That, like, fluoride is a toxin that affects your brain. And, what’s hard about this is it is true that at very high levels that that’s the case.

So, there are places–mostly China, India–where the groundwater fluoride levels are very high: like, five times as high as what you’d see in municipal water supplies in the United States. So, not like just a little higher, but much, much higher. But, there, you do see, at those levels there’s some evidence of negative impacts–not like it’s spectacularly large, but there’s some neurodevelopmental stuff that you’re seeing. We don’t see that in the data that looks at fluoride levels that are closer to, that are what we see in the United States.

But, that’s where, in some sense, that takeaway, if you are a person who spends a lot of time with data is: The dose matters here. That, at low doses, this seems to be good for protecting teeth and doesn’t seem to have any negative effects. At high doses, it seems to have negative effects. But then, you’re having an argument or a discussion about what’s the right dose: what’s the point at which it becomes too much? And that’s actually a really hard, nuanced question.

And, I’ll give you one example of why this is so hard.

So, somebody wrote to me the other day and they were, like, ‘What about this study?’. And, this was not, like, just somebody, but this was, like, a reporter from a major newspaper. They were, like: ‘What about this study that shows that concentrations at this level, which is the same as the level in the United States water supplies, shows that that’s negative?’ And, I opened the paper and I was, like: ‘Well, this is not measuring the water supply level. This is measuring the urinary fluoride level, which turns out to be basically half of the water.’ It was, like, something was so in the weeds about how are we measuring fluoride in these different studies? And that’s where I think you get into trouble–where it’s actually very hard to communicate nuance because there’s so much nuance, and we cannot possibly expect everybody to be an expert on the difference between municipal water fluoride levels and urinary fluoridation levels. That’s just a weird, esoteric thing that people don’t want to invest in.

And, because you can’t say fluoride is totally fine at any level, and because it’s also clear that probably is fine at some level, we’re arguing something in the middle that’s tough. And, I think that’s why this conversation has gotten so confusing.

Emily Oster: Yeah. I think it’s not–so, when I wrote this piece, a number of people were, like, accused me of sanewashing RFK [RFK Jr.–Econlib Ed.]. And, I think many of the things that RFK Jr. has said are–really do not make any sense and–

Emily Oster: Not the topic of this conversation–

Emily Oster: But, I mean, I do think in this case–it’s my belief having read the data and thought carefully about it–is that we should be fluoridating municipal water supplies. In part because I think that actually the people who would suffer from taking it out are largely the most vulnerable individuals because people with, like, good access to dentistry–people whose kids are going to the dentist all the time–they’re going to get these fluoride rinses or other ways that they’re getting fluoride. The value of putting it in the water is that it enables it to be available even to groups that are otherwise not getting as much access.

So, I think it’s a good idea, but I do understand why people want to talk about it. And, I think that’s kind of a–I don’t know, that’s just a tension.

Emily Oster: Their teeth are worse.

Emily Oster: I think it’s true. I think there’s a limit to how much fluoride you need to improve your dental health.

Emily Oster: I often think that if people were robots, that view would be fine. So, if your view was basically: the way that people are reacting to information is that they hear what you say and they do the thing you said. And, in a world which everyone is the same–like has the same preferences–it would be fine to just say, ‘Here is what we advise.’ And, let me actually–so my husband has some academic work on this where they think about, in his case, he’s thinking about how should you show data to economists? How much should we express our results as either the answer is two? or, here are all the data points?

In some sense, you could ask that question if people are really economists here, like: why is the bin scatter so great? That’s the source of the question.

But what is the amount of information we want to provide people? And, I think that their key insight is that it matters a lot, whether everybody has the same preferences–everybody’s interested in doing the same thing with the data.

In a world in which people’s preferences are heterogeneous, which I think is the world we live in, then it’s already a little tricky for there to be one piece of advice. Even if you thought everybody was going to do it, you might actually get wrong on average, if you said, ‘I, public health person, I’m going to take my preferences, apply the data to my preferences and tell you what to do.’ Well, if you’re not a person with my preferences, that’s going to be wrong.

I think that’s one problem; and that’s one reason why nuance can help people. If we explain, ‘Here are the costs, here are the benefits,’ they can combine that with their preferences to think about the right choices.

I think the second piece–which is probably more practically important–is that people aren’t going to listen to you. So, if you say, ‘You have to do this,’–okay, here is an example, Russ: co-sleeping. So, in the United States, we tell people, ‘Do not co-sleep with your baby. It’s very dangerous. That’s the sleep advice, do not co-sleep.’

Emily Oster: Don’t have your baby in your bed. Don’t have your baby in your bed: It’s very dangerous. That’s a very clear piece of advice that everybody gets.

Co-sleeping is incredibly common. Many people–very high shares of people–co-sleep with their babies, and that’s for a bunch of different reasons. And so, that advice, you could say, ‘Well, we just want to give that advice because that’s the best advice.’ But, the thing is that people aren’t listening to it and they’re doing something else. And then, you’ve lost the opportunity to explain to them, say, what’s a safer way to do this? Or to give them some more context for what circumstances might make this more or less risky. You’ve basically made the outcome worse by trying to tell people, ‘Do this, do this, do this,’ or in this case, don’t do it. And then, they don’t listen; and then they do it in a way that is less safe.

So, I think that nuance can be an opportunity for us to help people make the best choices, even if those best choices almost like aren’t the first-best. It is like a way to help people make a second-best choice.

But, the flip side of that, and I think this is important to acknowledge, is that: if you introduce this nuance, there will be some people who behave in the way that you don’t think is the right way. So, if we say, ‘Generally we don’t recommend co-sleeping, but if you’re going to co-sleep, here are the safest ways to do it,’ you are probably going to get a little bit of an uptick in the number of people who are co-sleeping. And, you need to trade that off with the fact that everyone or that people will be doing it in a safer way. And, that that has some benefits. And again, we’re economists: We’re very comfortable with the idea of there are risks, there are benefits, this is not necessarily a Pareto-improvement. I think that that’s much harder sometimes in public health.

Emily Oster: Yes. I agree with that. I think we also erode trust. I think there’s this sort of truth piece, but I think we also erode trust by not explaining uncertainty. And maybe that’s part of what you mean. But, I think in this kind of messaging, very, very unwilling to say: Here’s what we know now, and here’s what we don’t know, and here’s how we hope to learn. And, I mean, this came up–when we talk about COVID if you want–but this came up a lot in COVID when, like, information was coming out at all times and public health officials were changing their advice quite frequently, but never really explaining, like: Why? Like: What new information did you learn that made you do this?

And that’s a way that people–changing your mind without explaining is really a way to lose people’s trust, because they’re, like, ‘You told me to do a thing before. That turned out to be wrong. Why is this thing right now?’ And, I think if we had said, ‘We’re not sure. Here’s how we’re going to learn about it more, and we’ll come back and tell you later what we’ve learned and maybe it will change,’ I think that would have been a way to pull more people along.

Emily Oster: Fair enough–

Emily Oster: I agree. I think it’s hard. But, I also think that that explaining is a skill.

Emily Oster: And, it is something that could be developed and invested in.

And, one of the pieces of the advice–if anyone were ever to ask me, ‘What would you have public health invest in?’ I think one of the things I would have them invest in is this kind of, like, translation and explanation. How can you make it vivid to people? How can you explain uncertainty? How can you help them understand data? That it’s not magic: but it is hard. And it’s a different skill than producing the research. It’s explaining the research.

And, I don’t think it’s crazy to imagine that being something that public health authorities invested in learning more about and figuring out. What resonates with people? Do they like graphs? Hope so. Built my whole life out on that idea.

Emily Oster: Yep. Totally.

Emily Oster: I totally agree. And I think in fact, we way underestimate how much kids could learn this stuff. I think, like–I have a talk I give about where does data come from? Which is about–I start with: How do we know what share of people in America are overweight? People are like, ‘Well, we weigh them all.’ Well, that can’t be right. And then, you dive into–and I’ve given a version of that talk to 11-year-olds and to the Brown Corporation. And, if it’s, like, you can give that talk to 11-year-olds, they have great ideas. It’s like, ‘Well, how would you figure this out?’ Somebody’s, like, ‘Well, how would you get 30,000 people? Would you weigh people at a football game?’ And, someone else is, like, ‘Well, that’s stupid. Those are a particular kind of people.’ And, you’re like, ‘Yeah, okay.’ So, you can bring people along there. And that’s something actually kids are really good at. Sometimes better than adults.

Emily Oster: Nothing that I have seen since then has changed my mind. I mean, I wrote that piece with the title “Schools Aren’t Super-Spreaders” in October of 2020. And, I think that basically all of the–we were basing that on some data from Europe, some data we’d gotten from the United States, some of the early just basic early observations about what was going on in schools that were open.

Subsequent data–some of which was from us, some of which was from more careful, complicated studies–basically completely proved that out. Not that no one ever got COVID at schools, but that particularly that first school year when schools were closed, the schools that were open did not spread a lot of COVID.

So, I think on that, I just think that the data ended up–the data we had at that time–was supported by all the other data that came out later.

The thing that, even for me has been surprising, is how problematic the school closures have been long-term. So, looking at what were the downsides for kids.

Part of the reason I pushed this so hard in October of 2020 was it already seemed to me like this was really going to be very bad for kids and bad for their learning and probably bad for many other things. I think that’s proved to be right. I’ve continued a project in the wake of the pandemic about what’s happened to test scores. And, you know, test scores–every place, but especially in places that had closed schools–took an enormous hit during the pandemic. And they basically haven’t recovered.

So, like, they’re crawling back, crawling back. But, we have not seen a recovery. And it’s now many school years from then. And, I think that there are cohorts of kids who are going to be affected forever.

And so, I just think this was an even bigger mistake than I thought it was at the time. And, I think by and large, people have come around to that view. Certainly relative to the amount of hate I got in October of 2020, I think there are many more people who would now say, ‘Well, actually, you were right.’

Emily Oster: I mean, I think that there’s a far–maybe there’s a sort of far-left contingent on this. I mean, look, I make people angry on all sides of the spectrum. It’s one of my special talents.

But, on the particular issue of school closures: Yes, I think that there’s a set of people on the Left who still feel that school closures were a good idea. Maybe that they should have been closed longer, that if only we had shut everything down and done nothing, no one would have gotten COVID. And, I don’t think that data supports that. I also think that position has become more fringe.

For me, part of what was tough in the early parts of this were many people thought that this position, was crazy. You know, like, government official, like: ‘This position of we should open schools,’ was basically something that, like, a huge share of people were, like, ‘You’re not right.’ Not so much on the right, but a huge share of the Left, Center-Left, thought that this was not true. I think that has become much less true. Many people who at the time–

Emily Oster: said, ‘You’re wrong,’ have now come around. You’re always going to have–you know, whatever. There’s always going to be people who don’t like me. It’s okay.

Emily Oster: Yeah. No, and I think it really sort of illustrated, like, that way of thinking–that there are trade-offs and that you would want to think about the sides of the trade-offs, and in this case, think about the educational costs and think about how those are spread across social groups and whatever it is that there would be trade-offs–that that idea was just not–there was a very clear, almost lexicographic view that was, like, if one person got COVID at school and died, it would not have been worth it to open schools for everyone.

And, I think at some points, I got the sense that that’s kind of what people believe–

Emily Oster: That, if I told you I can open the schools–I can open all the schools 100% and give kids exactly this, but one additional elderly person will die–they would be, like, ‘That’s not worth it.’

And I think that’s a kind of view about this, that one could have; but it isn’t the view that I had. And I also think it would have been useful to say, ‘Well, that’s how we’re thinking about the trade-offs.’

I mean, the other thing that made me–that I found really complicated and problematic about this–was there was a kind of component of this, which is: you’re trying to hurt poor families who are going to be the ones who are most likely to have COVID. The answer is, like: Those kids are also the ones who lost the most. They’re the ones who are the most likely to have school closures, and they lost the most from school closures. This was not a simple thing.

And, the other thing I’ll just say on a personal level is that people would–at that time, a lot of people were, like, ‘You just want your kids out of the house.’ Like, ‘You’re just advocating for open schools because you don’t like having your kids around.’ And, I wanted to say, ‘Look, I am a person with resources. My kids go to private school. They were in school September 4, 2020–in school five days a week for the entire year. I am not doing this for my kids.’ First of all, I didn’t want them to have them in my house; but I love them very much. I just thought school was a better place for them. But the thing–this was about advocating for people who didn’t have those kind of resources. Not advocating for something personal. And I thought that was a really odd–that was a very frustrating interaction.

Emily Oster: There was this moment for me, Russ, in the spring of–maybe in May or something of 2020 where I got it. I was writing a lot about grandparents and daycare and this trade-offs for people, and I got an email from someone. It’s still, when I think about it, basically makes me cry. And it was from a grandma, and she said–she was, like, ‘Before COVID, I was spending a day–I have a two-month-old grandson, and before this, I was spending a day-a-week taking care of him, and it was the greatest thing ever. And, now my daughter doesn’t want me to see him because she’s afraid that I’m going to get sick.’ And she said, ‘I would rather die than not have this time with my grandkid.’ And this is–and it was just, like, this moment. Like: Oh my God, this is like this is this person’s–like, we are really missing something if we think–it was just a moment of: We’re really missing something.

Emily Oster: It’s interesting. I wrote to this person–so, a couple of years later, I was just like, this is something, it is an email I think about a lot. And, I wrote to this person–I had written to her, back to her at the time, and then I wrote back again. I was, like, ‘I just wanted to know what happened.’

And then it was very nice, because she wrote back. She was, like, ‘Yes, surely after that email, my daughter relented because we needed the child care. And, actually it was totally great, and now we have another grandkid,’ and everything. So, it was a very nice, sort of–it ended up being very nice. But I think it was a moment of realizing that’s a trade-off I want to make because this is–I don’t know, because there’s trade-offs.

Emily Oster: Yeah, I mean, I think we have not learned much. I think that–I continue to think the current messaging around COVID vaccines needs to be a little more targeted. I think we’re way under-emphasizing the value for older adults of getting consistent boosters and probably overestimating the value to younger people, at the cost. And the result is nobody’s getting them.

I think at the time, the conversation about masking was very complicated and probably could have been more nuanced. I mean, for me, the schools are the biggest policy failure, but that’s partly because that’s the one that I was most–like, I know the most about.

And then, of course, the fact that we failed to protect nursing homes, which were the source of such an enormous share of the actual deaths. I mean, I think that was just a–like, many terrible things happened there, which we could have focused more on.

Emily Oster: So, the idea is–this is in partnership with The Free Press, and the idea is to talk about some of these big parenting issues that have been coming up in the last couple of years that people are hearing a lot about–phones. Should I let my kid out more? Why is there so much mental health issues with kids? What are we feeding our kids? Why isn’t anyone having kids? And, to really make–I mean, the podcast is quite heavily produced. We talk to a lot of different people, try to get many different perspectives.

We have an episode on discipline in which we talk to everybody from the gentle parenting/never-punish side, all the way to a guy who spanks his kids, and trying to understand the differences across those perspectives. And, is there anything that’s bringing them together, and just–it’s basically about perspective, I would say.

Emily Oster: It feels like a tremendous amount of responsibility because I want to help people make good choices, but also huge privilege and a very good use of my talents relative to what I was doing before.

Emily Oster: Thanks, Russ.