Episode description:

Welcome to In Reality, the podcast about truth, disinformation and the media with your host Eric Schurenberg, a long time journalist and media executive, now the founder of the Alliance for Trust in Media.

On In Reality, we talk a lot about the supply side of the information ecosystem, about journalism and social media and how disinformation gets spread. We talk less about the demand side—how we readers and viewers of news can trustworthy information. We’ll fix that imbalance a bit today, with a special guest, Michael Caulfield.

Caufield is a former professor at University of Washington and researcher at the Center for an Informed Public. He’s the author with Sam Wineburg of Verified, a book with the highly explanatory subtitle How to Think Straight, Get Duped Less and Make Better Decisions about What to Believe Online. The book introduces what I have found to be a highly useful, easy to remember and very quick way to quickly vet a claim you come across online. Caulfield and Wineburg call that technique by its acronym SIFT. I hope you’ll find it as handy as Eric does.

Alliance for Trust in Media
alliancefortrust.com

Produced by Tom Platts at Sound Sapien
soundsapien.com

 

Transcript

Eric (00:01.932)
Mike Caulfield, welcome to In Reality.

Mike (00:04.871)
That’s great to be here.

Eric (00:06.67)
Tell us a little bit about yourself. Let’s just start at the beginning and how you came to be concerned about what many people refer to as media literacy, finding your way to the truth in a chaotic information environment.

Mike (00:19.965)
Yeah, I mean, there’s actually a kind of a specific moment, really. I used to work at this small college, Keen State College in Hampshire. And we rolled out in, this was like 2009, 2010, we rolled out some digital literacy outcomes that we were assessing college-wide.

Then the assessments came back and this is something where we pulled out from first year student papers, we pulled out, and this is 2010-ish, pulled out all web resources that people liked. So not a book they were citing, a journal article, but anything that was a web resource that was cited in the paper, we pulled a random sample of those. And the library was assessing those.

And they gave me a call and they said, hey, you might want to come down here and look at some of this stuff. We were doing the assessment. We found some weird things here. And I went over to the library and they showed me some of the things that the students were citing. And like, you know, one of them was a website. I forget what it had something on the top, like, I don’t know, like government slaves or something like that. they’re like, you know, colloidal silver.

Eric (01:29.336)
Mm-hmm.

Mike (01:46.167)
advertisements down the side and they’re like, this, this was something that students cited on like water policy or something. And, and I thought, wow, this is actually much worse than I thought. and so, that, that began my interest in figuring out what was going on and, you know, how we could help students, to, better assess the value of the sorts of things they found on them.

Eric (02:20.11)
Okay. And out of that, I assume came the book Verified that you did with Sam Weinberg and the SIFT method, which is what I cite a lot in my University of Chicago class and so do many other people. Take us through SIFT, what the acronym stands for. And then I might ask you to use the SIFT methodology to take us through a particular piece of contested information. But let’s start with what it means.

Mike (02:51.421)
Yeah, sure. So, yeah, a little background on that. In 2016, I met Sam Weinberg and he was interested in these things and he kind of looked at the stuff that I had found over the past however many years and it was really interesting. But at that time, was just sort of a bag of lot of tricks. You had like 25 different tricks you could use to kind of find.

Truth on the Web or something like that. He gave me some really important advice. He says, you gotta get it down to something really simple. So I kept on hammering it down and go back and say, nope, it’s gotta be more simple than that. And so finally, after a lot of experimentation, I got down to four things that you can do on the web.

Sometimes you do them sequence, sometimes you do them in different order, sometimes you do them recursively, but the first one is to stop. And stop just means that when we look at something on the web and when we feel strong emotion, and I don’t mean just rage, mean surprise, like delight, or most often just sort of righteous indignation, you know, that this thing absolutely proves that we were right to stop and say, do I know what I’m looking at here? And that’s the sort of question we use. Like, do I actually know what I’m looking at or am I assuming that I know what I’m looking at? That’s where most people go wrong. Once you stop and you realize, hey, maybe I don’t know what I’m looking at, the second thing we find very helpful is investigate the source. Just again, not a Pulitzer Prize winning investigation, but just is the source the sort of source I thought it was when I first looked?

Eric (04:28.046)
Mm-hmm.

Mike (04:46.781)
You know, if I thought it was a journalist, you know, is it a reputable journalist or is it maybe an advocacy blogger? You know, if I thought it was an academic paper, is it an academic paper or is it maybe just a preprint? And maybe there’s a little bit of a difference there in how I’m going to evaluate that. And so just investigate the source. Then find better coverage if the source is not really all that you thought it was, it doesn’t mean that’s the end. You can go and you can find a better source. One of the things we tell students is a lot of things that you see from bad sources are actually true. But you still got to go out and you still got to find a source that you trust. And so Find Better Coverage just reminds people that the source that brought you this thing that you’re interested in, you don’t kind of stick

Eric (05:34.828)
Mm-hmm.

Mike (05:44.625)
that you can go somewhere else and pursue that investigation. And then finally, Trace, which is Trace claims quotes in media to the original context. And that’s just that we find on the web, a lot of things has been stripped of its initial context. So you see a video and it’s a five second video, which should kind of trigger alarm bells that is five seconds long. If you look at the 20 second video, it tells a very different story.

You see a statistic, but you don’t realize that the statistic is from 10 years ago. You trace to the source, find out that’s from a long time ago. That might have changed. So trace is the final step.

Eric (06:24.206)
Mm-hmm.

Eric (06:28.032)
Okay, all right, so that’s the acronym. S for stop, I for investigate, the source, F for find better coverage, and T for trace the content back to a source. So let’s take a particularly famous contested claim, the one that was made by the, by now president-elect Trump during the debate with his opposite candidate Kamala Harris about

Haitian immigrants in Springfield, Ohio kidnapping other residents pets and eating them. So let’s say you see that claim in the debate, you wonder whether it’s true, sort of take us through the SIFT methodology of analyzing that.

Mike (07:11.451)
Yeah, so there’s two sides to approaching this. One is if you don’t believe it, and one is if you do believe it. And part of it is, like, I think a claim like that you should immediately think, I think it’s quite right to think a claim like that is probably implausible and so forth. if you go through it, if you

Eric (07:19.95)
Mm-hmm.

Mike (07:33.391)
If you do believe it, the thing you want to do is stop and say, you know, do I know what I’m looking at here? And part of what you’re asking yourself is what am I assuming is the story here, right? What am I assuming is the case? And if you don’t believe it, you do the same thing. And then the second thing you want to do is you want to say, well, where does this story come from? Who is the person or where is the reporting that told this story about…

…people eating dogs, cats, whatever this was. Where was the original reporting on that? What was the original source? If you were to actually track that down, you’d find that it is basically rumor, right? Is rumor circulating around this town. And then if you wanted to at that point, I think you’d probably be

Eric (08:26.018)
Mm-hmm.

Mike (08:31.547)
You know, you don’t have to go through every step. think at that point you could just stop and say, look, it is a inflammatory rumor. is not plausible in my, in my opinion. And there’s not a good source. It’s not worth my time. I could just stop here. But if you wanted to, you could go and you could see, look, can I find better coverage of this issue?

Eric (08:49.646)
Hmm.

Mike (08:58.365)
In which case you’d probably find the fact checks and things like that. And that might involve you throwing in a little bit of the claim into a search bar and typing in a simple word like fact check. So, know, Springfield pets, fact check, and that might surface some reliable coverage for you. And then, you know, that claim trace is not particularly useful, but one thing that you would find if you were to be sort of immersed in media at that time…

Eric (09:10.252)
Mm-hmm.

Mike (09:28.189)
…is other people would have surfaced, for example, a picture of a person that is supposedly, is supposedly carrying, I think there was like a picture of a person like carrying a goose or something like that. And it’s a picture and it’s claimed to be about this issue of these problems that Springfield is having. Well, did that photo come from Springfield? Was that person?

Eric (09:44.355)
Mm-hmm.

Mike (09:57.501)
…was a person of the demographics that described in this room or tracing things like that. So a lot of times you’ll find sort of a big story like this, and then people will marshal all these smaller pieces of both evidence and sort of quote evidence. And you got to figure out where that evidence that’s kind of flooding your stream came from. And that’s where you might do something like a reverse image search or something like that.

Eric (10:23.148)
Hmm. Describe it in reverse image search.

Mike (10:27.835)
Yeah, so on most browsers, if you right click or command click an image, you’ll get a little menu option that’ll pop up and it will say search this image or something similar. And if you do that, your search engine, whatever it is, will go and it will try to find places where that image has been used before. And that’s particularly helpful if you want to figure out…

…where an image came from. So for example, if someone’s sharing an image that is supposedly of something that happened, I don’t know, last week in France, and you reverse search that image and you find out it was first used in 2016 in a German newspaper, then that’s probably been, someone’s probably trying to dupe you there. And so reverse searching images can be really useful.

Eric (11:17.858)
Mm-hmm.

Mike (11:23.727)
On your phone you can do it too. It’s usually on your phone, it’s usually like a long press, not, you know, some slightly different sort of thing, but you can do it on your phone as well.

Eric (11:36.142)
Increasingly, a lot of the claims that might strike you as dubious are about technical issues in science or science in health. things are not necessarily kind of as questionable on the face of them as the rumor about…

…immigrants in Springfield, Ohio. So the anti vaccine movement, for example, often involves questioning scientific testing protocols, which are beyond the ken of most lay people. How do you evaluate something like that?

Mike (12:15.997)
So this may be hard for academics to hear, but the thing that you don’t want to do in a domain that you’re completely unfamiliar with is you don’t want to dive immediately into sort of primary scholarly sources. And this is just one thing that we see people, you know, sort of repeatedly overestimating their ability to sort of jump in the deep end of the pool…

…and kind of make sense of this. What you actually want to do is you want to some start with some reliable sources that sort of summarize the issue for you, lay out really what we call reading the room, lay out the map of the field. Like, you know, so for example, for an issue like fluoride in the water, like what is the current thinking on fluoride in the water?

Are there reputable, are there there sort of reputable experts that have concerns about it? What do those concerns involve? You know, try to get that sort of lay of the land first. And then and then if you you so desire, then you might want to go the next step. You might want to start looking at these at these sources. But the thing that we really advise starting with is figuring out who’s going to provide me a decent…

…summary of this issue. And you could get summaries from different perspectives too. It doesn’t mean that you just go one place and get a summary, but start with sort of reliable summaries, get your sense of the lay of the land of that issue, and then start to go in deeper once you kind of have oriented yourself to the space. If you dive in and you start reading a bunch of documents where you don’t know the terms, you don’t know the current debates in the field, you don’t know the history of the field.

You don’t know the main characters in it. You don’t know which journals are more respected and which journals are sort of predatory journals. If you dive into this and try to make sense of it at that level, that’s where we see a lot of people go wrong. And sometimes people get a little upset about this and they say, you’re telling people to rely on other people for these summaries and so forth. And I’m sorry, but…

Eric (14:26.946)
Mm-hmm.

Mike (14:38.779)
…you’re always relying on somebody. You are always relying on somebody. The question is, like, who are you going to give your trust initially? And what are you, what, know, one of the things that we talk about in the book is the different contexts, right? So we have like the context of the source, we have the context of the claim. But we also have something we call the context of you.

And the context of you is like, what is your knowledge? What are your abilities? What are you sort of currently capable of understanding? And you’re gonna build that, right? You’re gonna get smarter, but understanding where the current limitations of your knowledge are and using that to think smartly about where to engage, how to improve your understanding.

That’s just crucial. And so I do think with some of these more complex issues, start with a broader summary, understand your limitations, develop the skills to get deeper into that, to study things at a higher level of complexity, but really start with the basics and then dive into the deeper stuff.

Eric (15:55.86)
One of the problems with finding a reputable source for this kind of summary overview, sort of layman’s basics for complicated topics is that trust is eroding for all kinds of levels of expertise and fields of expertise. How would you sort of evaluate this theoretically unbiased source that kind of lays the groundwork for your greater understanding of a controversial issue.

Mike (16:29.467)
Well, I mean, I think you would start by trying to find a bunch of people that normally don’t agree on anything and see what they agree on. I mean, that’s generally what I advise doing. You know, so I think people really underestimate, for example, if you take academics, I think that when people look at academics, they really underestimate how

prone academics are to disagreeing on things. And so by the time that the American Physical Society, the physicist professional organization, the APS says in the 1990s, look, climate change is real and humankind is playing a role in that…

…when you actually have that organization, as an organization deciding to put out a statement on that, like that’s not the beginning of the debate, right? That’s pretty close to the end of it. And so I think that that’s what you’ve got to look for. And it’s important to get sort of a broad view. mean, so for example, I think it’s useful to look at what the American Medical Association…

…says, because again, this is a broad professional organization and it has to sort of negotiate debates between like anesthesiologists, medical researchers and surgeons, a whole group of people that are very prone to disagreeing. And so when they come out with something, I think it’s worthwhile to look at. And then I think the other thing is that you can go and look at the American Medical Association…

…you can go and you can look at medical associations in other countries. And if you find, look, the AMA says this, but the corollary to the AMA over in Japan or…

Mike (18:37.533)
…in Japan or in Italy or wherever says something different, then that might give you a pause. But if you find, look, when large organizations of medical professionals get together in just about any country with a wide variety of medical systems and a wide variety of sort of cultural backgrounds, and they all say roughly the same thing, like that’s where I would feel…

…more comfortable. One of the things you’ll find though is when you look at that, a lot of times they just don’t, right? you know, so I think we’ve got to, if we’re thinking about trust, we’ve got to realize that the amount of things that people agree on are actually a lot smaller than we think. And so when we find things that a lot of people agree on, should take that, I mean, it’s not unassailable, but we should take that pretty seriously.

Eric (19:37.324)
Hmm. I think that there’s a certain romantic legend around the Galileo type person, the scientist who discovers something that is different from the consensus and leads the way to a brighter, more realistic understanding of the universe. And that is actually extremely rare. And Galileo, to use that example, was starting from…

Mike (19:46.012)
Mm.

Eric (20:06.254)
…a consensus that was not built on science, but rather on faith or legends or incomplete understanding. And I think it’s a very useful tool, as you just described, to find people who ordinarily disagree about things and find out what they agree on and let that consensus carry a lot of weight in your opinion.

Mike (20:28.253)
Yeah, and I think too, one of the things that, so there’s this idea that there’s sort of a lone genius in the wilderness that is sort of saying these things over and over and nobody will listen. I think actually, if you look at the history of science, generally anybody that has an idea that maybe possibly has legs under it, if they do the work…

…you know, within 10 years or something, I mean, within a relatively short amount of time in sort of research years, they’ll find at least a small, they’ll develop a small community of like-minded scholars that will investigate this. So one of the things we talk about in the book is how to think about…

…disagreement and one of them is one of the things is consensus, right? But there’s another very common formation of opinion among scholars or plumbers or, you know, any group of professionals, which is a majority, what we call majority minority, right? Where the vast majority of people in a profession or in scholarship or somewhere else embrace…

…one idea, but there’s a small minority of people that embrace another idea and those people are in the field, right? And this is a kind of important, right? Those people are in the field, they have a different opinion, but they go to the same conferences, they get the same, you know, if they’re in a profession, they get the same certifications. They are talking to other people in the field, they are making their case to other people in the field, in the way that you make the case to other people in the field. And they’re doing this over time.

And so most things in science that are disagreements where eventually someone comes along and says, you know, you know, I don’t know, says, you hey, you know, it’s not really, it’s not really the direct consumption of cholesterol that’s that’s causing people’s cholesterol levels to go up. That, yes, that might have been a opinion that was that was not the mainstream opinion in a field.

Mike (22:47.869)
40 years ago, but if you go back 40 years, you’ll find a small group of people who are in the field writing those papers out at those conferences making the case. Now that’s very different from what a lot of people are talking about when they have this sort of romantic notion, which is a fringe theory. And we actually have, I think, a useful definition in the book of what a fringe theory is.

And it’s a theory that if you think about what fringe means, it’s literally on the fringe, meaning that it does not engage with the discipline. That is, it is not a bunch of sort of rogue physicists with a rogue idea about physics, right? Going to the conferences and a lot of people rolling their eyes, but being like, well, you know, that’s Jurgen, that’s Jurgen and his theory. No, it’s not that. It’s someone that does not engage with that field at all, right? Does not…

…learn the basics of that field in any way that allows them to communicate what they are doing. They’re not trying to publish in the journals of the field. Fringe is very different. It doesn’t engage with the profession. And this is not just for scholarship. Like I said, this could be a plumber. You have a plumber who wants to go about things. mean, code prevents you from doing a lot of those stupid things in your house.

Eric (23:57.931)
Mm-hmm.

Mike (24:09.361)
But you have a plumber or you have an architect that wants to go about things a certain different way. You know, that architect, you know, they still have to have the same certifications. They still have to know code. They might have a different way of approaching things that is gonna, they think is gonna build that house more cheaply, more economically and make it more efficient. They may be a rebel in that way, but they’re still in the field. They’re still engaging with the field.

And that’s, I think it’s very important when we look at these things to distinguish between situations of majority minority opinion and fringe.

Eric (24:47.18)
Mm-hmm. That’s very useful. One of the mistakes I think a lot of people make when they think about, you know, a literate response to claims in the media is that the process should be highly analytical and that you should address questions the way a fact checker would or a lawyer would. In the book, you make a point that your emotions can actually be a very powerful clue as to where to focus your attention. Could you elaborate on that?

Mike (25:20.701)
I’m so glad you brought that up. That’s actually one of the parts of the book that I’m most proud of because I think it gets us something that very few people are talking about. But something I think if you teach this stuff in a class, you start to realize this over time. And that’s where I realized this is one of the things that we were finding was when we taught students this stuff.

If we taught students something where they did not have an initial reaction to the thing, right? So, you know, we show some, show the students something in it, you know, let’s say we show the students something and it says, I don’t know, you know, KGB agent reveals that the sex pistols were a Soviet op or something like that.

They don’t know who the sex pistols are and they probably don’t know who the KGB is or anything like that. The process of fact checking that is just very hard because they don’t have an emotional reaction to, well, that’s ludicrous because blah, blah, or that makes sense because of this. They don’t really have a way to do this. And this is particularly true if you think about statistics. If you think about a statistic…

Imagine you show a statistic that someone has no sort of emotional reaction to, like for example, well, let’s start with one that you have a reaction to. If I show you that since 2018, know, fentanyl deaths have, and I don’t, I’m just sort of making this up off the cuff, but if I show you that since 2018, fentanyl deaths have tripled and are now the leading cause of overdose death, which I think might be roughly true, but don’t…

…coping on it, so you something on that scale. And you look at that and your reaction is, this is horrible, right? You know, and your reaction is, you know, maybe your reaction is like, why aren’t we doing more about this, right? Then when you go and you look at that and you find, you find, oh, well, maybe it didn’t triple, maybe it’s only up by two and a half times, right?

Mike (27:30.449)
You’re still gonna look at that and you’re gonna say, that’s still horrible. It doesn’t change it, right? Like I’ve gone, I found the facts. And yes, it’s two and a half times what it was in 2018. It’s not three, but it doesn’t really change the facts because you know where your emotional connection to that is. Whereas if I show you something, that a bunch of widgets, the…

There’s a certain widget in the US that we used to produce at a level of 1200, but now we produce it at a level of 800 a year. And you go and you find, actually we produce it at a level of 900 a year. So like, you don’t know, like, does that matter? Does that not matter? Or you find out, it’s not really that.

We have that widget, but there’s actually another widget that’s sort of like that widget that we produce. And if you add those together, like if you don’t know what the meaning is to you, you can’t assess whether the additional context matters because what you’re actually checking, and this is the most important thing for people to realize, when you do a personal fact check, what you’re actually checking is your reaction to the thing you looked at. And what you’re really asking is,

Was my reaction to this thing I looked at appropriate when I know the bigger story? Or now that I know the bigger story, would I have reacted a different way? And that’s kind of how we end up, that’s the psychology of how we make, I think, very complex judgments very quickly. We’re able, when we look at additional context, we’re able to kind of get in our head and say, let’s imagine an alternative me, let’s imagine I had known this before I looked at this. Would I have felt the same way? And if the answer is yes, then it checks out. And if the answer is no, you know what, if I had known that to start, I think I would have reacted very differently, then it’s like, no, this does not check out.

Eric (29:17.56)
Mm-hmm.

Eric (29:29.408)
It struck me that the SIFT methodology is targeted at web pages. So information that comes through, you know, digital publications or that kind of coverage or, you know, spoken word speeches and things like that. Most people, especially younger people, get their information from social media, TikTok, Instagram. Is there a different process involved with that?

Mike (29:57.981)
I mean, a little bit. It’s not a different process. There’s a small amount of additional skills. And we do have a chapter in the book called Video Games, video games, geez, I can’t remember the subtitle of it. Video Games, colon something. The deceptive tricks of video online. I should remember the titles of my own chapters.

We have a chapter on that and talk about video. the one thing that you have to develop with video is you have to develop a talent of figuring out what your keywords for a video is gonna be. Generally you are…

…once you kind of get that down, you’re following a very similar process. You’re searching for additional context on the video. You’re searching for the bigger story. You’re searching for what other people have said. But it could be a little more challenging with a video. Mean, so for example, if I’m looking at a text story, I mean, I have a headline to that story. It’s much easier to see what the main point of the story is. If I see a picture, I can kind of just right click it and do a search.

If I’m watching a video, particularly a longer video, it’s actually, it’s weirdly easier on TikTok in some ways than it is on YouTube. On TikTok, a video is normally about one thing, and I can generally summarize what that video is. You know, a YouTube podcast, you know, has like dozens of claims in it, you know, running through these like little…

…know, interwoven through like, you know, 45 minutes, an hour, sometimes much more, you know? And so it can be more difficult to check that. And part of what you have to do with a longer form video is you have to figure out what, again, it kind of comes down to that emotional piece. Like, what is the piece of this? I’m listening to this podcast. What is the piece of this that made me think, that surprises me? Or, that enrages me, right? Or,…

…that, you know, this proves that I was right all along. My family was wrong. I’m going to email them this podcast and it’s going to prove to everybody that I’m the smartest kid. You know, so you’ve got to think about what it is in that video and not just say, you know, and how to kind of summarize that. And that’s where I think the skills that we teach students

about how to concisely summarize a person’s claim, right? What does it mean to make a claim? And that’s where a lot of my work right now is going, is in the critical thinking space, really thinking about how to get students to think about what is the claim here? What is the claim being made? So that, because if you can get to that, if you look at the video and you can figure out, what is the claim that I’m interested in? And you can succinctly type that into a browser.

into a search tab, then it doesn’t matter if it’s a video or a TikTok or an image or a blog post or a tweet, you’re gonna be able to get to better context. But if you can’t do that, if you can’t listen to that conversation and figure out, okay, well, how would I succinctly describe this claim? It’s gonna be difficult.

Eric (33:36.14)
That’s good. All right. Yeah, I think that the feeling, the emotion is that I always knew that was true is probably the most dangerous. It’s the one that makes you the most vulnerable. Yes.

Mike (33:44.433)
Yes, righteous indignation is perhaps where we’re at our most vulnerable.

Eric (33:53.486)
How does artificial intelligence change a game?

Mike (33:57.693)
Well, it changes it in a number of ways. One of the things it does is it makes it. So we talk in our book, there’s a chapter on something we call cheap signals, right? And cheap signals are anything that you can kind of right? So, you know, if you think about think about a job interview or something like that, right? You know, you go into a job interview and you want to be a

executive at a bank or something like that. Anybody can buy a suit. Anybody can buy a suit and a tie and look like they look like a bank executive, right? Anybody can, anybody can, you know, probably find a friend or two, you know, to provide a reference is much harder to develop a history of having worked in the banking industry for

20 years, right? Like that’s actually something that takes a while to construct. That is harder to fake, right? So the cheap signal is like the suit, right? And the expensive signal is, have a verifiable history of working in the banking industry for 20 years, right? And what we want on the web is we want to look for the expensive signals, right? And we teach students not to look at the cheap signals. Don’t look at

you know, whether the page has spelling errors. Don’t look at whether it’s laid out well, whether it, you know, whether it, you know, has, you know, people used to tell people, oh, see if it has an email, if it has an email, it’s reputable, or look if it has a .org after it. You know, those are all cheap signals. Anybody can get those. It used to be at least a little bit hard.

to write in a scholarly style. And we know this because we tried to teach students to write in a scholarly style. And it’s very difficult. It’s not easy, right? So there used to be something there that showed at least you had gone through some type of schooling or something that you could write in this way that was convincingly scholarly. AI basically makes

Mike (36:13.693)
tone and style of that sort useless. You can plug what you wanted to AI and say, write this as a paper that would be in the journal of the American Medical Association, and it will do a fine job writing in that style. It may tell you a whole bunch of things that are wrong, but it will sound convincingly, it will sound convincingly a journal of the American Medical Association. So I think that’s huge. think that’s the last gasp of what we call surface features, right?

that you’re not gonna be able to look at the sort of surface and style of anything anymore and be able to say, well, I can tell because of the precision of this style that this is from this sort of person. AI in particular is great at style and so that’s off the table. Then comes the question of can you use AI for fact checking? And in the book, we say,

Eric (36:42.222)
Hmm.

Mike (37:09.821)
Um, we say, we really don’t want you to do that for a number of reasons. One, it can make things up and two, it’s very hard to tell, um, if what AI is telling you is true. And over the past, um, three quarters of a year, I’ve been doing, I think some really interesting work looking at AI. And I think I found some actual useful ways to use AI. but they’re very different than how people use AI.

Don’t use AI as a search engine. Like for the love of God, please stop using AI as a search engine. It is really, really inappropriate as a search engine because it’s not searching for anything. There’s nothing on the other end. There’s no set of things on the other end it’s trying to find. And so people keep on using it as a search engine and keep on getting ridiculous answers back. And I don’t think that that’s ever going to be resolved because a search engine presumes

that there’s something on the other end you’re searching for. And so we’ve developed this habit of using input boxes on the web, the search engines, AI does not do very well with that. What we have found AI very useful for is there are ways to set up, there are ways to put in, there are ways to prompt, I was trying to avoid this term prompt, but this what we talk about with AI. There are ways to prompt AI.

to have AI give you a larger context to an issue that helps you better understand what you’re looking at. And as long as it’s sort of a general context, AI turns out to be super useful for that. But once it gets down to the specific facts, right, then it turns out to be a lot less useful. And that’s something I’m hoping, I don’t know if any of your listeners,

Eric (38:44.374)
Hmm

Mike (39:04.829)
are looking for a speaker or a workshop in the future. That’s something I’m hoping to be my work over the next five years is to look at the ways that AI can be helpful for sense making on the web. As long as we use it in the appropriate way, the way appropriate to AI, which is, I believe, not a thing that most people are doing right now.

Eric (39:31.694)
That’s interesting. Well, that is a good segue to a question about media literacy training in general, using media literacy as a sort of umbrella term for any kind of tools and techniques for finding a way to the truth. There are programs, there are organizations like the News Literacy Project, for example, that promote

this kind of critical thinking in K through 12, and you taught it to undergraduates, it seems that adults are left out of this equation. Although, arguably adults are no better able, you don’t mature into media literacy, and you may be less digitally literate than people who grew up as digital natives. And of course, unlike K through 12 students, you vote. Are there

successful programs for adults and for someone who’s you know out of school and looking to sharpen their skills in this area what’s your advice?

Mike (40:37.115)
Well, mean, yeah, I hate to be an author, but you can buy the book. You know, that’s one way to do it. know, more to your point. Yeah, it’s harder to reach adults just generally. mean, a piece of that is really just, you know,

Eric (40:41.474)
Yeah.

Mike (40:59.161)
Any sort of education that reaches adults is just more difficult. With students, you have butts in seats and this stuff is can initially, it initially takes a little focus. You gotta have some, some attention to it. You gotta actually do it, right? Which is always the place where people go wrong. You’ve gotta practice the skills a little bit. And that’s, it’s harder for adults to get into a situation that really kind of walks them through that.

That said, there’s a program that Google sponsors that is based on a SIFT called Super Searchers that is, I think, rolled out at thousands of libraries around the world at this point in multiple languages. And it’s a 40 minute, it’s like a 40 minute session that librarians can give to patrons and that they are giving over the world.

to different patrons in these workshops. More importantly, one of the things that we realized when we were looking at these library programs was one of the ways we’re reaching adults is not necessarily that they come into that 40 minute workshop, you know, the super searchers workshop, but that they might come in to use the computer at the library, right?

or do research, and then they have a question to the librarian. And this is just sort of a core librarian job. They’re looking for a piece of information, right? And they’re asking for help from the librarian. The librarian doesn’t just have to go and sort of do everything for them. They can say, here, this is what I’m finding. Let me show you a couple neat tricks to figure out how to get to the information you want, how to get the context you want.

So what we are finding is it’s useful for public librarians to have these skills so that when they have these sort of side-by-sides with patrons, they can show them. Let me show you how to use Wikipedia to look up the organization that you got this from and figure out, you know, is this an advocacy organization? Is this research? What is this organization about? That sort of thing.

Eric (43:16.856)
Mm-hmm.

Mike (43:18.365)
And so I think that’s a really useful way of doing it. And then the other thing that has happened, I think, with SIFT that’s made me happy is a lot of times it’s been woven into other things. So there’s a number of hospitals, for example, that will give on their patient website, patients not only information about their health, but how to better think about researching their health.

And some of those hospitals have given patients SIFT as a method and shown them, okay, here are the steps to SIFT, which I think is, to me, is really heartening because I find SIFT to be empowering, not disempowering. I know that there are some people that feel that any sort of imposition of saying something is reputable and something is not is…

That’s some sort of Orwellian Buddha censorship to say, I don’t think that this thing is as solid as this thing. I just profoundly disagree with that, obviously. But I do believe that things like SIFT are empowerment and they’re empowerment for patients because you want… One of the things we’ve realized, if we think about citizens and we think about experts and we think about professionals, is that…

Yes, citizens are not necessarily experts in the science of medicine, right? But they’re experts in their affliction, right? They’re experts in what they need. They’re experts in dealing with whatever they’re dealing with. They bring an expertise to that table that the physician does not have, right? And so you want to figure out ways to empower people who have that expertise and need to have a voice.

in their affairs. Democracy is the same way. Yeah, of course we’re not like climatologists. We’re not a nation of climatologists. But we live in a democracy and climatologists don’t get extra votes. That’s not the way it’s supposed to work. We get the vote. So we have to understand it. We have to understand it. We have to challenge ourselves to understand it. So.

Mike (45:38.813)
Part of what we’re trying to do here is to make sure that when people go out and do this stuff, that they don’t feel disempowered. That we say, look, we want you to get more educated. We want you on the web. We want you researching these things. But we also want you to practice some basic digital hygiene here so that as you progress in this, you’re educating yourself and not necessarily spiraling down a rabbit hole.

And so I do think plugging it into a bunch of these smaller places where we really empower citizens and patients, but say, hey, please go out on the web. like, here’s a couple of tips. think that that’s a good way to approach it. And I think it speaks to the original empowerment vision I had of CIF.

Eric (46:27.246)
I love that Mike and the idea that these kinds of critical thinking skills are empowering is a wonderful thing to keep in focus. You talk in the book about, I forget the exact adjective you use, but it’s intelligence ignoring. basically you, critical ignoring, which is a wonderful thought that you can shed like the water of a duck’s back.

Mike (46:47.642)
critical ignoring. Critical ignoring. Yeah, yeah.

Eric (46:56.298)
a lot of fringe theories on very quick examination and then you kind of are less buffeted by information from all sources if you can distinguish what really should capture your imagination. I have a question about media. This is where I spent my career and media is taking a beating over the past eight years perhaps in its

Mike (47:11.096)
Yeah.

Eric (47:26.232)
perceived authority probably won’t get better over the next four years. Are there things that professional media can do to make it easier for people to identify trustworthy sources and use institutional professional journalism as that kind of reliable overview or a source that provides better coverage?

Mike (47:53.105)
I mean, the first thing they could do is make it more available and make it put it in the format that people are consuming it. I mean, I think that’s probably your biggest problem right now in media. You know, this is an oversimplification, but there are some people that have adopted the phrase, the truth is expensive and the lies are free. And I think that oversimplifies things. But

I mean, there is a real problem here. I will say when I first started looking at this in the 2010s, I first started looking at this at the 2010s, all news was basically free on the web, right? There weren’t pay walls, the New York Times hadn’t rolled out their strategy. Reporters have to get paid. There has to be a business model. I’m not in any way going against that, but…

One of the things that I’ve struggled with a little bit in SIFT is that even as I’ve done it, it’s gotten a little bit harder because, know, it used to be, it used to be for a while, everything was free. And then for a while, you know, you would get like five free articles a month before the paywall kicked in. And then it got to be one free article a month. And then there are a lot of places they’re saying it’s like one free a month, but I’m pretty sure it’s not because the first time I’m going there, it’s bouncing me back.

And so it’s really hard for a person that, you know, we say go out and get reputable stuff, but, you you know, a good, a good journal article for someone that’s not working at a university is like, you know, 30 bucks, you know, the best coverage on something is going to be a subscription, you know, to something. And so, so there are still papers out there, you know, there’s the, the guardian, there’s ProPublica, there’s, there’s some other

things out there. as that stuff goes more and more behind paywalls, as we hit this age of what others have called digital retrenchment, where there’s less and less stuff on the web, at least of value, I really worry about that. So I think that’s the first thing that media could do is think about the mission and think about profit and sustainability, but also think about

Mike (50:19.303)
I also think about what this is doing. What is it going to be in a world where all the reputable journalism is behind paywalls and everything that is out there trying to spin you is not? And we’re heading into that. The other piece is that there’s a sort of a massive appetite for short form video and for podcasts and things like that.

And, you know, a lot of people do that are great. They have talent at that. And I’m not in any way denigrating what’s currently out there. you know, if you want to go out and you want to educate the public right now, you got to be educating it. Or I shouldn’t say educate. Part of what journalists do is educate. you know, inform, just inform the public. You got to be.

You got to be in the formats that they’re consuming and you got to be on the platforms that they’re on. I’ve seen little experiments to do that. But very often, you know, if you look at like the TikTok team at a major publication, I guess a team, you know, it’s like four or five people, maybe, you know, and then you have paper of thousands, you know, and yet

Eric (51:17.704)
Mm-hmm.

Mike (51:43.909)
If you look at where the audience is, you know, it’s over here. And I understand, I am a print person to the core. I want, you give me something, please give it to me in print. It’s more, I go to my grave believing that consuming print is far more efficient than consuming video. That it gives us far more of a grasp on things. it makes us much more analytical in ways that are great. But, but if it’s not where people are…

…you gotta be where people are. And if we don’t get there, I think that’s the biggest gap I see. And I really worry that if we don’t close that soon, we’re in for a hell of a rough ride.

Eric (52:29.256)
That is a great point. Tell me, where do you go for your news and information?

Mike (52:35.419)
I go to the Google search box. I guess that’s not surprising. I go to the Google search box. so I actually have, I’m on a platform right now called Blue Sky and I have, I subscribe to a lot of journalists on Blue Sky who have gone over on that platform. And I try to, that’s the first thing is make sure that you’re subscribing to sort of a wide array of people that are feeding you the stuff that you want. Be really intentional about who you follow.

Eric (52:37.23)
Thank

Eric (52:53.45)
Mm-hmm.

Mike (53:03.055)
And if you’re not getting the stuff you want, be active and go out and get the sort of sources that you want. And then when stuff comes in through that, very often it just comes from the journalists that are publishing the stories, which is helpful because then I know where it’s coming from. But if something comes through it that I don’t recognize, then I probably unsurprisingly, I sift it and usually find a better source. So I am absolutely of the generation that

loves the personalization of this social media approach to the internet where the information that is important to you finds you, right? And that’s a lovely, lovely thing about social media is we, you don’t start from a stop and have to go out and find the things that interest you, the things that are interest you find you on the web. It’s amazing. But then, you know, as we talk about the story of the sort of bottle with the note in it that washes up in the shore and the book,

You know, a bottle with a note in it washes up on the shore, you pull it out, it says, know, some headline or something like that. You got to say, well, where did this come from? You know, who wrote it? Why? What do other people say about it? And so there’s no shame in engaging in the sort of the social media environment that we are. think it has wonderful benefit. could, I mean, we are in a technical environment that could make us some of the most informed people in history.

But we’ve got to approach it intentionally. And we’ve got to apply our critical reasoning skills to this, including getting the critical context of the things that we encounter on

Eric (54:44.876)
Would you say that we are winning or losing the battle against disinformation?

Mike (54:52.814)
that’s a hard one. I would say that…

I would say that we’re losing. I would say that we’re losing. You know, I think…

I we’re losing in an interesting way that you probably can’t summarize in the remainder of this podcast, but I think that we’re getting to a place where, I think we’re getting to a place where persuasion is actually almost impossible. And the reason why is that the internet,

as it’s developed and as we’ve learned to use it has become a tool that we use to justify our pre-existing beliefs. And it makes us immune to new information. I mean, it puts us in a position where we can watch, you know, collapsing ice shelves in Antarctica and then, you know, go and read something on the internet that tells us, well, you know, actually there’s always some ice loss and

And really, if you look at the North Pole versus South Pole, there’s a slight difference here. And there was a point in 1998 where it just provides this sort of endless stream of self-justification for us, where we just become immune to these things that should be challenging our worldviews. And that’s not just a liberal and conservative issue. mean, I’ll say,

Mike (56:33.845)
I’m a scholar, so I have to be honest. have to say that the right-wing media environment is a little more screwed up than the left-wing media environment. That’s for certain structural reasons. But it’s not about the people involved, right? It’s not like we’re all equally susceptible to this. And the media environment as a whole is, I think, very, very prone to feeding us

very personalized in this sense that it does it feeds us it feeds us information that tells us hey we don’t have to reconsider our beliefs we were we we were right yesterday we’re right today and we can watch we start to watch like these shocking things that should change our beliefs and somehow it doesn’t matter.

Eric (57:25.858)
That’s a great place to end it. And thank, and thanks, thanks.

Mike (58:02.392)
My pleasure. This has been a great conversation and yeah, and I love the questions and yeah, I hope people run out, get the book. Or even if you don’t, just think a little harder about what you see on the web.

Eric (58:23.66)
The book is verified and I highly recommend it. By Mike Caulfield and Sam Weinberg.

Mike (58:29.895)
Thanks.

---

Created & produced by: Podcast Partners / Published: Jan 2 2025

---

Share this episode: