I am getting asked the same question far too much lately: “What is your take on [x]; it seems pretty convincing; how do we know it’s not reliable?” Where [x] will be some crank on the internet, some ridiculous news headline, some random article somewhere, or something they found asserted repeatedly on social media. Now, I do not understand why anyone, anyone, has to ever ask me this question. How do you not already know how to answer that question? We are fucked as a society if this many smart, motivated people don’t know this already. We cannot afford to be Bad Thinkers anymore. We need to know this already. And anyone you know who doesn’t, they need to know this already. Hopefully this article will help you help them; or help yourself, if you need. We all must learn how to vet things on our own. This is an absolutely crucial skill in the 21st century, more important even than literacy itself.

Faux Critical Thinking: Conspiracy Style

Because I am noticing a trend. Nearly half my country (and that means, there’s almost a 50% chance I’m talking about you) has become completely divorced from reality, and is operating within a wholly delusional worldview, and defending it exactly the same way Christians defend their equally-bonkers “faith.” Principal among the fatal defects of their adopted methodology is a complete absence of critical thinking, even of the most rudimentary kind—even from people who otherwise think critically. And it is in fact this defect that traps them in their delusion, with no way to escape. So you need to make sure you’ve installed a basic firmware update in your mind, and help others to, in order to inoculate us all from falling into that insanity trap, to keep ourselves tethered to reality.

It begins with media literacy. No individual can be an expert at everything, much less “be everywhere and see everything.” We depend absolutely on trusting sources of information, for almost literally everything we believe, and need to believe—just to function, much less vote capably. This has always been true, but it has become more and more true owing to the very defining feature of civilization that makes it so useful and powerful: the division of labor. Which in turn leads to increasing complexification of information, knowledge, and process. A digitally advanced civilization like ours is way off the scale in this respect. People feel powerless not merely because they don’t understand nearly anything that happens in the world or why; but because they can’t. It’s too complex. So at some point in any chain of complexity we have to trust sources and experts and authorities somehow. This is normal. It’s also a dangerous pivot that can be exploited to control you by people who don’t care if you are tethered to reality—or even benefit when you aren’t. You should not want to be their puppet.

Humans evolved to quickly trust cues to reliable information based on old, no-longer-reliable heuristics based on prehistoric tribalism (“it comes from my tribe; it comes from people my tribe declares an authority; it fits my preconceptions…which I learned from my tribe”). Which is why delusional and conspiracy thinking today is also tribalistic, based on in-group identification (“identity”) rather than evidence and reason (“empiricism”). We thus tend to look for ways to verify and reinforce our prior beliefs, especially beliefs we have an emotional attachment to, and most of all beliefs we define ourselves by. These weak principles evolved as a stop-gap because we don’t have time to “independently vet” literally every piece of information we ever acquire all through childhood and adulthood much less in any and every endeavor or decision. But our evolved means of solving that problem is shit. What we need to do instead, to economize this and “tune” our information acquisition toward reliability, is vet sources. And then selectively vet specific examples of information, choosing wisely which ones to bother vetting and which ones to let go until it matters.

All this means that the problem with “conspiracy thinking” (which is today all the rage) is not really that it entails bizarrely false beliefs about what people are supposedly “doing” (drinking the blood of virgin children, ruling the universe in cohort with galactic lizards, rigging elections). The problem is, rather, epistemological: “conspiracy thinking” as a style of vetting beliefs always correlates with some belief-matrix that convinces you to distrust everything said by one set of sources and naively trust everything said by another set of sources. In other words, it fucks up your ability to critically evaluate sources of information, cutting you off from reliable ones, and enslaving you to unreliable ones. And you are vulnerable to falling into this trap the moment you stop being able to tell on your own which stories and sources are actually bogus. Emotional biases and in-group allegiances will then begin to dictate your beliefs (because they correlate with your “gut feelings”), rather than reason and evidence (which are supposed to function as an error check on your “gut feelings”).

As is said in The Enduring Allure of Conspiracies, “Once someone has fully bought into a conspiracy theory,” or even just “conspiracy theory” thinking as a model of interpreting evidence generally, “there’s very little research that actually shows you can come back from that.” Once trapped, you are doomed. You are now a Flat Earther. Or whatever “the bullshit” is you’ve tricked yourself into believing. And you will be forever. No evidence, no logic, will ever save you. Because “How do I know if that’s happened to me?” is a question you’ve been convinced never to ask yourself, or never to answer in any effectual way. And that’s how it begins. You’ve now become a Bad Thinker. So task one is to always ask that question about yourself and about what you believe, and make sure you have an effectual answer—as in, you know what methods can actually tell if that’s happened to you.

As that same article says, “People are sharing headlines that they could identify as being false if they bothered to think about it.” And that is the principal lesson you need to learn: because it means you could be doing this too. So you need a reliable way to check and make sure you aren’t. Because that is the only kind of person who will have a belief system that largely corresponds to reality. Media literacy and critical thinking converge on a simple toolkit for running a “mind virus” check on yourself, and purging false beliefs before they take root in your base code. Each tool in this kit requires some unpacking to know how to use it competently. So I’ll go into that. But let’s start by laying the tools out and seeing what they are; we’ll study how to use them (and how not to use them) in a moment.

This is the basic media literacy kit:

• Learn how to tell when sources are unreliable or even dishonest and covertly attempting to manipulate you.
• Stop trusting sources that have proven more than once that they are unreliable or even dishonest and covertly attempting to manipulate you.
• Stop trusting sources (or any information) based on their ideological or prior-belief bias; learn to identify trustworthiness instead by actual epistemic standards that would hold for any ideology or belief.
• Learn how to identify trustworthy sources without expecting or requiring them to be flawless or immune to error.
• Learn how to vet the assertions even of those trustworthy sources.
• For every assertion from a “trustworthy” source, always devote as much time to vetting it as the danger of being wrong about it warrants.

For every one of these tools, “delusional thinking” (or what I will call “incompetent thinking”) has developed a “defense,” a way to reject the tool and thus avoid using it. If you suffer from this cognitive illness, the delusional technique you will resort to is rewriting how the tool is used so that it won’t work, and then declaring the tool useless (or even a threat to the truth, rather than its savior; to be avoided rather than wielded). I’ll show you some examples shortly. But every time the general process is the same: deliberately getting wrong how to use the tool correctly, then showing that this incorrect use leads to intolerable defects, and then using that as an excuse to abandon the tool altogether. This defines all delusional epistemologies, religious and otherwise: they all have this inescapable trap-loop within them, whereby every possible way you could escape your delusion is blocked by the delusion itself convincing you not to use it. It is precisely at the moment you can no longer tell this is happening that you are trapped forever in a delusional mind. Fucked, for want of a better term.

So try not to be fucked.

Here’s how…

Why You Should Dismiss FOX News

I’ve given many examples before of how to approach sources critically, including professional media, social media, even scientific journals and expert peer reviewed literature (see my whole blog’s category series on Critical Thinking; I also teach a one-month Online Course in the subject—where you can even bring in any claims you’ve encountered or struggled with that you want to test these skills on, and we can work on them as study cases). But here I’ll use some simpler examples to illustrate how we need to employ each tool in the box correctly to avoid delusion.

The first one comes from something FOX News did recently; and this matters, because, as with any other case like this we might cite, once they did it, it was duplicated and spread like a virus by countless other “conservative” media sources, professional and amateur, often without even telling anyone this. It is a well-known cognitive bias that your brain unreliably assumes anything repeated many times, especially by “many sources,” must be true. And this is why all manner of conservative media will duplicate a FOX News story, even down to the exact headline, without telling you that (or making it as inobvious as they can).

They do this because they want to trick you into thinking that story comes from multiple “independent” sources; because they know how to use brain science to manipulate you. These stories thus metastasize like a cancer; and like a cancer, you need to always trace it to its source: Where did a story first appear? Where is this actually coming from? Don’t be a naive dupe. Contact-trace the story to its origin. Only then can you evaluate its quality. You will then know to dismiss all the times it was just “repeated,” as now you know those add nothing to the probability that the thing being repeated over and over is true. Repetition is not evidence. And you do have to keep telling yourself that, because your brain was not built to understand it.

• This lesson does not apply only to FOX News. It applies to literally everything. Always trace a story to its original source, and evaluate that source. And always dismiss mere duplications of the same story in other media as useless data.

The following is therefore just an example of a general pattern not particular to FOX News or even to conservative media. FOX just makes it easy to illustrate.

Hours after Biden signed the all-federal-properties mask mandate he removed his own mask to give a speech at the Lincoln memorial. And immediately FOX News (and then a litany of equally dishonest conservative toady orgs copying the text and title) ran a tsk-tsk pearl-clutching article about Democratic “hypocrisy” titled “Biden Spotted Maskless on Federal Property Hours after Signing Mandate.” This is disinformation. That article is lying to you. And you should be able to have figured that out on your own. Once you know (and by now you should) that FOX News is unreliable, the moment you saw this story you should have fact-checked it. Remember, FOX News itself has admitted in court that nothing it says can be trusted, and perversely claims “everyone knows this.” Only, FOX viewers literally don’t, and that’s the problem. But here we have a concrete example (there are countless others).

The executive order Biden signed actually orders everyone on federal grounds to follow “CDC guidelines” on pandemic safety including (sic) “wearing masks when around others.” Biden was not around others during his speech. He was thus in full compliance with his own executive order. Not telling you this, and ranting on about a non-existent hypocrisy instead, is lying. Yes, they didn’t “completely” lie (the order exists; the speech happened); but they have so deviated from the truth as to trick you into thinking something was true that wasn’t. And they did this on purpose. That’s not a mistake. That’s lying. And you should not trust sources that willingly lie to you about factual reality.

Worse even than lying, this disinformation is actually publicly dangerous and a threat to national security. That’s right: a threat to national security. We have already suffered more casualties from covid-19 in one single year than we lost in the whole entire span of World War II; in fact over one hundred times more casualties than all terrorist attacks on US soil in the last fifty years combined. Yes, that includes 9/11. By lying about this, FOX is using this story to discourage people from engaging in necessary safe behavior for the public good—worse even than “Loose Lips Sink Ships” and “Blackout means BLACK” and other behavior that risked thousands of lives in WW2—by falsely claiming to you that the safety standards you are being asked to follow for the national interest are stricter than in fact they are, so you will be less likely to comply with any real safety measures, and even more likely to attack those measures and thus spread dangerous disinformation, like a literally deadly virus, all because you falsely believe those standards to be unreasonably strict and not consistently followed even by those advocating them. And if you believed this, this will be a false belief you hold because you believed liars. Because you believe FOX News. So, stop.

Get in the habit of testing sources for lies and manipulation. Don’t just trust any meme or claim or tweet that comes across your view: trace where it came from, what the source is, and vet the source. Can you trust that source? This is the first question you must ask, and answer. So you have to find the source first. And if it turns out to be a source you have already previously established is untrustworthy, you can probably just dismiss all stories coming from it; waste no further time. Anything true will get picked up and reported by an honest source that reliably fact-checks what it repeats. But if you are worried it won’t, or it’s important to you to know sooner rather than later whether some claim is true, then fact-check it yourself.

This is also how you will test a source, case after case, for its honesty and reliability. As I did here: I checked what Biden’s order actually said (I didn’t trust the news to tell me the truth about that, and I didn’t need to: Biden’s order was freely available and easy to find), and I checked whether his speaking without a mask complied with it (I didn’t trust the news to tell me the truth about that, and I didn’t need to: from FOX News’ own photographs I could see it for myself). Thus, learn how to do this, how to fact-check a story from a source by yourself, to determine how honest and trustworthy that source is. This is tool number one.

You should never again trust any source of news that tries to manipulate you with lies. You should be outraged that they even think they can con you like this. And you should be ashamed if you ever let them. You need this outrage and shame, because it is the only thing that will protect you from being manipulated by liars. Nothing else will. Because no reliable source would ever allow or employ these tactics. Any excuse you then make to disregard this lesson and go on believing liars deliberately manipulating you means you are volunteering to be their puppet, and to continue cultivating false beliefs about the world in exactly the way, and for exactly the purposes, they want. Don’t be someone else’s puppet. Be an independent thinker, be someone who catches liars, rather than believes them; be someone who stops trusting them once they’ve been caught. This is tool number two.

And this applies all the way down the information food chain. If you find that you can’t trust a source, by its repeated demonstrations of dishonesty or rampant incompetence, and whatever information-channel by which its news got to you continues to make the same error, of repeating to you claims that turn out to be false, from sources that turn out to be unreliable, abandon those channels. Whether it’s a Twitter feed, a Facebook friend, or whoever, whatever. Nix them from your life. Stop listening to anyone who does that enough times to establish you cannot trust them to vet these things for you. They are an unreliable curator of information; so stop wasting time listening to them. Find reliable curators to replace them with. This is the only way to protect your tether to factual reality.

There are many ins and outs to using these tools effectively. For example, sometimes you can’t fact-check something because the information is not independently available to you, and in that case you just have to stick with the previously established prior probability that the source you are trusting is honest and competent. That is why you must fact-check sources: every time you can check them is an opportunity to test them and build up an expected rate of accuracy for them. This also teaches you things about their methodology, e.g. is it sloppy and based on innuendo and conflating opinions or inferences with facts, or does it follow sound professional industry standards in journalism, e.g. identifying and describing (even linking to) their own sources; requiring multiple sources for each fact reported or else admitting when there is only one; clearly distinguishing opinion, inference, and fact; and passing each story through fact-checkers tasked with trying to debunk it before publishing (learn about the SPJ Standards and School of Journalism Standards and why they work; then look for media that use them consistently). Because sources that repeatedly fail to employ good methodology you know will not be reliable enough to trust. These are the ways you can come to know, for example, that The Washington Post is far more reliable than FOX News.

…Or Rachel Maddow

So, this happened:

Maddow said [One America News] “really literally is paid Russian propaganda.” Herring Networks [who owns OAN] alleged that she made a false statement, in that OAN is not paid by the Russian government. In dismissing the suit on Friday, U.S. Judge Cynthia Bashant ruled that Maddow was giving her opinion based on an accurate summation of the article.

“A reasonable viewer would not actually think OAN is paid Russian propaganda, instead, he or she would follow the facts of the Daily Beast article; that OAN and Sputnik share a reporter and both pay this reporter to write articles,” Bashant wrote. “Anything beyond this is Maddow’s opinion or her exaggeration of the facts.”

This is true as a matter of law. It is exceedingly difficult to win a defamation suit, as by various devices the law literally protects false or misleading statements. But in the world of common thought, Maddow did make a false statement, by conflating a weak inference (“because one reporter works for both OAN and Sputnik, therefore maybe OAN works for Sputnik”) with a fact (“OAN works for Sputnik”). And you could have been misled by that, if you aren’t savvy enough to realize she is making an inference that may or may not be all that sound, and not stating something she or anyone proved.

Yes, there is a reporter who works for both companies, and yes that is cause for concern, enough to do a double-check on anything published under that reporter’s byline; but there are lots of ways a reporter can find themselves working for multiple media sources, even dubious ones, not “just” that OAN is buying Russian propaganda or even conspiring with Russia to publish its propaganda. The one thing you cannot confidently say is “OAN really literally is paid Russian propaganda.” And accordingly you should never quote Maddow saying that. Because it isn’t factually true—as the fact that her network did not file a truth defense against that lawsuit makes clear.

If Maddow can so easily botch this statement, we can reliably infer she may be botching other statements; so we should start vetting her reliability, to see how often she misstates the facts. Just as with FOX News. To be clear, I don’t intend any false equivalency between Rachel Maddow and FOX News. FOX is all but 100% unreliable, a shit show of persistent lies, distortions, and manipulation. How reliable Maddow is, by comparison, is something you need to learn how to figure out for yourself. That’s the point. But what you need to burn into your brain is this simple fact: you don’t “figure it out” by in-group-bias procedures. “She always lies because she’s a liberal” or “she is lying because I don’t like what she says” are each as false an inference as “FOX always lies because it’s conservative” or “FOX lies because I don’t like what it says.”

Ideology or tribe, or how you “feel” about what is being said, have nothing whatever to do with reliability. The only reason we can conclude FOX News lies over and over is because we have caught it lying over and over. Period. An actual epistemic standard, not a tribalist gut judgment. You need to actually vet Maddow. As in, check her facts; build a base rate of accuracy for her. Don’t “vet” her ideology; vet her reliability. Apply the same standards you expect anyone to meet, to every source you want to use, even the sources that “agree” with you. Use methods that are ideologically blind, that would catch liars and misreporters whoever they are.

And here again you need basic media literacy. Professional news media sources (whether text, audio, or video) have always held to a standard of demarcating and distinguishing editorials from reporting. Opinion pieces are not held to the same standard as straight news, and you are expected to know that. Earlier when I cited a statement from FOX News admitting in court nothing it says should be regarded as true, it was actually talking about Tucker Carlson, an opinion presenter, not a news presenter. It just so happens that the argument FOX gave for Carlson not being reliable also applies even to FOX’s straight news, as we can find exactly the same distortion filter there—indicating FOX is not actually demarcating opinion from reporting. But we can’t assume that; we have to discover it, by fact-checking it. In more reliable news media you won’t find so much of this; there, opinion reportage is different from straight reportage.

Rachel Maddow is a pundit, not a reporter; an opinion writer, not a journalist. Never trust her on any claims to fact. She will draw inferences and opinions from facts; that’s what she was hired to do and what people watch her for, but you need to not use her as a source for any facts. She is not doing factual reporting. She is gathering facts others reported, and using them to arrive at opinions. Reliable news companies will keep these two things distinct; so you need to as well. If someone is a pundit, an opinion reporter, an editorial writer, do not confuse them with a journalist. Even a news reader is not a journalist: they are just reading off a teleprompter what someone else researched and reported (whom you won’t even learn the name of, which is why TV news is so unreliable; stick to print). Remember the rule: always trace a claim back to its source, as far as you can go. So you need to go check Maddow’s facts yourself, and only believe the ones that hold up, and abandon the ones that don’t. Never just trust her to have the facts right. Same as Sean Hannity on FOX or any other pundit: these are not reporters. Their own networks admit this. So don’t trust them like they are. Always distinguish readers and pundits from actual journalists and reporters.

Understanding this is crucial because it’s a symptom of “conspiracy thinking” (a.k.a. “incompetent thinking”) to find an opinion presenter unreliable and then conclude the straight news from that same media source is equally unreliable. That will almost never be the case. Even in the case of FOX News it would have been irrational to make that inference; that conclusion can only be reached with evidence. So never find yourself ever making that argument to dismiss a news source. “Maddow is wrong about everything, therefore NBC News is wrong about everything” is irrational. You should be scared the moment you ever catch yourself thinking that way. Because it means you are starting to develop one of those trap-loops I warned you about, which only serve to trap you in a delusional worldview, by cutting you off from reliable sources of information.

Another such trap-loop is to fall into another common cognitive bias: black-and-white thinking, an “all or nothing” mindset. In no way whatever is NBC News the most reliable source of information. It is far more reliable than FOX News, as it adheres to many of the ethical and epistemic standards demanded by professional journalist organizations. But it isn’t going to be right all the time, or make the right calls all the time. As with most video media it will over-simplify, report selectively, and make a lot of mistakes. But it will rarely if ever outright lie to you to manipulate you the way FOX does. NBC’s mistakes and errors and distortions will also be fewer and less egregious; and often corrected. Even when it aims to influence you, it will tell you it is doing that. That is actually a major difference between a reliable and an unreliable source. As is a source’s readiness to correct a factual error. And to state or link to its sources of information. And to correctly describe what its sources actually said. And so on.

A source is not “never to be trusted” nor “always to be trusted,” so as soon as you catch yourself thinking that, stop. Because that is the beginning of a delusional trap-loop. Even FOX News, for example, will occasionally say something true; and even The Washington Post, for example, will occasionally say something false. Merely catching it at one or the other does not dictate its entire reliability. You need a trend, and it needs to be strong. If a source is usually accurate, then grant that it is and act on it. If a source is usually lying, then grant that it is and act on it. And decide whether it is “usually” doing either based on an evidential fact-check of multiple claims across a span of time; don’t base it on evidence of its ideology or bias, as neither actually matter when sound methods are used. A source can state its bias, and still fact-check its premises reliably enough for you to trust their factual claims—even if you don’t share their opinions, which are simply the conclusions they reach from those facts. A source that makes it easier for you to tell those apart (facts from conclusions) is by definition a more reliable source for you to use than a source that doesn’t do that. Never confuse your disagreement with someone’s conclusions as evidence of the unreliability of their facts.

• This lesson does not apply only to Rachel Maddow and NBC News. It applies to literally everything. Always tell the difference between opinion reporting and straight reporting (know which you are looking at); and don’t rate a straight news source’s reliability by the reliability or even content of its editorial and opinion sections (rate them independently). And never dismiss a source as wholly unreliable (nor gullibly treat it as wholly reliable)—always maintain a sense of its actual relative degree of reliability, and act accordingly.

Epistemically, once you have a sense of a source’s degree of reliability, you know that the higher that degree is, the more you can trust it on claims that are not controversial or weighty. Claims that are controversial or weighty might still call for a fact-check to make sure the source is reporting reliably and competently. The prior probability that a source has things right meets with the converse prior probability that a claim is shocking or dubious enough in itself to be false, leaving you with a balance of both: the more dubious a claim is in itself, the more reliable a source needs to habitually be to trust it. Whereas with unreliable sources, sources that lie a lot for example, the reverse is the case. Though you shouldn’t assume everything they say is false, you still should not trust anything they say is true—until it survives your own independent fact-check, or that of someone else (or some other source) with a proven track-record of reliability.

And here we get to another trap-loop you need to avoid: because no source is 100% reliable, even the most reliable sources you can usually trust for most everything they report, you must be a critical thinker, not a dismissive thinker. Finding rare, isolated examples of errors or distortions does not logically warrant concluding that source is never to be trusted again. If you ever catch yourself reasoning that way, be scared. And stop. Because that’s the beginning of delusion. You need a well-established pattern of a high frequency of uncorrected errors before getting to that conclusion. Attend to every one of those required steps before getting there. Never “jump” there too soon. Only a delusional mind does that.

However, like egregious failures of methodology (rather than mere mistakes), lying is far more indicative a problem than mere errors. So once you actually prove a source lied to you, that weighs a lot more against you ever being able to trust it, than if it merely made a mistake, even a mistake caused by an ideological bias. Which means you need also to be able to tell the difference. How do you tell when a false statement is an error, evidence merely of a bias slipping through cracks in their methods, or a deliberate, intentionally-manipulating lie. Delusional people—incompetent thinkers—will decide this based on their “feelings” rather than evidence. That is a trap-loop. “They are liberal, they made a mistake, therefore they lied and are trying to manipulate me” is even more fallacious and irrational a train of thought than Maddow’s “because one reporter works for both OAN and Sputnik, therefore OAN works for Sputnik.”

After that, of course, you still need to apply all the standard tools of critical thinking beyond mere media literacy. Good sources will spell out their methods, describe their sources’ value and known biases, seek and publish comments from both sides in any dispute, demarcate facts from opinions or conclusions, choose words carefully (using crucial descriptors like “seems” and “alleged,” so always look very carefully at a source’s exact choices of words), and otherwise write in a way that makes it easier for you to evaluate the strength and merits of what’s reported.

You can often tell when a story is not so secure on its facts, because it will have told you it is not, by giving you enough verbal clues and honest information to make clear that it is working from incomplete or unreliable information and only trying to get at something otherwise hard to get at. Use that. Don’t fall into the trap-loop of dismissing a source because it does that, when to the contrary its doing that is what makes it a reliable source, not an unreliable one. It will have a higher error rate for reported facts that it clearly gives you enough information to tell are less certain. But that does not translate into the source itself having a higher error rate, but in fact a lower one: the more consistently a source demarcates certain from less certain information, the less often it will really be “in error,” because what it reports as uncertain you already expect to frequently turn out wrong. So read and use sources with enough critical intelligence to tell when it is reporting something less certain than when it is reporting something with much greater certainty (such as reporting how something “seems” rather than how something “is”).

Model Procedures

Here is the advice I want you to follow from now on:

Before asking me whether some story or claim you read is true—like, say, “Archeologists Find the Apostle Peter’s House”—show me that you deserve any of my time and attention by spending some of your own on the question. Come to me already armed with your own attempts at a fact-check and what you found: state to me what you already did to vet that claim, and what you found, and what you think those findings mean for answering your question. Tell me you traced it to its original source and vetted the general reliability of that source. Tell me you looked for corroboration in more reliable sources—like, say, an article in a peer reviewed archaeology journal—and tell me what you found. And tell me what you think that means—if you could find no peer reviewed article claiming “Archeologists Find the Apostle Peter’s House,” how likely is it, do you think, that archeologists actually found the Apostle Peter’s house? If you did find a corresponding peer reviewed article, or the original source whatever it is, does what it say actually match that headline, or does it actually say something else? And what it does say, if it is even remarkable, what evidence does it provide that its claims are true? And do you think that is a sufficient amount of evidence to reach that conclusion?

Then I’ll know this issue matters enough to you to matter in any way at all to me. If it’s not important enough for you to do any of that, then it will be even less important to me. After all, why should I spend any time on something you don’t even really care that much about? If you care, you’ll do the work. Then I’ll know you care. You’ll also have saved me time and effort by bringing me a package of research I can now work from to carry the ball further for you; so I will reward you for that with my own effort in turn.

Likewise, when you want to make some assertion in defense of some point you want to make at me—like, say, that there is actual video evidence proving Australia has Muslim “No-Go Zones” (see There Are No Muslim “No Go” Zones) or that peer reviewed science actually proved there is no systemic racism in the American justice system (see Actually, Fryer Proved Systemic Racism in American Policing)—stop. Double check first. Don’t just “believe” something that satisfies your preconceived ideas or ideological expectations. No matter how “right” or “true” it sounds or “feels,” make sure. Because you ought to damn well know I will check. So don’t embarrass yourself by gullibly trusting a bad source of information. Make sure that source got the facts right, and that you are reporting them correctly and without bias or distortion. And this entails the same procedure: trace the claim to its actual original source (thus bypassing all distorting filters on its way to you); vet the general reliability of that source; read the actual original story or article (which often will say very different and less impressive things than you thought); accurately report what that source actually said; and critically evaluate the evidence it gives for that claim, which means answering one simple question: would you trust a conclusion based on that same kind of evidence if the conclusion were against you?

That last technique I’ve explained before as cognitive Forced Perspective, giving then the example of testing the validity of your beliefs about feminism:

A lot of the reasons people give for not wanting to call themselves a feminist, or for saying feminism is bad, are identical to reasons people give for not wanting to call themselves an atheist, or for saying atheism is bad. If those reasons make no sense in the case of atheism, why should those reasons continue to be used in the case of feminism?

The tool being used here is an intellectual manifestation of forced perspective: look to see if the reasons you have for maintaining a position or attitude will hold up when used analogously on something else suitably similar, especially something you hold the opposite position or attitude about. And if they don’t hold up, ask why then are you still using them?

A good example of that is how I now apply this rule whenever forming or hearing judgments about women: I immediately reverse her gender and query whether the reaction or opinion would be the same. I’ve done this for so long now that it has become an almost automatic check on the sexist filter my culture installed in me. Which in turn has exposed to me how many sexist assumptions I had in me that I wasn’t even aware of. And likewise in others.

The basic idea? Flip the gender (or race, or religion, or nation, or person, whatever the analog is), swap one in for the other, and then see if your thinking gets the same results. Question why it doesn’t. And to do all that, you have to follow rule number one for all critical thought: check the facts. Especially, go directly to the source: not what someone says a woman said, but what she actually said, and the actual context in which she said it.

A similar tool is one I’ve detailed already elsewhere: the first objective of any genuine critical thinker is to do their damnedest to prove themselves wrong, before ever believing they’re right (see Advice on Probabilistic Reasoning). Cognitive forced perspective combines with that: here the trap-loop of an incompetent thinker (you see this in Christians all the time) is to only try to prove yourself wrong using weak or inept methods, methods that could not possibly succeed in discovering your beliefs are false. And the kicker is, even an incompetent thinker would immediately recognize this if they saw any of their ideological opponents using those methods. You need to treat the things you believe in with the exact same fierce, critical testing as you treat the things you don’t believe in. Because if your beliefs cannot pass the same test you expect things you don’t believe in to pass, you are probably trapped in a delusion. The trick of not subjecting your beliefs to the same tests, the same standards—of replacing those with impossible or useless standards or tests instead—is the very trap-loop imprisoning you in a delusion. Get out.

I’ve already written up some guides for this, with tips and examples for how to spot unreliable stories and unreliable sources in news media:

• From Lead Codices to Mummy Gospels: Essential Links on Dubious Tales
• Shaun Skills: How to Learn from Exemplary Cases
• On School Nutrition: How Republicans Create an Alternative Reality

A separate issue is how to be critical of articles about actual peer reviewed science (see, for example, Dumb Vegan Propaganda: A Lesson in Critical Thinking) and when to distrust or be critical of an expert consensus (see On Evaluating Arguments from Consensus). I have more to say on this point below. These same standards also tell us when to trust these sources: when to believe a story in a news article or the findings in a science journal or the assertions of an expert consensus is when it passes these tests—as opposed to “skepticism” leading you to just irrationally distrust every source you don’t agree with: that’s a trap-loop.

Antifa Did Not Storm the Capitol

Here is a working example to test your skills on: a day after the Capitol insurrection, after fake stories began circulating within hours of the attack that it was “really” caused by Antifa posing as Trump supporters, The Washingtion Times ran an article headlined “Facial Recognition Firm Claims Antifa Infiltrated Trump Protesters Who Stormed Capitol,” in which it says “a retired military officer told The Washington Times that the firm XRVision used its software to do facial recognition of protesters and matched two Philadelphia Antifa members to two men inside the Senate” and that “the source provided the photo match to The Times.”

The Washington Times slogan is “Real. Trusted. News.” This article was none of those things. The Times has a marginal reliability rating from Ad Fontes and GroundNews (which are two useful sites for this purpose), also scoring high on right-wing bias (which doesn’t tell you which stories it publishes are false but does tell you which stories and angles it pushes to be most suspicious of and thus fact-check). It has been flagged many times posting false stories. In fact it scores worse on all three measures than FOX News. And here it allowed a story through that failed every professional standard of journalism. Which is how I immediately recognized it as a false story. So should you.

This is how you know you can’t trust a source:

• It does not name its sources or explain why they are not named (much less give a good reason), nor does it explain how their sources have the information in question or what possible biases they may have that could affect their reliability. Reliable media rarely omit these things, much less fail on every single measure of them in a single article.
• It does not even try to find a corroborating independent source (much less cite one), nor even to check their source’s claim with their own stated source: at no point does this article say it ever even tried contacting the “facial recognition” company XRVision to confirm their story, least of all tell us whether they corroborated it or not.
• Its stated evidence for the conclusion of the headline (evidence we would later find out is fake) does not even merit their conclusion. Antifa isn’t an organization, for one thing (so any story assuming that it is is probably false); but in neither case is any evidence mentioned of the persons in question ever associating with any Antifa writing or protest actions.

Any reliable source would have caught all the above mistakes at the independent fact-checking stage before publication. Which means The Washington Times does not have an independent fact-checking stage. And this is one of the surest indicators of a source you can never trust: because if it didn’t have that for this article, it evidently doesn’t have this for any articles. Therefore, you should never, ever trust The Washington Times.

The third error is one you can spot on your own immediately: when headlining conclusions don’t even logically follow from their own stated evidence, you know you have a source whose headlines you cannot trust. That by itself wouldn’t nix a source. It can actually be a good sign that a source tells you how it is drawing its conclusions, and then merely reasons incompetently. Then you know its conclusions can’t be trusted; but maybe at least its factual reporting still holds up. But when it puts its dubious conclusions, not its facts, in the headline, that’s a bad sign. And of course when even those facts turn out to be false, well. You know what then.

In this case the Times article claims one of the two people identified had “a tattoo that indicates he is a Stalinist sympathizer,” which is not the same thing as Antifa; it also doesn’t explain how a tattoo could or would even indicate “Stalinism.” That demonstrates incompetence at basic reasoning. The other person in question was reportedly documented to attend “climate and Black Lives Matter protests,” neither of which is Antifa—and any source that doesn’t know that is too incompetent to trust. Indeed, one might wonder if “attended” is even being honestly clear here: as a protester or counter-protester? I’m sure many a Trump supporter has been photographed “at” BLM protests; that’s not the same thing as joining that protest. Which illustrates the importance of attending to the exact words used to push a story. Are they conning you with coy language?

Those were definite red flags. But really it was the first and second errors that guaranteed this story could not be trusted. The extremely poor handling of sources indicates even the facts cannot be trusted, much less the conclusions. Of course I was prompted to immediately check if XRVision even existed (it does), much less had a press release about this remarkable, newsworthy result on its website (it didn’t). Then I immediately looked for anyone else independently reporting this story—anyone who actually spoke to XRVision. I found no one.

Comparing this dismal set of facts with the extensive video evidence of actual Trump supporters driving the assault and engaging in all the violence and even boasting about it on their own social media, no one should have believed this Times article. No one. And anyone who did, should have thereby learned they have a seriously defective epistemology that they need to fix right away. You should never be this gullible. Media literacy—understanding what real journalistic standards are and thus how to spot fake stories by the complete abandonment of those standards—is crucial to your recovery. Learn it. Live it.

I was right of course. Within mere hours XRVision denounced the Times story as completely false, indeed even libelously so. It was all made up. In response, The Washington Times did issue a correction and a completely rewritten story. XRVision had in fact id’d Nazis in the Capitol attack, not Antifa. So the Times tries to save face by now claiming “There is other evidence Antifa members may have been there” and then commits the same incompetent sin again: it makes a claim but cites no actual evidence, just asserting without explanation that “The New York Post, quoting a law enforcement source, said two Antifa from New York were in the crowd.” In fact this is an even worse example of bogus evidence: That Post article cites only anonymous sources, doesn’t explain how it knows those unnamed sources actually “are” law enforcement in any relevant sense, and does not explain how those “unnamed sources” verified anything. “Someone we won’t name says they saw someone who maybe sort of looked like someone else they think they might have once spotted in another crowd” is not what competent people consider “reliable evidence,” and no reliable source would ever use such information. So now you know you can’t trust the Post either. These are the red flags you should be looking for.

Incidentally, only one anti-Trump participant in the Capitol attack has actually been identified, and he did not lead or engage in any of the breaking-in, violence, theft, or property damage, he wasn’t a supporter of Antifa but BLM—which any competent person should know are not the same thing—and he was only filming the event from the inside for posterity and, being himself black, pretending to be one of them to avoid being killed. You should be able to confirm every single one of those points yourself with any competent search of reliably-sourced reporting online in under ten minutes time. Test your skills: see if you can find reliable source material on this, and contrast it with unreliable source material on it.

Finally, for a different skill often overlooked in critical thinking, you can explore the example in Critical Thinking isn’t Just a Process, which teaches you how to look for information in what is being omitted from a source—what someone is not saying, or is avoiding saying—using another Trump administration example, but the principle generalizes to every subject and source. When you can put together how someone is concealing information, not just in whether they are giving false information, you can learn a lot about their reliability as well as about what the truth is that they are trying to hide. Obviously you can’t always know what information a person has they they are keeping to themselves; but you’d be surprised how often you do know what information they should have, and by analyzing how they talk and dance around it or use carefully parsed wording, extract it.

Useful Resources

For guidance in knowing what to look for in identifying unreliable media, and thus what to look for to identify reliable media—since that will be any source that regularly does the opposite, point-by-point, of what unreliable sources do—check out these:

• Reading NPR’s A Finder’s Guide To Facts and Harvard’s 4 Tips for Spotting a Fake News Story will add some essential media literacy.
• On how to critically interact with modern media see Bruce Bartlett’s book Truth Matters (“A Citizen’s Guide to Separating Facts from Lies and Stopping Fake News in Its Tracks”).
• On detecting online disinformation campaigns see Geoffrey Fowler’s “You Are Probably Spreading Misinformation: Here’s How to Stop.”
• I strongly recommend watching Shaun’s 40 minute docu-video on Outrage News for examples and techniques of media manipulation and how to protect yourself from them (in short: do what he does).
• Then bookmark and often refer to the Ad Fontes Media Bias Chart. Below that is an alpha list of media sources, linking to a full entry on each.
• Another resource that does the same is Media Bias FactCheck which has an even larger database of worldwide sources it vets. Plus it can be handy to have two independent evaluations to consult.
• And a valuable read is Perry Bacon’s “Eight Things the Trump Era Taught Me About Covering Politics,” which is about what a political reporter’s methods should be, but you can use it to detect which political reporters are following more rather than less of his toolkit.
• For those who want to pay or register for rigorous reliability evaluation of news sources, see NewsGuard (that’s what actual journalists use).

Note that with the Ad Fontes bias-and-reliability chart, each entry will say that a reliability score of at least 25 (out of 64) is “okay” and at least 33 “generally good” but in my experience that is dangerously overgenerous. From my own experience vetting sources on their chart, really you should never trust any source that scores below 33 (sorry, MSNBC); you should never implicitly trust any source that scores below 40 (sorry, CNN TV), and you should always prefer sources that score at least 42 (e.g. CNN Online, The Wall Street Journal, Reuters, The Washington Post—keeping in mind, for the reasons I’ve explained already, never to confuse their opinion/editorial sections for their news reporting).

There is also a decent article on What Are the Best Nonpartisan News Sources? that is worth familiarizing yourself with. Partisanship does not correlate with reliability. But knowing in what direction the errors in a source are likely to be can be helpful in knowing what they report that you should double-check before trusting and when. Moreover, a competent thinker should know they must never “echo chamber” themselves by only reading and trusting sources that agree with them; so knowing what the best sources are on the other side of your views, and reading them with some frequency, is important to competent thought. This is how you keep yourself tethered to reality.

And of course learn how to rely on existing resources for independent fact-checking claims and stories:

• Snopes
• FactCheck
• PolitiFact
• PunditFact
• Fact-Checker
• CISA Rumor Control
• Reuters FactCheck

There is also one reasonably reliable conservative-leaning fact-checking website:

• Check Your Fact

I list additional resources in my own “Skeptical Humanities” article. There are also already resources tracking fake sources that you should make use of, like RealOrSatire, which will tell you if a site you are looking at is actually a deliberately fake site or not. For advice on dealing with video evidence critically, see How to Spot a Fake Video. And if you want to do a deep dive into the psychology behind all this, a really good resource is The Oxford Handbook of Media Psychology.

The Question of Peer-Review

There is also the matter of “peer reviewed” scholarship. One way to narrow a list of sources that way is to use Google Scholar. Not everything it picks up is peer reviewed or even pretending to be. But a lot more of it will be than if you searched just “Google.”

There are other academic databases but few accessible to the average person—other than possibly through your local public library. With a free library card in my area of residence, for example, in Georgia I can access the Gale Database, while in the California Bay Area I could access the entire contents of the JSTOR database. You can sometimes pay for access. For instance, for a couple hundred dollars a year I maintain access to JPASS, which is a subsection of JSTOR. I also subscribe to the L’Année Philologique which is a fairly thorough database on ancient classical history, although apart from also being expensive is primarily in French and thus requires my knowledge of French to navigate. There are many other databases to look into, from Atla Religion (expensive and rare) to PhilPapers (online and free) and beyond. For some hard sciences, ArXiv is handy, but much of its content is pre-peer preview, so extra caution is warranted. And so on.

Not all peer review is equal, however; or even real. Different fields have different standards; and different journals vary extremely, from entirely bogus to very stringent. For vetting purportedly “peer reviewed” sources read the Calling Bullshit guide “Research Tools: How Do You Know a Paper Is Legit?” as well as the Stony Brook resource “How Do You Know a Journal Is Legitimate?” and the Oregon State resource “Scholarly Articles: How Can I Tell?” (you might find some useful tools as well at ISTE’s “Top 10 Sites to Help Students Check Their Facts”). You can also check the raw metrics for indications of the quality of a peer-reviewed journal, but it is difficult to tell when those are fake or trumped-up; and they aren’t commensurate across disciplines (e.g. a well-performing philosophy journal does not share the reliability of a science journal).

The general rule is that a “real” study means one that has actually been peer-reviewed and recognized as such by the mainstream majority of experts in its respective field. The problem has become exacerbated by the rise of fake journals (part of the Age of Disinformation we now live in), as well as rising academic fraud. So it can now be a chore to figure out which journals are legit (or at least baseline trustworthy) and which are bogus (or at least untrustworthy). The above links provide some tools and advice on that. More are likely to be developed, because the problem is growing (please point out suggestions in comments so I can keep this updated).

But on top of that, you also have to understand that even quality peer-review is only a higher bar for truth and accuracy, not a guarantee of it. In this sense, quality journalism is peer-reviewed. At such venues, in their news (rather than opinion) sections, everything their authors write is subject to (in-house but nevertheless independent) fact-checkers and editorial review for standards compliance. For example, the Washington Post and The Wall Street Journal follow these standards; while most non-print news (i.e. television news and news sources that exist solely on the web) and tabloids do not (or not enough to merit). Academic peer-review operates similarly, but is (ironically) more prone to bias and vague or inconsistent standards.

So, whether in journalism or academics, peer-review is not a guarantee of accuracy or truth, but it is an important bar that weeds out a lot of poor or dubious claims. You still have to approach even legit peer-reviewed articles and studies critically (as I suggested earlier here). I have since written more extensively on that point in The Korean “Comfort Women” Dust-Up and the Function of Peer Review in History, where I explain the process more and how to still be critical of its results. But even apart from having to be wary of bogus or less reliable journals, and wary of poor work that slips through the net even in quality journals, you also have to be wary of entirely legitimate work that is nevertheless still wrong. Because peer review does not even attempt to eliminate that.

Peer reviewers do not have to agree with the conclusions of any article or study they pass; they only have to agree that it was argued using sound standards, and none of its factual premises are obviously false. It thus ensures an article or study is at least worth your trouble to read and evaluate (unlike stuff published online by randos or propagandists; their reliability you have to assess for yourselves). And for that I have written elsewhere on how to adjudicate between legit but conflicting articles or studies: in On Evaluating Arguments from Consensus (as I recommended earlier) and Galatians 1:19, Ancient Grammar, and How to Evaluate Expert Testimony (which takes up a specific example in a specific field of history, but its advice can be generalized to any dispute in any field, even journalism).

Conclusion

Recently a random doctor on Twitter claimed “Biden is now reversing the executive order put into place by President Trump to reduce pricing for insulin and epinephrine” and stoked outrage at this by declaring “This is NOT a partisan issue and will harm Americans.” You shouldn’t trust a claim like that. You should always have fact-checked it before repeating or sharing it. Doctors are just as often liars and manipulators as anyone else; and being an expert in medicine does not make you an expert in political administration. So, check this stuff first.

What you will find when you do some internet searching for what executive orders she is referring to, is a completely different reality from the fake narrative she is pushing on Twitter. You can easily find those “orders” on your own, both Trump’s and “Biden’s” although, as you’ll discover when you do this, in fact there is no such “Biden” order. Biden actually didn’t directly do what this doctor on Twitter claims. But you’ll also discover the Trump order in question did not actually do anything either. It had only ordered community health centers to do what most were already doing; and community health centers do not set insulin prices. Nor had Trump’s order ever gone into effect: he only signed it in December (a month ago now), and it was set to take effect later.

You’d learn all this by finding reliable sources relating the details such as “Biden’s HHS Freezes Trump Insulin, Epinephrine Rule Until March” at Bloomberg Law (notice in the very headline it makes clear Biden himself did not freeze the order; his HHS secretary did) or “HHS Freezes Rule Affecting Community Health Center’s 340B Drug Discounts” at The National Law Review (notice in the very headline it makes clear this wasn’t an insulin pricing policy but a policy governing only community centers). And above all, you should have easily found and read the actual pause order itself. Which never mentions insulin. It only says to pause all rules until they can be reviewed; and it just so “happens” one of those rules was about insulin.

In actual reality—the reality you need to make sure you stay tethered to—Trump wrote a ton of bizarre, confusing, inexplicable executive orders around health policy, most in the last days of his Presidency, all of which directing policy changes to take effect in future. Biden couldn’t vet them all in a moment’s time, so he directed the HHS to pause all those rules to review each and every one; and any that pass muster by this March will be enacted, not quashed. So neither he nor his HHS administration “reversed” Trump’s order. That order had never gone into effect; and if it passes review, will go into effect. To claim otherwise is either flabbergasting incompetence or outright lying. So you now know you can’t trust this doctor on Twitter.

Another factual distortion in her Tweet indicative of fantastical incompetence or shameless lying is that the order in question only affected community centers—not, say, hospitals or primary care providers or commercial pharmacies, in other words all the places that in fact people usually buy their insulin or epi. And the order only directs that those centers pass on wholesale discounts on those drugs to their customers—despite there being no evidence that would even substantially change their prices. But above all, one should have red-flagged this Tweet right away with the simple realization that the Federal government can’t fix insulin and epi prices. This is why the order only covered “community health centers,” in fact only those that receive “federal grants,” the only way they could be directed to follow a policy to qualify for those grants. They could just stop receiving federal grants and charge whatever they want. The President has no other power here. And that’s civics 101—we should already know this.

Instead, we got a disinformation meme on Twitter, that countless respondents and sharers of that Tweet just gullibly “believed.” Almost no one flagged it as fishy or bothered to fact-check it. This is dangerous to society. And it is dangerous to you, if you behave this way. Never just believe these things. Always check them first. Always. And educate yourself in media and civics literacy so you will know when things like this sound fishy and so you’ll know how to vet them competently.

Worse, I found that when any of the people believing this meme were told this, they refused to believe me. They said irrational, illogical things like “I’ll believe a medical doctor before you” (in fact they weren’t asked to believe me: they could check the evidence for themselves), which is a trap-loop: an excuse to avoid doing the one simple thing they could have done to discover a doctor was lying to them, thus “trapping” them in that doctor’s lie, unable ever to escape, severed forever from the real world of actual facts. Some even made bizarre claims like that Trump’s order had reduced their insulin costs—when if they would just look at the facts securely in evidence, Trump’s order was actually written in December and was not to go into effect until late January, so their claim about their insulin prices having been lowered by it was literally impossible. Indeed it was impossible even apart from the fact that, had the rule ever actually gone into effect, they would have to have been getting their insulin from a “community health center” for that order to have even possibly affected what they paid for it. But the inability to comprehend even how a calendar works trapped these people in a delusional fantasy world contrary to reality. When even this was pointed out, they rationalized remaining in their delusion with nonsensical assertions such as “I know better what I paid for my insulin than you do,” a trap-loop assertion that is a complete non sequitur yet “sounds” and “feels” like a good reason not to look at any of the evidence refuting their fantasy.

In actual fact, pausing all last-minute rules for a couple months to review them first is what any competent leader should do when taking over an administration, especially one run as incompetently and shadily as the last was. Biden directed all his orgs to do this; so to comply with Biden’s instruction, his Health and Human Services department wrote an order to do this for all of Trump’s health policy directives; among those, was a directive regarding how certain community centers should price their insulin and epinephrine; a directive there is no evidence would have even significantly changed those prices, and which never went into effect yet, and which there is no evidence Biden’s HHS won’t end up approving in just a couple of months anyway. That’s reality. Everything else is a delusional fantasy fueled by runaway political emotions and tribalism.

Don’t fall into that trap. Inoculate yourself with real, actual media literacy and real, actual critical thinking; not a bogus “skepticism” that in fact blinds you to the real world and enslaves you to anyone keen to manipulate you.

-:-

Update: Even just after I published this article FOX News pushed fake news about Kamala Harris’s Book and Joe Biden’s Climate Plan. The account of how these stories originated and spread across unreliable conservative source networks is educational and shows why these sources can never be trusted. Though FOX was eventually shamed into retracting these absurdly false stories, they should never have survived any competent fact-check, and thus should never have aired in the first place. FOX News simply has no journalistic standards. That’s why it had to be shamed by other news sources into correcting these embarrassing mistakes.

You might also be interested in the reading and viewing list in How to Talk to a Science Denier. Although the advice there is sometimes based on what is actually a rather poor conversion rate relative to the problem (curing the delusions of a few hundred out of a few hundred thousand is actually not going to solve any societal problem), it can be precisely what you need when you have abundant time and a personal relationship with a victim of delusion. And among the resources listed is advice that operates on a macro-level as well.