Call it fake news, bad information, or merely a well-intentioned reporter whose nose for news outweighs his/her data skills … learning to comprehend data in the news is tougher than ever before.
In this June New Yorker article, Michelle Nijhuis shares some of the strategies from the University of Washington’s course “Calling Bullshit in the Era of Big Data,” taught by Jevin West and Carl Bergstrom.
Here are some tips they recommend:
- Recognize that bullshitters are different from liars, and be alert for both. To paraphrase the philosopher Harry Frankfurt, the liar knows the truth and leads others away from it; the bullshitter either doesn’t know the truth or doesn’t care about it, and is most interested in showing off his or her advantages …
- Upon encountering a piece of information, in any form, ask, “Who is telling me this? How does he or she know it? What is he or she trying to sell me?”
- Remember that if a data-based claim seems too good to be true, it probably is. Conclusions that dramatically confirm your personal opinions or experiences should be especially suspect …
- Use Enrico Fermi’s guesstimation techniques to check the plausibility of data-based claims …
- Watch out for unfair comparisons. Claims that many more people watched the video stream of the Trump Inauguration than that of the first Obama Inauguration, for instance, failed to acknowledge the vastly greater availability of streaming video in 2017.
- Remember that correlation doesn’t imply causation. A correlation between two variables (ice-cream consumption and shark attacks) may well be due to a third variable (summer weather). These days, spurious correlations often emerge from data mining, the increasingly common practice of trawling large amounts of information for possible relationships …
- Beware of Big Data hubris. The Google Flu Trends project, which claimed, with much fanfare, to anticipate seasonal flu outbreaks by tracking user searches for flu-related terms, proved to be a less reliable predictor of outbreaks than a simple model of local temperatures … Like all data-based claims, if an algorithm’s abilities sound too good to be true, they probably are.
- Know that machines can be racist (or sexist, or otherwise prejudiced) …
- Mind the Bullshit Asymmetry Principle, articulated by the Italian software developer Alberto Brandolini in 2013: the amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it.