An index of unreliable news websites
By Barrett Golding
Misinformation is a thriving industry on the internet, supported by social media shares, advertising dollars and political donations.
In the United States, scores of research reports try to measure how falsehoods spread online. These studies often require lists of untrustworthy “news” sources, but many of these lists have grown out-of-date and incomplete.
Better data means better results for researchers, reporters and readers. So the International Fact-Checking Network built a more complete dataset: an index of unreliable news sites.
To create the index, we combined five major lists (see below), then eliminated the sites that were no longer active. We only used lists that were public and curated by established journalists or academics, contained original data (rather than information from other lists), stated their criteria for inclusion and defined how they graded different sites (see our methodology for more).
The index’s initial release relies on the five major lists’ determinations and definitions of “unreliable.” Our next release will include more sites and the criteria we used for inclusion.
But for now, here are the databases of unreliable news sites that we pulled from to create our index:
- FactCheck.org’s Misinformation Directory (FC), created by the Annenberg Public Policy Center at the University of Pennsylvania.
- Fake News Codex (FN), widely quoted by Snopes and others, maintained by web developer and data designer Chris Herbert.
- OpenSources (OS), run by Merrimack University media studies professor Melissa Zimdars.
- PolitiFact’s Fake News Almanac (PF), by PolitiFact, a joint project of the Tampa Bay Times and Poynter.
- Snopes’ Field Guide to Fake News Sites and Hoax Purveyors (SN), created by Snopes, the oldest and largest online fact-checking organization.
This is only phase one of the IFCN’s unreliable news index.
The next phase of the project will be to automatically remove inactive sites, add sites by following URL redirects (which often lead to new fake news schemes) and harvest related domains.
For example, notorious conspiracy site YourNewsWire.com now redirects to NewsPunch.com to attempt to avoid detection by fact-checkers and the platforms. The SpyOnWeb research tool detects the relationship between misinformation sites, along with other domains connected by IP address or Google Analytics and AdSense ID (see Bellingcat’s site research guide for more on these tools).
Blacklisting bots, fraud and false-news sites
Fake news is a business. Much of that business is ad-supported.
Advertisers don’t want to support publishers that might tar their brand with hate speech, falsehoods or some kinds of political messaging — but too often, they have little choice in the matter.
Most ad-tech dashboards make it hard for businesses to prevent their ads from appearing on (and funding) disreputable sites. Marketers can create blacklists, but many of those lists have been out-of-date or incomplete.
Aside from journalists, researchers and news consumers, we hope that the index will be useful for advertisers that want to stop funding misinformation.
Methodology
Our index compiles existing site lists, curated by academic and journalists. For now, we depend on their expertise for accuracy. (A protocol to review and add sites is in the works.)
The site tags above come from those assigned by the original list curators. We grouped their differing labels into our set of six tags.
| Our tag | Their tag and description |
|---|---|
| bias: | OpenSources: "Extreme Bias: Sources that come from a particular point of view and may rely on propaganda, decontextualized information, and opinions distorted as facts." |
| conspiracy: | OpenSources: "Conspiracy Theory: Sources that are well-known promoters of kooky conspiracy theories." |
| clickbait: | OpenSources: "Clickbait: Sources that provide generally credible content, but use exaggerated, misleading, or questionable headlines, social media descriptions, and/or images." |
| fake: | Fake News Codex: "Sites that are fake,… A site doesn't need to exclusively publish fake content to qualify. In fact, many publish a great deal of authentic material, though it’s typically presented in a biased and tawdry fashion. This 'real' content serves as cover for the fake." OpenSources: "Fake News: Sources that entirely fabricate information, disseminate deceptive content, or grossly distort actual news reports." Politifact: "Fake news sites: There's little consistency of content or style among fake news sites — the common thread appears to be that they distribute fabricated content, but the reasons aren’t always apparent." Politifact: "News imposter sites: Adding to the fog of fake news online, several websites appear to try to confuse readers into thinking they are the online outlets of traditional or mainstream media sources." Snopes: "Fake News Sites and hoax purveyors… spreading fake news and outlandish rumors" and "false, disruptive claims" that "regularly fabricate salacious and attention-grabbing tales." |
| satire: | Fake News Codex: "Sites that are not necessarily intended to mislead (such as The Onion and its legion of imitators), but that can be misunderstood by naive readers."
OpenSources: "Satire: Sources that use humor, irony, exaggeration, ridicule, and false information to comment on current events." Politifact: "Parody or joke sites: Many of the deliberately false or fake news stories we see in social media feeds begin on websites that attempt to parody real news — imagine the humor website The Onion, but without the name recognition (or often the comedic writing talent)." |
| unreliable: | FactCheck.org: "Websites that have posted deceptive content."
Fake News Codex: Sites that are "extremely misleading… We do not include sites that merely have a clear political or ideological bias." OpenSources: "Hate News: Sources that actively promote racism, misogyny, homophobia, and other forms of discrimination." OpenSources: "Junk Science: Sources that promote pseudoscience, metaphysics, naturalistic fallacies, and other scientifically dubious claims." OpenSources: "Rumor Mill: Sources that traffic in rumors, gossip, innuendo, and unverified claims." OpenSources: "State News: Sources in repressive states operating under government sanction." OpenSources: "Unreliable/Proceed With Caution: Sources that may be reliable but whose contents require further verification." Politifact: "Sites that contain some fake news: Finally, some websites appear to get duped like the rest of us." |
The lists we combined to create the index had 1,043 unique domain names.
Of these, as of November 2018, 515 were still active and another 528 were inactive (51 percent) — either no longer online or no longer posting stories. We detected inactive sites programmatically by retrieving HTTP status codes (404s or 301s), using auto-generated screenshots and, in some cases, by visual inspection.
We curated the resulting list, trimming it a bit, by removing several sites whose stories, though highly politicized, were mostly not fake: alternet.org, cato.org, heritage.org, nationalreview.com, thedailybeast.com, theintercept.com, thinkprogress.org, and weeklystandard.com. We determined this by checking the veracity of their stories at PolitiFact and Snopes.
Several sites we reviewed had mostly false verdicts from fact-checking sites. These stayed on the list: addictinginfo.org, breitbart.com, dailycaller.com, dailykos.com, and judicialwatch.org
Our Google spreadsheet has additional data: the year of domain registration and the number of scripts each site uses for advertising and tracking (thanks to BuiltWith).
Corrections?
If you have additions or corrections, please use the form below to notify us. Remember: Our list includes only sites whose stories are demonstrably false — not merely biased or partisan. Send links to fact checks demonstrating whether the site you’d like us to review publishes fake or fact-based news.
Corrections
May 1: This index previously listed The Washington Examiner and FirstPost as unreliable news sources. After reviewing our methodology, we found that neither met the criteria for inclusion, so both were removed.
Written by Barrett Golding
Research by Barrett Golding, Factcheck.org, Fake News Codex, OpenSources and PolitiFact
Designed by Daniel Funke and Ren LaForme
Edited by Barbara Allen and Daniel Funke