A truth-telling app gets a rigorous test

A few minutes into the 2018 State of the Union speech, the president makes his first factual claim: “Since the election we have created 2.4 million new jobs.” FactCheck.org is on the case immediately, offering a quick take. “In Trump’s first eleven months, employment increased by 1.84 million,” it notes, “12 percent lower than the 2 million jobs that were created in the previous eleven months.” That quick take moves out on FactStream, an automatic fact-checking app getting its first beta test.

“This has been the dream,” says Bill Adair, director of the DeWitt Wallace Center for Media & Democracy in the Sanford School of Public Policy. “To have instantaneous fact checks. When we started PolitiFact in 2007, we got calls: ‘Wouldn’t it be great if we could have instantaneous fact checks, and it would pop up on your TV?’ ”

Wouldn’t it, though? FactStream is a step toward that fact-checking Holy Grail.

Not its arrival, though—a step. The FactStream app, downloadable for your phone, is the first undertaking of the Duke Tech & Check Cooperative, started in late 2017 by the Duke Reporters’ Lab and funded by a $1.2 million grant from the John S. and James L. Knight Foundation, the Facebook Journalism Project, and the Craig Newmark Foundation. The Reporters’ Lab works on new forms of journalism, commonly enlisting students in its research and other enterprises. In its first incarnation, FactStream combines fact checks by Politi-Fact, FactCheck.org, and The Washington Post and pumps them out as the president speaks. Some thousand users have already downloaded the app.

To demonstrate where he hopes this will all lead, Adair earlier asked the Amazon Echo on his desk whether Donald Trump opposed the war in Iraq. “Washington Post rated it four Pinocchios when Donald Trump said that ‘I opposed the Iraq war,’ ” Alexa said. Again, Alexa did not perform a natural-language search of the entire Internet. That’s eventual; this is now. What happened is that Alexa scoured an Internet fact-checking database. Working with Jigsaw (a Google subentity) and schema.org, which defines standards for the Web, the Reporters’ Lab has created a tag that identifies fact checks online. “Even better,” says Reporters’ Lab codirector Mark Stencel, the tags carry metadata: “This is a fact check on this topic, carried out by this person, on this date, that got this result.” So even the fact checks are checkable.

With further progress in natural-language processing and an ever-growing database of tagged fact checks, FactStream will be able to hear the claim and find the tagged fact checks all by itself. But “what you’re going to see tonight,” Adair says, “is human-powered. We’re using the fact checkers themselves to listen to the claims.” And they’ll access the same fact-checking database Alexa accessed, itself a major step toward that automatic future. Adair calls that cooperation among fact checkers to tag their checks the “secret sauce” that the Reporters’ Lab has introduced into the fact-checking ecosystem.

As the president speaks, the test goes well. A claim of “the biggest tax cuts” in American history is rated false (a link clicks you directly to an October fact check of that claim by PolitiFact); “that’s accurate” is the response to a claim that prescription drugs are cheaper in other countries. There’s a hiccup with The Washington Post fact checker, whose fact check URLs somehow complicated their absorption into FactStream (one almost led, somehow, to a Gossip Cop post exposing a false post about comedian Kevin Hart). Some users report having trouble getting to the stream. Finding problems like that, says Stencel—seeing how the app works for both users and fact checkers—“is the purpose of this whole event.” Adair rates himself “really happy with how the test went.”

Sounds true.