We sawin 1997, and in 2017. On Monday, a computer edged out a victory over people in a far more nuanced competition: debate.
IBM created a system called Project Debater that competes in what the company calls computational argumentation — knowing a subject, presenting a position and defending it against opposition. At a press event, IBM pitted the system against two humans with a track record of winning debates.
In one debate, Noa Ovadia overall nudged two people among a few dozens in a human audience toward her perspective that governments shouldn’t subsidize space exploration. But in the second, Project Debater soundly defeated Dan Zafrir, pulling nine audience members toward its stance that we should increase the use of telemedicine.
You’re not going to lose your job yet to Project Debater’s commercial spin-off right away as Big Blue tries to profit from the IBM Research project. Project Debater betrayed its inhuman nature several times over the course of its 20 minutes of off-the-cuff speech. But Debater did demonstrate that artificial intelligence can handle some complexities of human interaction, not just the clear-cut rules and victories of a board game or game show.
“Our life is not black or white. It’s ambiguous, it’s subjective,” said Ranit Aharonov, director of Project Debater. “AI will have to navigate in that territory.”
Project Debater was trained in advance on debating methods, but not the details of the debate itself, which it found out about only moments before the debate started. To formulate its argument, it had at its disposal a collection of 300 million news articles and scholarly papers — previously indexed for quick search results. But it had to find the information, package it persuasively, listen to its opponents’ arguments and formulate a rebuttal.
“It’s amazing to see this technology pull from 300 million sources and distill it into what sounds like a conversational narrative in debate,” said Clea Conner Chang, chief operating officer of Intelligence Squared Debates, an organization that runs its own debates with a similar argue-and-respond style.
Echoes of 2001: A Space Odyssey
Project Debater’s brain is a group of computers in a distant data center, but IBM chose to represent it as a black pillar on the stage. Sci-fi fans might have been reminded of the alien obelisk in— the book and movie that also made famous HAL 9000, a well-spoken but malevolent AI. Debater paid homage to the author, too, at one point quoting his famous maxim, “Any sufficiently advanced technology is indistinguishable from magic.”
Although IBM didn’t try to pretend Debater is human, it speaks in a well-modulated female voice, and researchers call it “she.” At the top of the pillar was a display showing its state of mind with soothing blue curvaceous shapes.
Three circles swapped places when Project Debater was thinking. They merged into into an oval with an waving sinusoidal line when it was speaking. And when it was listening to its human opponent, the three circles hopped up and down like the animations you see when your computer is waiting for a software update to install.
Debater showed a grasp of the nuts and bolts of debating. It marshaled evidence, told you its stance, explained how its argument would proceed and made its case. It quoted authorities, like German ministers touting the economic benefits of space exploration or scientific studies showing better health outcomes for diabetes patients monitored from afar.
Plenty of its words were lifted directly from its corpora of data, IBM said. But it does construct its own sentences, too, for example when presenting its summary of the arguments it plans to make. And it does plenty of other language work, too — assessing arguments, deciding which points to raise specifically or broadly, and evaluating the counterarguments made by its opponent.
Project Debater, run out of IBM’s labs in Haifa, Israel, began in 2011 when IBM was looking for the next challenge after its Jeopardy victory. Researcher Noam Slonim suggested a human vs. computer debate, and Aya Soffer, who runs IBM Research’s global AI team, liked it as well. It was one among 30 projects IBM evaluated, she said.
“From our perspective, the debate format is the means and not the end. It’s a way to push the technology forward and part of our bigger strategy of mastering language,” Soffer said. “In general, computers are lagging significantly in understanding and being able to express themselves. If we expect AI to be useful, being able to communicate with people is critical.”
It throws in jokes, too — human-written jokes, but it has to master the timing.
“I can’t say it makes my blood boil, because I have no blood, but it seems some people naturally suspect technology because it’s new,” Debater said in its closing argument in favor of telemedicine.
And arguing for space exploration subsidies, Debater said it supports technology, “being as I am a prime example of its power.” The audience chuckled, though an audience outside IBM’s Watson West offices in San Francisco might react differently to a computer that can figure out the right moment to inject that comment into a discussion.
There were gaffes, like the moment Debater said subsidizing space exploration “is more important than good roads, improved schools or better health care.” That statement would torpedo most would-be politician’s chances at election.
Debate ‘defines what it is to be human’
But Chris Reed, a professor at the University of Dundee, said we shouldn’t take the achievements for granted. He was impressed that Debater successfully figured out what raw material supported the argument it was making, that it got verb tenses correct and mastered other basics that have tripped up computer debaters.
Debater even demonstrated procatalepsis — the ability to anticipate its opponents’ arguments and preemptively attack them, Reed said. He was clearly impressed.
“Argumentation is one of the fundamental things that defines what it is to be human,” Reed said. “Argument and debate is the engine that drives the process of science, that characterizes what happens in political fora, that frames our conception of religion. if we can go any way toward building computer models that tackle these issues, we’re doing something really important.”