Last week, an IBM computer named Watson beat Ken Jennings and Brad Rutter, the two greatest Jeopardy! players of all time, in a nationally televised event. The Man vs. Machine construct is a powerful one (I’ve even used it myself), as these contests have always captured progressive imaginations. Are humans powerful enough to build a rock so heavy, not even we can lift it?

Watson was named for Thomas J. Watson, IBM’s first president. But he could just as easily have been named after John B. Watson, the American psychologist who is considered to be the father of behaviorism. Behaviorism is a view of psychology that disregards the inner workings of the mind and focuses only on stimuli and responses. This input leads to that output. Watson was heavily influenced by the salivating dog experiments of Ivan Pavlov, and was himself influential in the operant conditioning experiments of B.F. Skinner. Though there are few strict behaviorists today, the movement was quite dominant in the early 20th century.

The behaviorists would have loved the idea of a computer playing Jeopardy! as well as a human. They would have considered it a validation of their theory that the mind could be viewed as merely generating a series of predictable outputs when given a specific set of inputs. Playing Jeopardy! is qualitatively different from playing chess. The rules of chess are discrete and unambiguous, and the possibilities are ultimately finite. As Noam Chomsky argues, language possibilities are infinite. Chess may one day be solved, but Jeopardy! never will be. So Watson’s victory here is a significant milestone.

Much has been made of whether or not the contest was “fair.” Well, of course it wasn’t fair. How could that word possibly have any meaning in this context. There are things computers naturally do much better than humans, and vice versa. The question instead should have been in which direction would the unfairness be decisive. Some complained that the computer’s superior buzzer speed gave it the advantage, but buzzer speed is the whole point.

Watson has to do three things before buzzing in: 1) understand what question the clue is asking, 2) retrieve that information from its database, and 3) develop a sufficient confidence level for its top answer. In order to achieve a win, IBM had to build a machine that could do those things fast enough to beat the humans to the buzzer. Quick reflexes are an important part of the game to be sure, but if that were the whole story, computers would have dominated quiz shows decades ago.

To my way of thinking, it’s actually the comprehensive database of information that gives Watson the real edge. We may think of Ken and Brad as walking encyclopedias, but that status was hard earned. Think of the hours upon hours they must have spent studying classical composers, vice-presidential nicknames, and foods that start with the letter Q. Even a prepared human might temporarily forget the Best Picture Oscar winner for 1959 when the moment comes, but Watson never will. (It was Ben-Hur.)

In fact, given what I could see, Watson’s biggest challenge seemed to be understanding what the clue was asking. To avoid the complications introduced by Searle’s Chinese Room thought experiement, we’ll adopt a behaviorist, pragmatic definition of “understanding” and take it to mean that Watson is able to give the correct response to a clue, or at least a reasonable guess. (After all, you can understand a question and still get it wrong.) Watching the show on television, we are able to see Watson’s top three responses, and his confidence level for each. This gives us remarkable insight into the machine’s process, allowing us a deeper level of analysis.

A lot of my own work lately has been in training school-based data inquiry teams how to examine testing data to learn where students need extra help, and that work involves examining individual testing items. So naturally, when I see three responses to a prompt, I want to figure out what they mean. In this case, Watson was generating the choices rather than simply choosing among them, but that actually makes them more helpful in sifting through his method.

One problem I see a lot in schools is that students are often unable to correctly identify what kind of answer the question is asking for. In as much as Watson has what we would call a student learning problem, this is it. When a human is asked to come up with three responses to a clue, all of the responses would presumably be of the correct answer type. See if you can come up with three possible responses to this clue:

Category: Hedgehog-Pogde
Clue: Hedgehogs are covered with quills or spines, which are hollow hairs made stiff by this protein

Watson correctly answered Keratin with a confidence rating of 99%, but his other two answers were Porcupine (36%) and Fur (8%). I would have expected all three candidate answers to be proteins, especially since the words “this protein” ended the clue. In many cases, the three potential responses seemed to reflect three possible questions being asked rather than three possible answers to a correct question, for example:

Category: One Buck or Less
Clue: In 2002, Eminem signed this rapper to a 7-figure deal, obviously worth a lot more than his name implies

Ken was first to the buzzer on this one and Alex confirmed the correct response, both men pronouncing 50 Cent as “Fiddy Cent” to the delight of humans everywhere. Watson’s top three responses were 50 Cent (39%), Marshall Mathers (20%), and Dr. Dre (14%). This time, the words “this rapper” prompted Watson to consider three rappers, but not three potential rappers that could have been signed by Eminem in 2002. It was Dr. Dre who signed Eminem, and Marshall Mathers is Eminem’s real name. So again, Watson wasn’t considering three possible answers to a question; he was considering three possible questions. And alas, we will never know if Watson would have said “Fiddy.”

It seemed as though the more confident Watson was in his first guess, the more likely the second and third guesses would be way off base:

Category: Familiar Sayings
Clue: It’s a poor workman who blames these

Watson’s first answer Tools (84%) was correct, but his other answer candidates were Yogi Berra (10%) and Explorer (3%). However Watson is processing these clues, it isn’t the way humans do it. The confidence levels seemed to be a pretty good predictor of whether or not a response was correct, which is why we can forgive Watson his occassional lapses into the bizarre. Yeah, he put down Toronto when the category was US Cities, but it was a Final Jeopardy, where answers are forced, and his multiple question marks were an indicator that his confidence was low. Similarly cornered in a Daily Double, he prefaced his answer with “I’ll take a guess.” That time, he got it right. I’m just looking into how the program works, not making excuses for Watson. After all, it’s a poor workman who blames Yogi Berra.

But the fact that Watson interpreted so many clues accurately was impressive, especially since Jeopardy! clues sometimes contain so much wordplay that even the sharpest of humans need an extra moment to unpack what’s being asked, and understanding language is our thing. Watson can’t hear the the other players, which means he can’t eliminate their incorrect responses when he buzzes in second. It also means that he doesn’t learn the correct answer unless he gives it, which makes it difficult for him to catch on to category themes. He managed it pretty well, though. After stumbling blindly through the category “Also on Your Computer Keys,” Watson finally caught on for the last clue:

Category: Also on Your Computer Keys
Clue: Proverbially, it’s “where the heart is”

Watson’s answers were Home is where the heart is (20%), Delete Key (11%), and Elvis Presley quickly changed to Encryption (8%). The fact that Watson was considering “Delete Key” as an option means that he was starting to understand that all of the correct responses in the category were also names of keys on the keyboard.

Watson also is not emotionally affected by game play. After giving the embarrassingly wrong answer “Dorothy Parker” when the Daily Double clue was clearly asking for the title of a book, Watson just jumped right back in like nothing had happened. A human would likely have been thrown by that. And while Alex and the audience may have laughed at Watson’s precise wagers, that was a cultural expectation on their part. There’s no reason a wager needs to be rounded off to the nearest hundred, other than the limitations of human mental calculation under pressure. This wasn’t a Turing test. Watson was trying to beat the humans, not emulate them. And he did.

So where does that leave us? Computers that can understand natural language requests and retrieve information accurately could make for a very interesting decade to come. As speech recognition improves, we might start to see computers who can hold up their end of a conversation. Watson wasn’t hooked up to the Internet, but developing technologies could be. The day may come when I have a bluetooth headset hooked up to my smart phone and I can just ask it questions like the computer on Star Trek. As programs get smarter about interpreting language, it may be easier to make connections across ideas, creating a new kind of Web. One day, we may even say “Thank you, Autocorrect.”

It’s important to keep in mind, though, that these will be human achievements. Humans are amazing. Humans can organize into complex societies. Humans can form research teams and develop awesome technologies. Humans can program computers to understand natural language clues and access a comprehensive database of knowledge. Who won here? Humanity did.

Ken Jennings can do things beyond any computer’s ability. He can tie his shoes, ride a bicycle, develop a witty blog post comparing Proust translations, appreciate a sunset, write a trivia book, raise two children, and so on. At the end of the tournament, he walked behind Watson and waved his arms around to make it look like they were Watson’s arms. That still takes a human.

UPDATE: I’m told (by no less of an authority than Millionaire winner Ed Toutant) that Watson was given the correct answer at the end of every clue, after it was out of play. I had been going crazy wondering where “Delete Key” came from, and now it makes a lot more sense. Thanks, Ed!

I just watched the March 20th episode of Jeopardy! on the DVR. (I’m a little backed up.) I think I may have found an error in one of the clues.

The category was Battle Cries and the $2000 clue was as follows:

“Per Shakespeare, the British battle cry in this Oct. 25, 1415 battle was ‘God for Harry! England & Saint George!'”

The response given was “What is the Battle of Agincourt?” This was accepted as correct. However, I believe this question has no correct answer.

The Battle of Agincourt is depicted in Shakespeare’s Henry V, and the date in the clue is the correct date of the battle. But the quote comes from an earlier scene in the play, before Henry’s troops take Harfleur. The more famous St. Crispin’s Day speech is given before the Battle of Agincourt later in the play.

I imagine there is a lot of pressure being a writer for this show. If you’re interested in the topic, Ken Jennings just posted to his blog an interview he did with former writer Carlo Panno, which you can read here and here.

Last night, Ken Jennings of Jeopardy! fame had a book signing event at the Barnes & Noble in Chelsea. There were probably around eighty people there. Ken spoke for a bit and took some questions from the audience. Then he hosted an informal trivia game, giving away a free copy of his new book, Ken Jennings’s Trivia Almanac, to the two winners.

After the event, he was signing books. He signed my free copy of the Trivia Almanac “To Bill — The Chelsea Barnes & Noble Trivia Champ!”

I mentioned that I was a regular reader of his blog and poster to his message board. He didn’t recognize the name, understandably, but when I said “Shakespeare Teacher” he lit up. He told me that he had been to the blog and was in admiration of the 3-D Pic Tac Toe puzzles. He told me “I wouldn’t have thought that was possible.” I had a good comeback. “I wouldn’t have thought it was possible to win 74 consecutive games of Jeopardy!”

I asked if he would sign my copy of his first book, Braniac, and he signed it “To Bill — My Pic Tac Toe Guru!” which absolutely made my day.

In person, Ken turns out to be exactly the nice down-to-earth guy you’d expect him to be from his public persona. It was a great pleasure to meet him.

I grew up reading Games magazine, so I was delighted when Ken Jennings posted a good old “Pic Tac Toe” on his blog. But when he recently posted a plea for others to do likewise, I thought it would be a good time to bring Conundrum out of its summer hibernation.

In a “Pic Tac Toe” puzzle, there are nine pictures in a three-by-three grid, like Tic-Tac-Toe. In each row, column, and diagonal, there is a common theme that unites the three pictures. The challenge is to find the eight themes.

You can click on each image to see a larger version:

I don’t have a message board like Ken has, so just post whatever you come up with in the comments section.

Enjoy!

UPDATE: Correct themes provided by Annalisa (4) and DeLisa (2). Alternate theme suggested by DeLisa (1). See comments for all answers.

Ken Jennings says:

True confession: when celebrity deaths come in threes, as with Ingmar Bergman, Bill Walsh, and Tom Snyder yesterday, I like to imagine the deceased shooting each other in a three-way standoff, like at the end of Reservoir Dogs.

That’s funny. For me, when celebrity deaths come in threes, I like to make up jokes about the three of them arriving at the Pearly Gates at the same time.

Anybody want to take a crack at this one?

Since we’ve been talking about Ken Jennings and Jeopardy, I’ve been thinking about the Cheers episode “What is…Cliff Clavin?” in which Cliff Clavin, a postal carrier who lives with his mother, goes on Jeopardy. As a bar know-it-all, Cliff’s reputation and self-image are on the line. Guest star Alex Trebek reads off the categories:

CIVIL SERVANTS
STAMPS FROM AROUND THE WORLD
MOTHERS AND SONS
BEER
BAR TRIVIA
CELIBACY

And then Cliff knows this is his game. This is his dream board.

What’s your dream board on Jeopardy? You could answer this question in two different ways. You could either choose the six categories you’re strongest in, and that you would most like to see if you were an actual contenstant on Jeopardy. (I’m pretty good with questions about Shakespeare.) Or, as in the example above, you could create a mini-autobiography by choosing six categories that sum up your personality and interests. Or maybe that’s the same list.

What categories would be on your “dream board”?

First, read the rules of the game.

This week’s challenge is Jeopardy champion and Brainiac Ken Jennings.

I was able to link Ken Jennings to Sir Francis Bacon in six degrees or fewer, though that shouldn’t stop you from posting a longer response, or looking for a shorter one. Entries will be accepted until midnight on Thursday, May 17.

Good luck!

And congratulations to DeLisa for winning last week’s challenge by linking Jimmy Wales to Sir Francis Bacon in four degrees:

Jimmy Wales > Ayn Rand > Fyodor Dostoevsky > William Shakespeare > Sir Francis Bacon

Jimmy Wales is a follower of the Objectivist philosophy developed by Ayn Rand, who was heavily influenced by Fyodor Dostoevsky, who was heavily influenced by William Shakespeare, who is believed by some to be Sir Francis Bacon.

I am home to the new kids and Jenny’s proud song;
I help children build castles, but nothing too strong;
I’m the end of a question Ken Jennings got wrong;
And a way to tell actors just where they belong.

Who am I?

UPDATE: Riddle solved by DeLisa. See comments for answer.