Tuesday, February 15, 2011

Where Watson falls down - day 2

This is my second post as I follow the Jeopardy challenge between top human competitors and Watson, the IBM supercomputer. My first post discussed what happened on day 1.

Watson did fabulously compared to its human opponents today, building up a huge lead at the half-way point in the two-game series. But it made an interesting blunder in Final Jeopardy.

The category was "US cities" and the answer was the city "...  whose largest airport is named after a World War II hero, and whose second largest airport is named after a WW II battle".

Watson goofed badly by giving the answer of "Toronto", although with question marks indicating uncertainty. Toronto has Pearson airport and Toronto Island Airport, often just called "Island Airport"

I thought this question was extremely easy for final Jeopardy. The Battle if Midway is so well known, so that quickly led me to the answer of Chicago. Then I remembered seeing an exhibit about Edward O'Hare while flying through O'Hare airport.

So how did Watson go wrong? It is interesting to try to make an educated guess:

It turns out there are lots of references to various people called Pearson who are war heroes, and this page, for example, calls Lester Pearson (after whom Toronto's largest airport is named) a hero.

However what I think is most important is that there are large numbers of "Island" battles (e.g. Wake Island, Aleutian Islands, etc.) recorded in World War II. It seems likely that the sheer number of such battles may have overwhelmed Watson's probabilistic analysis, even apparently overriding the category "US Cities" since Toronto is certainly not in the US!

This shows the shallow state of understanding that Watson has in certain areas. We humans know that no airport would never be actually given the generic name "Island" after any of the island battles.

No comments:

Post a Comment