News & Analysis
Comment
KingofthePaupers
Jct: "Doing this ends inflation of money?" is one question I'm the only person ...
goafrit
You are correct, probably. Sometimes we miss that simple answer. Thank you.
IBM's Watson computer beats humans at Jeopardy
R Colin Johnson
2/3/2011 7:26 PM EST
PORTLAND, Ore.—Later this month, Jeopardy, billed as American's favorite game show, will air a special edition where the all-time greatest human champions, Ken Jennings and Brad Rutter, will compete against IBM's Watson cluster computer.
The grand prize of $1 million goes to the winner, with second place earning of $300,000 and third place of $200,000. Instead of shipping Watson to Hollywood, this special edition of Jeopardy was taped last month at IBM's Hawthorne Lab (New York), where a mock-up of the original Jeopardy studio was constructed.
During trial sessions, Watson beat the humans—just barely. But the real contest will be aired on Feb. 14, 15 and 16.
IBM has set several "grand challenges" for itself in the past, but the Jeopardy challenge is not just a PR stunt—a criticism leveled at its earlier chess challenge that resulted in the Deep Blue supercomputer, which beat grand champion Gary Kasparov, but was built from specialized hardware that could not be readily adapted to other applications.
Watson, on the other hand, was built from the ground up to be adaptable to the kind of business applications for which IBM is famous. Unlike chess-playing Deep Blue, Jeopardy-playing Watson runs on a cluster of 90 commercially available Power 750 servers with 2,880 Power 7 cores and 500Gbytes per second throughput.
To be fair, Deep Blue, which beat Kasparov in 1997, did lead to IBM's 1999 grand challenge to build the world's first petaflop computer, IBM’s Blue Gene supercomputer, which is used today to solve problems requiring massive computation over large data—from weather forecasting to protein folding. However, the DeepQA architecture that has made Watson rival humans in precision, confidence, and speed, was specifically crafted to adapt to business solutions.
"Watson is the result of a long commitment to human language technology, large data analytics, and supercomputing, all coming together," said John Kelly, director of IBM Research. "We knew that if we could get[Watson] to work on a commercial system, then the roll-out into other fields of Deep Q&A would be greatly accelerated."
IBM’s DeepQA architecture for Watson rests on a foundation provided by its 24 man-year effort to create a Practical Intelligent Question Answering Technology (PIQUANT) which was retread for the Jeopardy Challenge, and which led to the Open Advancement of Question Answering (OAQA) systems initiative to make question-answering algorithms reusable across applications.
“IBM created the OAQA as a way to promote an open approach to building question answering technology that enables different teams to share components, perform end-to-end evaluations, and advance the overall state-of-the-art,” said Eric Brown, a research staff member at the IBM T.J. Watson Research Center.
Next: But is it AI?
The grand prize of $1 million goes to the winner, with second place earning of $300,000 and third place of $200,000. Instead of shipping Watson to Hollywood, this special edition of Jeopardy was taped last month at IBM's Hawthorne Lab (New York), where a mock-up of the original Jeopardy studio was constructed.
During trial sessions, Watson beat the humans—just barely. But the real contest will be aired on Feb. 14, 15 and 16.
IBM has set several "grand challenges" for itself in the past, but the Jeopardy challenge is not just a PR stunt—a criticism leveled at its earlier chess challenge that resulted in the Deep Blue supercomputer, which beat grand champion Gary Kasparov, but was built from specialized hardware that could not be readily adapted to other applications.
Watson, on the other hand, was built from the ground up to be adaptable to the kind of business applications for which IBM is famous. Unlike chess-playing Deep Blue, Jeopardy-playing Watson runs on a cluster of 90 commercially available Power 750 servers with 2,880 Power 7 cores and 500Gbytes per second throughput.
To be fair, Deep Blue, which beat Kasparov in 1997, did lead to IBM's 1999 grand challenge to build the world's first petaflop computer, IBM’s Blue Gene supercomputer, which is used today to solve problems requiring massive computation over large data—from weather forecasting to protein folding. However, the DeepQA architecture that has made Watson rival humans in precision, confidence, and speed, was specifically crafted to adapt to business solutions.
"Watson is the result of a long commitment to human language technology, large data analytics, and supercomputing, all coming together," said John Kelly, director of IBM Research. "We knew that if we could get[Watson] to work on a commercial system, then the roll-out into other fields of Deep Q&A would be greatly accelerated."
Jeopardy host, Alex Trebek (standing), leads Watson discussion (from left): Harry Friedman, executive producer, Jeopardy; Brad Rutter, Jeopardy champion (earning $3,255,102); Ken Jennings, Jeopardy champion (earning $2.5 million); David Ferrucci, principal investigator of Watson DeepQA technology, IBM Research; and John E. Kelly III, IBM senior vice president and director of IBM Research.
IBM’s DeepQA architecture for Watson rests on a foundation provided by its 24 man-year effort to create a Practical Intelligent Question Answering Technology (PIQUANT) which was retread for the Jeopardy Challenge, and which led to the Open Advancement of Question Answering (OAQA) systems initiative to make question-answering algorithms reusable across applications.
“IBM created the OAQA as a way to promote an open approach to building question answering technology that enables different teams to share components, perform end-to-end evaluations, and advance the overall state-of-the-art,” said Eric Brown, a research staff member at the IBM T.J. Watson Research Center.
Next: But is it AI?
|
|
|
|
|
Navigate to related information
R_Colin_Johnson
2/3/2011 9:29 PM EST
By running on a configuration of off-the-shelf cluster computers, IBM will be able to quickly target business applications to solve using the same DeepQA architecture, just with a different knowledge base plugged in. Here are a few applications IBM told me they may target in the future with Watson-like cluster computers:
Healthcare: For medical applications Watson will be able to act as an expert consultant for doctors, simultaneously comparing your symptoms to those of millions of patients worldwide, along with the diagnoses that cured them and all the latest information from medical journals, supplying a ranked list of possible diagnoses.
Retail: Watson can integrate queries from retailers that simultaneously consider past buying patterns, inventory, order management and supply chain issues to supply customer relationship management (CRM) answers that are highly targeted, perhaps offer personalized "sales" while you are still standing at the register.
Financial Services: Watson's realtime analytics will enable financial institutions to what-if scenarios that include market data, current events, the opinion of analysts and a thousand other unstructured information sources that are difficult to encapsulate in to conventional algorithms--plus its machine learning capabilities will enable Watson to hone its predictive abilities even finer over time.
Government: Corporations today must deal with a dizzying array of laws and regulations that could be simultaneously considered by Watson, instead of occupying weeks of a human experts time, allowing businesses to optimize their profits in realtime while still meeting all their regulatory obligations.
What Watson Can't Do: Because the resources of the entire cluster computer are dedicated to answering each question--one at a time--Watson will not be directly amenable to traditional Internet searches, which must serve thousands of amateur users making ad-hoc queries simultaneously.
Sign in to Reply
prabhakar_deosthali
2/4/2011 12:45 AM EST
This is a great information. One question to be answered at a time is fine. I am just curious to know what is the expected response time for such a response in a single user mode. If it is a matter of seconds or even a minute, still it worth it. Because such high performance machine will be used only to get answers for some critical questions which humans are unable to find in a massive data archive.
Sign in to Reply
R_Colin_Johnson
2/4/2011 12:37 PM EST
The Jeopardy applications requires about three seconds of cluster computer time, but my guess is that financial applications might require slightly more time and retail apps may require less. Also I am sure IBM is working on way to queue up these queries for more efficiency.
Sign in to Reply
selinz
2/4/2011 12:55 PM EST
This is definitely a completely different challenge than chess. This is not "computable" so much as reliant on massive amounts of data. It will be interesting to see how it responds to some of the more abstract clues...
Sign in to Reply
yalanand
2/5/2011 6:40 AM EST
Wow, cant imagine how much effort has gone into to build this massive system. Kudos to IBM for building interesting and challenging systems.
@R_Colin_Johnson By the can we see the game online ?
Sign in to Reply
gatorsrule
2/5/2011 12:16 PM EST
From a science standpoint, I'm missing the "wow" factor here. They've taken a very well known data set (the list of all previous questions) and applied search algorithms to extrapolate that data to the same setting. "Wow" in a sense of they really spent a lot of money on doing this, but certainly not "wow" in terms of any innovation or technical breakthroughs.
Sign in to Reply
ReneCardenas
2/9/2011 1:58 PM EST
gatorsrule, ditto with the unimpressive premise to see who can retrive data faster. I would like to gauge how that system implemetation captures and responds to abstract concepts.
Sign in to Reply
Ratzo
2/6/2011 11:03 AM EST
Won't it be funny if Watson gets every answer correct, but hasn't been programmed to answer in the form of a question?
Sign in to Reply
goafrit
2/6/2011 1:34 PM EST
This is awesome. I will like to challenge this beast on how to save the world from malnutrition and famine. Maybe, it can churn out the secret formula that we have longed for ages in Africa.
I cannot wait for watch Jeoparday for this classico
Sign in to Reply
David Ashton
2/6/2011 5:11 PM EST
Goafrit, the formula to sort out Africa's problems is not a secret. Get rid of dictators like Robert Mugabe and you'll be more than half way there.
It's not rocket science......
Sign in to Reply
goafrit
2/18/2011 2:37 PM EST
You are correct, probably. Sometimes we miss that simple answer. Thank you.
Sign in to Reply
KB3001
2/6/2011 1:42 PM EST
Good publicity for IBM. The kind of research behind it is however being done in many research labs world-wide.
Sign in to Reply
Ratzo
2/6/2011 2:38 PM EST
I can't wait to see if the computer addresses the game's host by name.
"I'll take 'Sixties Sitcoms' for $100, Alex."
Sign in to Reply
CamilleK
2/6/2011 3:31 PM EST
I would love to see a '2001 Space Odyssey' category with 'who is HAL' being the answer followed by some joke about shutting down and expressing emotions. Seriously now, IBM deserves accolades for thinking outside the box and for pushing the frontiers of computing, semantics, human thought and all the underpinnings of meaning, understanding and knowledge. Jacques Lacan once said (in a different context) "language points to a lack". This is a case where a
'lack (of sentient being) will point to language' to prevail. We need smart shows on TV and I am certainly planning to watch the series.
Sign in to Reply
p_g
2/6/2011 6:04 PM EST
Its intersting to see, human brain putting all its inteligence to develop something more inteligent than brain himself.
I guess the only place computer beat humans and data churning.
Sign in to Reply
Neo1
2/6/2011 9:45 PM EST
p_g I don't think they are trying to make one above human brain, if at all they can. The challenge is to make it do any task which humans do with ease but these tasks have been shown to have enormous computational requirement which makes it daunting.
This is a great breakthrough and would sure go a long way in understanding the analytics of human cognition.
Will the game be telecast live? Hope someone puts it on youtube for all users.
Sign in to Reply
Duane Benson
2/7/2011 11:51 AM EST
I also fail to see the "wow factor" in this. But, that's part of the beauty of it in my mind. A very complex problem broken down to a massive amount of very simple problems. Isn't that what any computer is? Simply a massive amount of very basic logic gates.
Quite a number of years ago, I was told that the color television was the most complex device ever created. I'm sure there is some hyperbole in that statement, but it was an extremely complex design. Taking that functionality and replicating it on a generic computing platform, to a certain extent, has the same lack of wow as using a massive computer to win at Jeopardy, but as I see it, the "wow factor" is in the ability to simplify and use more or less generic hardware to solve those problems.
Regardless of how it is done, at some point, such a machine will be indistinguishable from a human. When that is the case, does really it matter whether it's brute force or a system designed to truly emulate the human brain?
Sign in to Reply
iniewski
2/7/2011 12:53 PM EST
I agree with Duane and others: where is the "wow" factor here? Show me something that requires intelligence not some massive brute force computations!.... dr Kris
Sign in to Reply
Ratzo
2/7/2011 1:12 PM EST
Would you be WOWed if the computer engaged in witty repartee with Alex in the "getting to know you" part of the show?
Alex: So, Watson, I understand you come from a non-traditional family.
Watson: That's right, Alex. My two dads are David Ferrucci and John E. Kelly III. They may be nerds, but they are the Best Dads Ever! In fact, when I was driving them to the show today...
Alex: Just a minute there, Watson! YOU drove THEM to the show?
Watson: Yes, Alex, but it wasn't easy. You might say it was a hard drive! Ha! Ha!
Sign in to Reply
Ratzo
2/7/2011 1:24 PM EST
... Worst case, I guess, would be if the witty repartee went something like this...
Alex: Watson, do you come from a nontraditional family?
Watson: What is the first question that Alex asked Watson today?
Alex: What?
Watson: What is the unit of power, as defined by the Scottish engineer James Watt?
Alex: Ay Jesus.
Watson: What were the last words of Charles V, King of France?
Sign in to Reply
Sheetal.Pandey
2/7/2011 4:33 PM EST
well humans have made computers and now computers will compete against humans. Great idea! It sounds great to hear as long as its a game or something for fun. IBM deserves appreciation for keeping on doing something to prove they are masters in this field.
Sign in to Reply
Warren
2/8/2011 8:28 PM EST
"Well humans made computers" and, imagining advances in gene therapy and other genetic modifications and methodologies, one can imagine computers making humans [better]. Or worse.
Sign in to Reply
R_Colin_Johnson
2/9/2011 12:40 PM EST
Just remember that these AIs are one-trick-ponies and are only as good as their specialized algorithms, which so far, are highly application specific.
Sign in to Reply
ReneCardenas
2/9/2011 1:52 PM EST
R_Colin, earlier attempts to crack a close information system using AI failed to consider the complexity of human communication (with cues and noise source in the form of modulation, accent, etc).
Were the logistics described to you, I did not get the details form this piece of that competition.
Is the question prefetched in the system by human? Speech or vision recognition?
I wonder if a system like this would be able to capture effectively all the relevant details for an effective query.
For example, in the medical case, only if the device is able to capture the essence of all relevant data, can I see that the diagnosis to be effective.
My thought is the GIGO principle can limit the effectiveness of this effort in real commercial applications.
Sign in to Reply
R_Colin_Johnson
2/9/2011 3:42 PM EST
Watson does not perform speech recognition, rather the clues are delivered in text form.
Sign in to Reply
DrQuine
2/9/2011 8:13 AM EST
The extraordinary task that Watson is taking on is contextual analysis and problem solving in essentially random domains. Anyone can design a machine to grind flour (2 rocks work) but the raw ingredients must be presented just right and the machine doesn't do anything else useful. In this case, Watson is taking on a "knowledge" problem (not just a brute force mathematical calculation or the very constrained game of chess) that challenges the most skilled humans in the world. Kudos!
Sign in to Reply
DrQuine
2/9/2011 12:49 PM EST
I'd say the nature of the Jeopardy application positions it for expansion to a wide range of applications (not just a one-trick-pony). It is mining and combining information across a wide range of human knowledge and also managing linguistic features. That is hard work. Even educated people with access to reference materials can't necessarily answer some of these questions.
Sign in to Reply
R_Colin_Johnson
2/9/2011 3:47 PM EST
IBM has all sorts of plans to expand Watson's reach, using the lessons they learned in founding the Open Advancement of Question Answering (OAQA) systems initiative. By following OAQA principles, it should be possible to house nearly any knowledge domain in a searchable format, enabling natural language queries for all sorts of applications. The only major limitation is the computing horsepower required--several seconds of supercomputer time per query--which will restrict users to those that can afford it.
Sign in to Reply
Ratzo
2/9/2011 1:29 PM EST
Watson a one-trick-pony? Nah, I think he'd be very competitive on "Wheel of Fortune". Admittedly, he wouldn't last long on "Survivor", unless he's programmed to betray his allies.
Sign in to Reply
ReneCardenas
2/9/2011 5:21 PM EST
Very clever Ratzo, very clever... I cracked a loud laughter.
Thanks for the humor.
Sign in to Reply
RWasserman
2/9/2011 6:09 PM EST
You might be onto something, Ratso.
Betraying your allies is the most human behavior.
Sign in to Reply
R0ckstar
2/10/2011 3:28 PM EST
I must be too easily impressed, because this is really impressive. A milestone even. From here, it's just a few short years before Watson's kids are answering their own questions recursively from being loaded up with all manner of highly specialized, obscure, & complex research material in medical, materials science and physics, and spitting out unified field theory and transparent aluminum formulas. WOW.
Sign in to Reply
KingofthePaupers
2/19/2011 4:41 PM EST
Jct: "Doing this ends inflation of money?" is one question I'm the only person claims to have solved and I'd bet Watson cannot.
Sign in to Reply