Dr, Watson is composed on many components each with each own testing and debugging methods. For example there's a part that break sentences to a data structure that represents meaning. You test this by creating a huge number of example sentences and their representations and testing how close watson got it.
Debugging : some parts of watson use machine learning. Machine learning themselfs are math algorithms so you debug them at that level. But than if you setup the parameters incorrectly and you see weird results , you try another combination of parameters ,probably guided by machine learning theory.
I guess similar methods scale to the top levels of watson .
If you're interested in more details, there are some guys from watson on quora , usually answr questions.
"You test this by creating a huge number of example sentences and their representations and testing how close watson got it."
Yep- the need to create a large number of example sentences to cover all of your application seems like a testing nightmare to me. It's like creating a large number of test packets to exercise your ethernet implementation. Takes years...
Dr, those kinds of things might be an industry effort. And you can always start by creating a small set of examles that are aimed at what engineers think are "corner cases" and let developers work with that, it might take a long time to conquer those. Also defining some of those test cases might be done by non experts with minimal training, because humans are good at language.
There are also some kinds of tricks to gather examples, like google started with books which have been translated as data for theil translation algorithm.And this is different than ASIC, in ASIC everything needs to work, here only some cases.
Rick, while I realize TiECon is not the venue to go much more in-depth, I wish the Qualcomm exec had disclosed some info on the planned development of I-o-T Silicon. It seems to me that he could have addressed areas where Qualcomm intends make a distinction that the I-o-T devices need, such as:
1. Power management including enegy harvesting 2. More sensor integration with CMOS processes 3. Low power multi-band / multi-protocol radios, etc.
The position where right now the IOT stands in the global market has been summarized aptly in the form of 3 sentences spoken by Steve Mollenkopf. Chip making for the IOT is slow at the moment and QUALCOMM CEO understands it. However the direction where his company will go after IOT starts playing in the electronics supply chain will be governed by what business policies his company has adopted in the advent of IOT.