Back in 2003 IBM executives gave Halim carte blanche to solve the data overload problem in general, so that diverse domains could use a common application. The result was a programming language called Spade that allows its recently released InfoSphere Streams software to access the common resources that IBM has developed to solve the problems associated with analyzing high-speed data streams on-the-fly.
"What we were aiming for here, was not to solve all of the problems for everybody right out of the box," said Halim. "But rather to provide the basic elements from which engineers can develop applications where the common parts involving data movement are all drawn from a common pool of algorithms while the specialized parts can be done by domain experts."
To analyze sunspot activity, the domain experts were the scientists who developed the special 3D radio telescope that collects data from space using a multiple tri-axial antennas. The magnitude of physical information encoded in these signals promise to enable more precise automatic recognition of dangerous solar patterns than ever before, but the amount of raw data generated by a single antenna is too large to store so must be analyzed on-the-fly.
"Rather than having a single big expensive antenna, the idea was to bring separate feeds from multiple smaller inexpensive antennas that are spread out all over the landscape so you would have the equivalent of a very very large instrument," said Halim. "But then you have to bring the data feeds from each of the smallessentially probe pointsinto one computing center and have the computation synthesize the images into a composite view of the sky electronically through software means."
The InfoSphere Streams software handles synchronization of phase among the incoming signals where variable distances to the antennas affect observations made at the exact same moment. The feeds are joined together in a composite view, interpolating data in any gaps left by antennas that are not working that day, then evaluating its relevance to the applications detection criteriain this case whether there's a dangerous sun-spot present or not.
Besides just the alarm function of warning of incoming interference from dangerous solar plumes, the application also provides a variety of visualization modes that filter out all but the most relevant data. The team will also develop models of the upper atmosphere and its interactions with inbound plasma clouds from the Sun which may cause magnetic storms here on Earth, and that it hopes can provide a space-weather forecast capability.