WASHINGTON -- The U.S. Department of Energy this month will kick off a strategic effort aimed at realizing by the year 2001 a 30-teraflops supercomputer--fully 10 times the performance of the highest-powered existing systems.

Called "Pathforward," the DOE effort--in partnership with Lawrence Livermore, Sandia and Los Alamos National Laboratories--will commence with the announcement of a contract for a high-speed system-level interconnect that can tie together clusters of supercomputers to create machines far faster than currently possible, DOE officials said.

"We're talking about bandwidths in the 2- to 4-Gbyte-per-second range each way, end-to-end latencies in the microsecond range and cross-section bandwidths up in the terabytes-per-second range," said Richard Watson, chief scientist at Lawrence Livermore National Laboratory (Livermore, Calif.) and a Pathforward program manager.

A group of vendors led by Hewlett-Packard Co. is expected to snare the contract, to the tune of approximately $10 million, according to industry sources. HP's Richardson, Texas-based Convex Division will lead the effort to create interconnect technology it is calling the "Gigabyte System Network." Silicon Graphics Inc. and E Systems Inc. are also reportedly contributing to the effort. Officials at both DOE and HP declined to comment.

Such interconnects are vital to achieving the goal, since current-generation supercomputers top out between 1 and 3 teraflops. Achieving 30 teraflops will require hooking multiple systems together.

"In 2001 we're aiming for a 30-teraflops system," said Watson. "Around the 2004 or 2005 time frame, we want to achieve 100-teraflops systems."

Watson pointed to symmetric multiprocessing systems as a model. "Today you've got SMP boxes that may have anywhere from 16 to 128 or more processors inside them," he said. "These machines all have their own internal interconnects. We need this [external, system-level] interconnect to hook up many of these SMP boxes to each other. Just to give you a rough idea, let's assume you had 50 to 100 processors in an SMP box. You would need 100 or more of those boxes to get you up in the 30- to-100-terabyte range, depending on the speed of the processors."

Already working
Pathforward will buttress work currently in process at a sister DOE program called the Accelerated Strategic Computing Initiative (ASCI). Under that project, the first single-teraflops-class supercomputers are beginning to come together. For example, a 1-teraflops Intel supercomputer known as the ASCI "Red" machine has already been delivered to Sandia (Albuquerque, N.M.). The system boasts some 9,000 Pentium Pro-class CPUs.

In 1998 or early 1999, two 3-teraflops machines--code named ASCI "Blue"--are due to come online. One of those will go to Lawrence Livermore. It will be an IBM system built using the company's Power architecture, a multichip implementation of the PowerPC CPU. The IBM machine will have 512 nodes, with eight CPUs per node.

The second ASCI Blue machine, which will be delivered to Los Alamos, will be manufactured by Silicon Graphics/Cray. Architecturally, it's expected to be a derivative of SGI's Power Challenge design.

While ASCI focuses on heavy iron, Pathforward is centered around the missing pieces of the teraflops puzzle. "The basic building blocks are there," said Watson. "What isn't there are some of the scaling and integration technologies to put all those pieces together to create 30- and 100-teraflops-class systems. That's what Pathforward is about, to speed up the creation of those technologies and to be sure there are a number of vendors out there who have product and technology capabilities, and the ability to build these large systems."

But this may not be easy, according to one industry expert intimately familiar with Pathforward, who requested anonymity. "The Washington view of this is, 'we're going to push the field ahead in an enormous way,'" the expert said. "The reality is, it's going to be a lot harder than it looks."

Perhaps the biggest challenge, according to the expert, is to take the teraflops speeds that have been recently demonstrated running "embarrassingly parallel" software benchmarks such as the industry-standard Linpack library, and translate them into equivalent performance in real-world physics-applications code.

The issue of software is highly relevant, because Pathforward and ASCI aren't simply funding exercises to boost the U.S. supercomputer industry. Rather, they have a very specific goal: to corral enough computing power to "verify the nuclear stockpile"--that is, to run software that can simulate the behavior of nuclear weapons involved in complex accident scenarios such as fires and earthquakes.

The requirement, according to DOE documents, stems from the current "moratorium on nuclear testing, the termination of new-weapons development and the closure of many weapons-production facilities."

In the immediate future, forging Pathforward's system-level interconnects will be no small challenge, either. Indeed, today's widely used data-transfer rates come in at the 100-Mbyte/s level achieved by the current-generation Hippi-800 link. The coming state of the art is defined by an emerging standard called Hippi-6400, which is eight times faster than Hippi-800.

Hippi-6400 is being developed by the supercomputing industry. But the gigabyte link DOE plans to procure under Pathforward is well beyond even this technology. DOE officials admit as much.

"In terms of actually hooking up symmetric multiprocessors into these large 30- and 100-teraflops ASCI systems, we need faster interconnect technology than Hippi-6400," Watson said.

To reach the 2- to-4-Gbyte throughput rates required, "They've got to be able to speed up the whole environment," he said, "from the processing out through their memory systems out through the interface adapter boxes out into the signaling links, through all the switching components in the network. They're [also] going to have to speed up the actual signaling rate on the links, and they're going to have to speed up the interface to their memory systems."

Taking up the challenge, HP thinks it has a potential solution. The company's Gigabyte System Network (GSN) interconnect technology could tie together as many as 64 simultaneous Hippi-6400 links to support a 30-teraflops supercomputer.

Since Hippi-6400 links haven't yet been built, HP will have its work cut out for it. But company engineers have formulated a high-performance, intelligent Hippi-6400 adapter. A key component will be the hardware-message engine, an HP-designed ASIC incorporating a processing core that can support the Scheduled Transfer Protocol over a Hippi-6400 physical layer. HP plans to use the SuMAC chip currently in development by Silicon Graphics to implement the Hippi-6400 physical layer. GSN will also include local RAM and a proprietary host interface.

Commercial questions
Moving ahead, the Pathforward project plans to tackle two other technology areas--optical-mass storage and parallel-applications software--that must advance if 30-teraflops supercomputers are to be realized.

According to industry sources, DOE initially intended both areas to be part of the first phase of Pathforward. However, during a January 1997 meeting hosted by DOE officials, industry representatives loudly expressed concerns that the goals were well beyond the state of the art. Indeed, sources who attended the meeting reported that the question of whether the interconnect portion of Pathforward was technically feasible was also raised.

Industry concerns were compounded, the attendees reported, by the fact that Pathforward calls for the DOE to procure commercial, off-the-shelf technologies rather than the gold-plated ones developed in an earlier era under funding from the Defense Advanced Research Projects Agency (Darpa).

Indeed, ASCI program manager and deputy assistant secretary of defense Gil Weigand reportedly addressed the government's attitudinal sea change by reminding those attending that "This isn't Darpa in the 1980s, it's DOE in the next millennium." Weigand was not available for comment.

Despite lingering industry concerns, DOE officials appear intent on sticking to their mandate. "We think of Pathforward as a development and engineering project--it's not research," said Livermore's Watson. "Our whole strategy is to build out of industry commodity building blocks. When industry can speed up interconnect technology, then eventually that technology will become part of their standard product line, down the road."

Free Subscription to EE Times First Name Last Name Company Name Title Email address   Click here for your Free Subscription to EETimes Europe