Pleiades uses more than 12,000 Intel quad-core Xeon chips linked on dual 20 Gbit/second Infiniband interconnects to deliver 487 TFlops of Linpack performance. NASA hopes to expand the installation within two years to hit performance of more than a petaflops.

The design marks a shift from Intel Itanium processors used in NASA's previous Columbia supercomputer. It also represents one of the first attempts to extend a supercomputer's interconnect as a high-performance local area network linking multiple systems.

Itanium and IBM Power6 cores provide more raw performance than Xeon cores, NASA found. However, at the system level the quad-core Xeon provided more bang for the buck and more flexibility in how much RAM per core NASA could deliver.

Pleiades extends its so-called HyperCube interconnect to 64 end points outside the system. The move lets researchers tap into the supercomputer at the ultra low latencies of Infiniband, making it easier to handle visualizations of climate change and other NASA applications.

The installation uses a whopping 21 miles of Infiniband cables, mainly internal to the Pleiades system which links about 100 cabinets. NASA researchers modified the open source Infiniband fabric software to enable the interconnect's use as a high performance LAN.

So far, the Infiniband net has been performing better than expected, said Robert Ciotti, chief architect of NASA's advanced systems facility who helped define Pleiades. Ciotti talked about the technology challenges building the supercomputer in an interview inside the Pleiades data center.