Fast serial links have popped up from every major technology constituency in the past 18 months to the point where they are a burden rather than a blessing for systems makers. It's high time for consolidation in interconnects. RapidIO, 3GIO, PCI-X 2.0, HyperTransport and Infiniband have joined high-speed Ethernet and Fibre Channel in a battle to become the interconnect of the future.
Each approach has technical nuances that make it interesting, and each carries technology agendas that need to be understood. All will probably survive in some form, but only a few will become mainstream.
The proliferation of I/O schemes reflects a real need for faster and more robust interconnect, but it's also partly an outgrowth of component vendor opportunism. This has caused some consternation among system designers, who essentially must make a risky bet on those they believe have the legs to keep up with their technology road map.
"There is quite a bit of overlap between [all the new interconnects] and that worries us quite a bit," said Louis-Franois Pau, general manager of Ericsson Utveckling, the Swedish group responsible for developing Ericsson's basestations, routers and switches. "The proliferation is a worry because we have to support the diversity of possible new configurations as well as legacy technologies. We have to continue to support some switches for 10, even 30 years."
RapidIO is tightly aligned with Motorola Inc.'s processor strategy. HyperTransport stems from Advanced Micro Devices Inc.'s processor road map. These two have so far bifurcated the emerging network processor terrain between them. (The Network Processor Forum could play a decisive role here, but more on that later.)
Intel Corp.'s investment in extending PCI is at the heart of 3GIO. Solving some fundamental issues in server design is the raison d'tre for Infiniband. PCI-X 2.0 is in some ways Compaq Computer Corp.'s EISA initiative for the 21st century.
Meanwhile, communications companies have deep investments in Ethernet, which is now emerging in 1- and 10-Gbit versions. And storage systems makers have similar commitments to Fibre Channel.
When you put all this together, it means the seemingly arcane area of interconnect technology may not only be pivotal in determining the relative success and failure of key processor architectures, it could also become a no-man's land in a broader conflict between computer and communications industry standards.
No one knows how it will all shake out. It may take a few years before observers can look back at the design decisions made in 2002 and determine how this polyglot mess was resolved who won, who lost and what lessons were learned.
How we got here
To be sure, some of the interfaces do not directly compete. HyperTransport, 3GIO, RapidIO and PCI-X 2.0 are generally intended for connecting components on a printed-circuit board or linking two boards in a chassis. Infiniband, Ethernet and Fibre Channel primarily aim at tying together separate systems. Few would dispute Ethernet is the king of long-haul links where latency is not an issue, but in the confined space of a data center, competition among the three is keen.
Ericsson's Pau is backing RapidIO due to its telecom-ready reliability and performance features. However, he also expects to support 3GIO because telecommunications gear must link to back-end servers, the domain of Infiniband and the PC.
RapidIO got its start in 1997 when Motorola's PowerPC architects realized that, in the system-on-chip era, a new processor bus for their next-generation CPUs needed characteristics of both a traditional CPU mezzanine bus and a system I/O bus. "They had to converge into a common interconnect," said Dan Bouvier, architecture manager for the PowerPC at Motorola, lead developer of RapidIO and chairman of the RapidIO technical working group.
Systems makers liked the notion of a standard interconnect used to link both the processors they bought and the ASICs they made, but they insisted it become an open standard, Bouvier added.
Now Motorola intends to bolt the interface onto its PowerPCs, digital signal processors and network processors like the C-Port C-10, and has many RapidIO-related announcements queued up for this year. Meanwhile, Motorola is cultivating the RapidIO ecosystem of chip makers and OEMs, which are innovating around one of its core technologies. Thus, system makers that opt for RapidIO are predisposed to choosing processors from Motorola or a handful of DSP vendors backing the spec.
A similar strategy has unfolded around AMD's Hammer processor family, which has adopted the HyperTransport bus developed by AMD and multiprocessor specialist Alpha Processor Inc. A key AMD Hammer designer, Jim Keller, went on to join network processor startup SiByte, and used HyperTransport on its chip. SiByte was later acquired by Broadcom Corp. and its chip was designed in to equipment from Cisco Systems Inc. The cachet attached to the Cisco association, sources said, has prompted other MIPS-based network processor makers such as PMC-Sierra and Sandcraft to adopt HyperTransport too, in hopes their devices will also wind up in Cisco sockets.
Failure to concur
In 1999, in an effort to come to an early truce, the HyperTransport and RapidIO camps discussed merging their efforts, but Motorola and AMD could not agree on a common spec. "At that time, I would have never thought we would be competing in the networking space and that Cisco was interested in [HyperTransport]," said Bouvier of Motorola.
AMD sees it somewhat differently, no doubt. The MIPS network processor vendors are helping to create broader support around technology AMD hopes to use to compete in the PC space with Intel.
For its part, Intel was spearheading the Infiniband movement in an attempt to solve growing heat, density and throughput problems in PC-based servers. The resulting interconnect opens significant new design options for servers that needed to push I/O out to a separate peripheral chassis.
Initially, some at Intel had grand ambitions that Infiniband could be a unifying interconnect for servers and communications systems. But those hopes have largely faded, given that adoption has been limited to servers and storage systems. Many telecom companies continue to look down their noses at Infiniband as a "PC technology," while extolling the virtues of Gigabit Ethernet, said sources at Ericsson and Cisco.
Today, Infiniband has garnered wide backing from venture capitalists and computer and storage OEMs but only after a long and public battle among server makers trying to converge on a single spec.
With so much activity surrounding the welter of new interconnects, engineers at Compaq began to fret that no one was paying attention to PCI, which was at the core of their systems. They helped kick-start work on PCI-X and PCI-X 2.0 in an effort to extend the life of the peripheral component interconnect in a user base of corporate IT managers for whom change is often anathema. Ironically, by extending PCI rather than leaping to a new technology, Compaq may be echoing a failed 1980s bid to establish the Extended ISA bus rather than migrate users to PCI.
Intel's influence
Somewhere along the way, Intel realized it needed to weigh in on the PCI road map in a broader way or risk losing influence to Motorola, AMD and Compaq. That was at least one factor in the birth of 3GIO, one of the last entrants in the interconnect race. As a testament to Intel's clout, the 0.9 version of the spec is being reviewed by 60 companies including competitors Sun Microsystems Inc. and AMD and is expected to be adopted by the PCI Special Interest Group as a kind of serial PCI 3.0 later this year.
Almost incidentally while all this was happening, developers pushed Ethernet to speeds of 10 Gbits/second, and Fibre Channel made further inroads into what has been until recently a hot storage systems market.
Now it's time for consolidation and making things work well together. Sun has already wisely chosen not to field a homegrown interconnect it had discussed with the IEEE as a possible standard. Anyone else with a similar gambit brewing in the labs should do the same. But the industry needs much more than an injunction on any more I/O initiatives. It needs developers to roll up their sleeves and hammer out some paths to interoperability.
Signs of that work are emerging. The RapidIO group and the 3GIO developers (known as the Arapaho Working Group) have had discussions aimed at identifying areas where the two specs could be harmonized. For starters, Pau of Ericsson suggests they could align low-level hardware issues by using the same voltage levels for active and standby modes. This is the time for both sides to show maximum flexibility, before specs are frozen.
Other camps, likewise, could benefit from sitting and talking. "We're hoping HyperTransport and 3GIO get together," said Steve MacKay, chief technologist at Sun's Computer Systems Group, responsible for mapping out Sun's I/O strategy. That statement alone from a company that will drive a flood of design wins should be enough to motivate the parties concerned.
Warring I/O camps should also consider the praise that iWarp, a spec defining fast data transfers over Internet Protocol networks, is getting from developers. The iWarp work is bringing Infiniband-like characteristics to Gigabit Ethernet. Now it's time for Infiniband developers to get cracking on full support for IP over Infiniband.
Forum play?
But perhaps the Network Processor Forum presents the best opportunity. Its work establishing standards for network processor interfaces could tip the balance for any interconnect in the space. The forum's much-watched LA-2 specification will be decided later this year.
A word of advice: Don't add yet another interconnect to the heap; find a way to build bridges among those out there.
Now on the short list for the LA-2 interface are RapidIO, HyperTransport (with extensions) and derivatives of the familiar SPI-4 and SPI-5 comms interfaces, which could be reasonable choices. Other possibilities are newcomers, including LiteLink from Cypress Semiconductor Corp. and HiBus from Intel. These should be dismissed if they cannot readily bridge the existing alternatives.
The same principles should be applied to a separate discussion at the forum around defining a standard interface for a merged control and data plane geared for low-end comms systems. Adopt the mantra of the home-networking crowd: "No new wires."
Systems makers must become enlightened consumers as they pick interfaces into which they will pour their development dollars. Then they have to become talented conductors, making the mess of strategic interfaces now pushing into the industry play well on tomorrow's systems.
"The guy who figures out how to glue all these things together will be able to create the best solution," said Michael Krause, who heads interconnect development at Hewlett-Packard. "I think there are a number of people who underestimate the complexity of this. This is an area that's ripe for innovation."