While Intel has been making headway in high-end computing with its Xeon and Itanium processor lines, it has lacked a platform to turn such CPUs into multiprocessing systems. Intel's answer is the 870, which is expected to roll out early next year for a new version of the Xeon 32-bit processor and for the next-generation IA-64 processor, or McKinley.

Intel, which stumbled in its last attempt to crack the server market, said it will license to OEMs what appears to be a new electrical and protocol interface, so that vendors can build systems that scale up to 256 processors.

"It's another step by Intel to get into computing markets outside of the pure PC space," said Dean McCarron, principal analyst with Mercury Research (Scottsdale, Ariz.). "Intel is doing the kind of engineering that was once done only in server and high-end computing. They're significantly lowering the barrier to enterprise computing."

In the wake of OEMs' reluctance to support Intel-led initiatives like the switch to Rambus DRAM memory, Intel has loosened its grip over server design engineering. The slow takeoff of Rambus contributed to the downfall of Intel's last server chip set, according to sources. This time, the company is giving OEMs wide latitude to make their own design decisions and differentiate their products.

Intel's willingness to license its key I/O switching technology is one example of this newfound openness. "What we're doing is providing the ability to design building blocks so you can expand by bringing in different topologies," Fay Briggs, director of chip set architecture for Intel's enterprise products group, told EE Times after his presentation here.

Intel's switch chip, called the Scalability Port Switch, has six ports that link to four-way processing nodes, neighboring switches and an I/O hub that connects to system buses. Third-party chip designers that license the switching I/O technology are free to design their own switches with more ports. "OEMs can design their own switches with six to 10 ports using a hierarchy of switches," Briggs said.

Intel has a long history of working with key OEMs and third-party chip set vendors in multiprocessor platforms. More recently, third-party chip set vendors have become a lifeline for the company's server efforts.

Intel last year ran into problems with a server chip set design and decided to cancel the project. Companies like IBM and Compaq were designing their own server chip sets for Xeon and Itanium, but for those OEMs that lacked chip set expertise, Intel had to anoint ServerWorks as its de facto chip set provider for multiprocessor servers.

"ServerWorks became part of Intel's road map but it was clear that was not a long-term arrangement," said analyst McCarron. "With Itanium, there is strong evidence that Intel is coming back into the competitive fray for multiprocessing chip sets."

Making up for lost ground

After licking its wounds, Intel is trying to make up for lost ground with its homegrown 870. Even so, the company kept the design team relatively small, and relied heavily on feedback from OEM partners for architectural decisions like the memory subsystem and switching technology, according to several sources.

Intel described the switch port as a packet-based, point-to-point coherent interconnect that transfers data at 6.4 Gbytes/second in both directions. There are six of these ports in Intel's switch device, giving each switch an aggregate bandwidth of 76.8 Gbytes/s.

The switch's I/O is divided into physical, link and protocol layers. The link layer determines flow control and transmissions and the physical layer controls the electrical signals. The two are closely related and grouped into one megacell. The protocol is for high-level communications such as coherency and ordering, and is divided into distributed and centralized functions, Briggs said.

Intel's implementation of the switch includes 1-Mbyte snoop filtering, to let individual processors keep tabs on memory data touched by other processors. Based on the commonly used MESI protocol, the snoop filter operates at 400 MHz and can process 266 million lookup-update transactions per second. The switch also contains a crossbar and bypass buses for traffic that needs coherency, Intel said.

Further upstream is the scalable node controller, which is the equivalent of a north bridge. One of Intel's key design decisions here was to use two channels of double-data-rate DRAMs instead of Rambus DRAMs, once favored by Intel as the memory of choice. Briggs said the decision to go with DDR was based on customer request.

Other observers said it's easier to build large banks of main memory with DDR than with RDRAM, because of the limitations in how many devices one RDRAM channel can support. OEMs also balked at Intel's attempt to get SDRAM to talk to a Rambus interface through a memory translator hub, an error-prone device that failed in the PC market.

"DDR offers a much larger memory configuration than Rambus and if you look at server systems, memory capacity is very important. Usually, the bigger the better," said one source familiar with Intel's plans.

With the 870, each DDR channel supports four dual-in-line memory module slots. When 1-Gbit DRAMs hit the market, each controller will be able to support 128 Gbytes of main memory. The node controller links to separate switches and other node controllers, forming a matrix of interconnected processing nodes. Each controller also has its own "firmware hub," a flash-memory-based device that lets each node independently perform self-tests. Each node controller can support four CPUs, but Intel did not disclose further details on the processor-to-controller interface.

Coherency resolved

Further downstream, the third major component of the 870 is the I/O hub, which talks to two separate switches and the system buses, including PCI-X, Infiniband and legacy I/O. Largely a repository of cache and buffer memory, the device is geared to quickly resolve coherency by means of prefetching to make up for long waiting periods when accessing main memory.

Intel has two ways of doing prefetch. When the read length isn't known, such as in the case of PCI, speculative prefetch is employed, and can be throttled on the fly depending on the number of active streams. In the case of PCI-X or Infiniband, where the read length is known, nonspeculative prefetch is used, and subsequent reads are stored in buffers.

Prefetching was once the exclusive domain of CPUs, but designers are starting to add the capability to supporting chip sets.

"We're getting to such high processor clock rates that the penalty for a stall of any type gets amplified," Mercury Research's McCarron said. "With a gigahertz processor you're missing a billion cycles a second during a stall, so the more you can do to prevent the processor from waiting the higher performance you're going to get."

Features to address reliability, availability and serviceability — known in the industry as RAS — are also being built into the chip set. Processor, memory and I/O nodes can be hot-swapped while the operating system is running, assuming support for this feature is built into the OS. The 870 also supports domain features, which allows a system to be partitioned into smaller systems so it can run separate applications or OSes.