High-speed serial fabrics are becoming commonplace in both proprietary and commercial off-the-shelf systems. These systems use a variety of topologies including "mesh" and "star" configurations. Both CompactPCI and AdvancedTCA offer open architecture system specification that incorporate high speed serial interconnects.

CompactPCI offers a serial mesh standard through PICMG 2.20 with a 1X channel density. AdvancedTCA incorporates a mesh fabric with a 4X channel density. With the 4X channel density of AdvancedTCA, aggregate data rates of 10 Gbit/second per link are possible with conventional copper connectors and standard backplane materials.

Star topologies are important in applications where the distributed traffic flow of the mesh is not beneficial, or the switch architecture lends itself to a star topology. Here, channel bandwidth is everything. Bandwidth can be increased by widening channels, to the limits of the number of connections at the switch.

Mesh topologies offer significant bandwidth for applications that can take advantage of the distributed fabric. Expanding capacity in mesh fabrics with wider channels reaches limitation due to the number of interconnects. Higher link speeds can extend the life of mesh fabrics. Increasing link speed and maintaining interoperability with existing systems can be key to extending copper backplane systems for a decade or more.

A new generation of devices that combine transceiver technology, built-in serial/deserialization (serdes), and programmable logic facilitates the high-speed serial link between components connected by a mesh fabric, both in the current generation of fabrics that deliver up to 3.125 Gbit/second, and in next-generation fabrics that deliver greater data rates.

These devices can increase data rates in the current and coming generation of mesh fabrics. Advances in backplanes, connectors, and next-generation transceivers can further increase mesh fabric bandwidth.

The leading vendors offer FPGAs with integrated transceiver circuitry. These devices combine the functionality to handle high-speed serial data with a customizable pool of logic, enabling designers to combine the link to the backplane with traffic management and other functions into a single device. Specific characteristics of these devices ease the development of serial fabric-based systems, such as relatively low power consumption — as little as 450 mW per 4-channel transceiver block at 3.125 Gbit/second — and the ability to drive up to 40 inches of FR4 board material and two connectors, which facilitates the development of large, multi-card backplanes. In addition, they feature built-in clock-data recovery structures, which are cable of recovering encoded clock signals at multiple rates, ranging from 1/4th to 1/20th the data rate.

Other built-in hardware elements provide high levels of data throughput by implementing critical functions in dedicated circuitry without using programmable logic resources. For example, these devices offer built-in serdes functions. These serdes functions can support multiple deserialization factors, for support of data streams that use 8B/10B encoding as well as other types of encoding/decoding. In addition, the pattern (or comma) detection and word alignment functions that work with the serdes functions provide dedicated support for multiple patterns. These include the K28.1, K28.5, and K28.7 patterns for support of Gigabit Ethernet, XAUI, FibreChannel, InfiniBand and Serial RapidIO, as well as the A1A1 and A1A1A2A2 patterns used by SONET/SDH.

In addition to built-in pattern detection, these devices can provide very robust support for specific protocols. For example, the transceiver block of a Stratix GX device contains all the elements needed for implementing the physical layer of the 10 Gbit/sec attachment unit interface (XAUI). This layer is composed of the Physical Coding Sublayer (PCS), Physical Medium Attachment (PMA), and Physical Medium Dependent (PMD) sublayers, including a channel aligner state machine that aligns the data of the four receiver channels to the same recovered clock before sending it on to the next layer. The higher XAUI layers, such as reconciliation, MAC, switching functions, and protocol bridging, can be implemented in the programmable logic portion of the FPGA.

Finally, these FPGAs also provide robust resources for testing their built-in transceivers, including multiple serial and parallel loopback modes (normal and reverse) and embedded pseudo-random bit stream (PRBS) generators and verifiers for built-in self-test (BIST) functionality. Complete embedded testing resources like these are vitally important for debugging high-speed transceiver operation without consuming programmable logic resources.

Overcoming limitations

Copper interconnects are limited by three components: the backplane material, the connectors, and the transceiver technology. Today, link rates of 3.125Gbit/sec are achievable with cost effective implementations. New dielectric materials are being produced all the time, but there are cost limitations. Materials that perform to 5 or 6Gbit/s ec are available for a modest premium (130% to 150%), with 10Gbit/sec possible for the most money (200-300%).

A major component affecting link speeds is the connector technology. There are many shielded differential connectors with 5 and 10Gbit/sec capability in the contact design. Most become limited by the connector to backplane interface. A press fit pin that inserts into a plated hole creates a "stub" that has significance between 4 and 5Gbit/sec. The "zero" created by this stub causes a dip in the frequency response and a shift in the phase that limits the upper data rate.

A "surface mount" connector eliminates the stubs. It allows the copper interface to better exploit the capabilities of the contact design and the backplane dielectric materials.

To be effective, a contact that penetrates the outer layer of the backplane and mates directly with the first stripline signal layer creates an optimal impedance characteristic. The backplane would have to be built with blind and buried vias. While this increases fabrication costs, they are offset by improved routing densities and significantly fewer layers. Mesh backplanes especially benefit from this kind of connector design. It is an area where immediate improvements can be achieved.

Transceiver technologies also continue to improve. Improvements in drive and frequency response can even offset for some of the deficiencies in the connector and backplane environment. Transceivers that signal at multiple frequencies can offer interoperability with older technologies. Features such as auto-detection of signaling rates can also facilitate backward compatibility.

Increases in backplane, connector, and transceiver technologies can provide linear increases in link speed. These technologies can benefit new systems, but are not always backward compatible, and may even require new electrical and mechanical standards. Another approach puts the burden on physical interface devices to solve the compatibility problem.

Multi-level signaling can transfer higher data rates that work with existing connectors and materials. Viterbi encoding allows signaling patterns to be selected that do not violate frequency limitations and maintain clock recovery.

Multi-level signaling has the advantage that it can be used in a single level mode providing backward compatibility. The addition of Viterbi encoding also improves error rate. It has the disadvantage that it is a significantly more complex channel to build. This technology has been used in hard disk drives for years to extend data density beyond the physical bit density of the medium. The problem is analogous to the limitations of the medium provided by the connectors and backplane.

The demise of copper interconnect has been predicted for many years. Optical interconnects were going to replace copper at speeds above 1Gbit/sec. Today 3.125Gbit/sec over a single link is practical, and various technologies are coming to support rates to 10Gbit/sec. Technologies such as high speed SERDES embedded in FPGAs make high speed serial based systems easier and cost effective to build. While the research in optical interconnects is promising, there is still much that can be done with connector and transceiver technology to continue to make copper interconnects useful for new generations of systems.