10GbE SFP+ PHYs: Requirements and leading solutions

Great strides have been made over the past few years in bringing 10 Gigabit Ethernet (10GbE) products to market, and the technology has progressed from lab demonstrations to commercial deployment. To date, however, the 10-GbE market-ramp is slower than forecasted, with cost cited as a major reason for the delay.

Within this context, a new optical transceiver form factor has emerged as the catalyst to spark widespread 10GbE adoption. The original 10G optical module form factors of XENPAK, X2 and XFP are now being replaced by the SFP+ module, enabling the 10GbE transition in enterprise networks by meeting a variety of customer needs better than with previous modules.

SFP+ Overview
The SFP+ (Small Form factor Pluggable) module is a variant of the SFP optical transceiver. SFP modules have shipped in volume for years in Gigabit Ethernet and 1-, 2-, and 4-Gigabit Fibre Channel applications. The SFP+ module form factor is 30% smaller, burns less power, requires fewer components, and is less expensive than the XFP form factor (which itself is significantly smaller and lower power than XAUI-based XENPAK and X2 form factors). The SFP+ MSA (multi-source agreement)--defined in the SFF8431 document (ftp://ftp.seagate.com/sff/SFF-8431.PDF), enhances the mechanical form factor of SFP, by adding improved signal integrity and EMI shielding appropriate to higher data rates, and defines new electrical interface--the so-called SFI interface.

Since the SFP+ form factor significantly simplifies the functionality of the 10G optical module, such functions as CDR, Electronic Dispersion Compensation (EDC) and 10G SerDes, that traditionally resided inside the XAUI-based modules, must now be integrated into 10GbE Physical Layer (PHY) devices residing on the HBA/NIC or line card.. While this helps with the reduction of overall system cost (the cost basis for CDR, SerDes, EDC functions not artificially increased as in the optical modules due to poor optics yield), it puts a much sharper focus on the features and performance of the PHY.

The second main advantage of SFP+ is a higher port density such a form factor enables. Today, vendors are shipping modular 48-port switches with GbE line cards. When end users replace one piece of equipment with another, ideally, they would like to have the same number of ports. They don't want to replace a 48-port switch only to replace it with a 4-port switch, because such a replacement would require more equipment, which in turn would take up even more real estate. Moreover, when there are fewer ports, the cost per port is much higher.

SFP+ optical transceivers enable the highest port counts per-card of any of the 10-Gbit/sec optical modules. The SFP+ offers densities comparable to Gigabit Ethernet optical modules and to copper Gigabit Ethernet cards with RJ-45 connectors.

SFP+ Media Options
A range of 10GBASE SFP+ modules are available for a wide variety of 10 Gigabit Ethernet connectivity options in data center, enterprise wiring closet, and service provider transport applications.

10GBASE-SR modules utilize 850nm optics to support a link length of 26m on standard Fiber Distributed Data Interface (FDDI)-grade multimode fiber (MMF). By using 2000 MHz*km MMF (OM3), up to 300m link lengths are possible. The 10GBASE-LR modules, with 1310nm optics, support a link length of 10 kilometers on standard single-mode fiber (SMF). SMF is designed to support high-capacity, low-cost transmission, and can be used for cross-aisle and inter"datacenter applications. MMF fiber however, is the more ubiquitous and cost-effective fiber of the two, and is found in nearly all data centers today. 10GBASE-LRM modules support connectivity up to 220m on OM1 MMF links using 1310nm optics. 10GBASE-LRM links also utilize EDC technology that is resident in the SFP+ PHY devices, located directly behind the SFP+ optical modules.

Recently, a Direct-Attach Copper (DAC) version (twin-ax cable carrying 10G serial data, terminated with SFP+ connectors and fitting interchangeably into the pluggable SFP+ cages) has gained popularity in the datacenter. These twin-ax cables are available in various gauges and lengths of up to 15 meters and beyond, although the SFF8431 MSA specifies shorter distances (actually specifies channel characteristics that translate into shorter distances of twin-ax cable).

Because the cabling distance is limited, each server is directly connected to a top of-rack (ToR) switch with no intermediate patch panels requiring management. This approach dramatically simplifies cabling and termination since the cabling is contained within a single rack, and it works well with the concept of a modular datacenter.

SFP+ PHY Requirements
The market for 10 Gigabit Ethernet SFP+ PHYs is expected to grow rapidly in datacenter, enterprise and carrier networks, driven by the increasing demand for bandwidth-hungry video content and virtualization. Industry analysts expect the market for 10GbE SFP+ PHYs to grow tenfold in port count over the next five years, representing 58 percent CAGR over that period.

Additionally, the performance characteristics of the PHY now become key features in the choice of device since the SFP+ functionality has been simplified significantly (no CDR inside module) and an additional channel of PC board (FR4) has been added between the module and the PHY. Essentially, the PHY now has to be capable of receiving (and transmitting) 10G signals over up to 8 inches of FR4, through the SFP+ connectors (which are getting more complex with double-stacked cages) and the requisite length of fiber/DAC, before reversing the sequence at the other end of the link. Besides this, the PHY/SerDes devices also have to be in small package sizes and consume less power to allow high port density designs for both embedded switches and top-of-rack switches.

Port Density
From the standpoint of datacenter-focused 10GbE networking equipment such as high-end routers, network aggregation switches, blade servers and top-of-rack datacenter switches, the main requirement is for SFP+ PHYs to enable increased port density while maintaining increased power efficiency. 10GbE switch port counts typically vary from 8 to 16 10GbE channels, to as many as 48 channels on next generation 10GbE blade server or enterprise chassis systems.

The obvious feature of the PHY/SerDes that enables high port density is the package size. Current offerings for single channel PHY/SerDes range from 10mmx10mm to 13mmx13mm packages; while dual channel devices range from 16mmx16mm to 23mmx23mm packages.

Given the requirement for additional clearance to allow for manufacturability and the need to not route high-speed 10G lines close to each other, the 23mmx23mm size starts becoming unsuitable for dense designs with DFM principles in mind.

While single and dual-port 10GbE SFP+ PHYs have been shipping in volume for several quarters now, the emergence of dense quad port PHYs is viewed as critical for the development of switches with 48 ports and above. Two clear architectures seem to be emerging with the next generation switch devices needed to support these high-density systems: i) PHY/Switch interfaces that are XAUI and RXAUI (reduced pin count XAUI) compatible, and ii) XFI (10GF serial) interfaces.

While vendors of the switch devices have clearly made their choices regarding these interfaces, the 10GbE PHY/SerDes needs to be flexible enough to satisfy all of these interfaces (XFI, RXAUI and XAUI) in as compact a form factor as possible. Small-sized packages, with flexible system-side interfaces seem to be key features in upcoming quad-channel devices that will enable these high-density systems envisioned by next-generation datacenter designers.