Editor's Note: Like many folks, I use Ethernet in one form or another every day without actually having much of a clue as to how it works. Of course, the fact that FPGAs are increasing being used in Ethernet-based systems means that it would be useful for me to discover more. Hence, I was delighted to be presented with the opportunity to (a) read this two-part article and (b) share it with you.
See also Part 2
Ethernet audio video bridging (AVB) is a new IEEE 802 specification, under development, that provides bandwidth and latency guarantees for interactive multimedia applications. In part one of this two-part article series, the focus is on the enhancements brought about by AVB and the details of various components of the proposed standard.
Ethernet-based networks are the most dominant form of networks used in the world today. A significant portion of the Internet infrastructure is built on Ethernet-based bridges, hubs, switches and routers. Over the years, Ethernet has improved to support faster data rates and different operating schemes (full and half duplex). However, Ethernet was originally designed to handle purely static data and reliability; ordering and packet delay were not important considerations. The old Ethernet networks simply were not designed to address these new and growing convergence needs from voice, media and other dynamic content.
Traditionally, multimedia applications over Ethernet have been facilitated by either the high bandwidth or prioritization schemes defined at layer 2. These enhancements ensure that streams are transported, meeting very loose latency and bandwidth requirements. The multimedia traffic had to contend with TCP traffic, however, causing excessive delay and large jitters in packet arrival time.
Layer 3 networking standards have built-in quality of service (QoS) mechanisms in the form of differentiated services and integrated services to handle multimedia traffic, guaranteeing them assured bandwidth and small latency with low jitter. These schemes rely on resource reservation (RSVP) to negotiate such guarantees with the network. And because Ethernet is the backbone of networks everywhere, it needed updating to transport these streams with similar guarantees.
Historical advances in Ethernet
Ethernet was invented in 1978 by Bob Metcalfe. It was originally designed to be used for light industrial load applications for the purpose of simple data transport. Given its simple nature, flexibility, and low cost, it soon became the most popular local area network (LAN) technology.
Various technological advancements have increased Ethernet data throughput from 1 Mbps to 10 Mbps, 10 Mbps to 100 Mbps, and 1 Gbps to 10 Gbps. Many vendors use Ethernet in proprietary ways and support non-standard speeds such as 2 Gbps for switching backplane applications. Other ongoing advances in Ethernet include the development of 100 Gbps Ethernet and its use for metropolitan area networks (MAN). Today, there are more Ethernet ports than any other connectivity technology in the world.
Ethernet media types
Ethernet supports many different media types, ranging from low-cost, unshielded twisted pair to fiber optical networks. These different media types enable various Ethernet applications and provide good cost/performance trade-offs. For low-bandwidth home needs, low-cost, unshielded twisted pair cable is suitable, whereas metro networks or university campuses with high-bandwidth needs can use the costly fiber optical cables. The fiber optical cables support speeds as fast as 10 Gbps (proven up to 100 Gbps in laboratories) and transport data over long distances.
Advances in physical media technology such as CAT-5, shielded twisted pair and fiber optical cables have also enabled higher Ethernet speeds on traditional media, thus enabling high-bandwidth multimedia applications. In many cases, media plays an important role in application deployment because actual data throughput is lower than the actual media capacity. Consequently, over-provisioning – providing more bandwidth than an application needs – allows traffic to flow uninterrupted.
The original Ethernet specifications relied on the carrier sense multiple access/collision detect (CSMA/CD) algorithm and supported only the half-duplex mode of operation (i.e., data sent only in one direction). In this mode, various stations either connected together on a single wire, sharing the transmission medium, or stations connected through a physical layer device called a hub.
In CSMA/CD, the station intending to send data first senses the medium idle for a small period, after which it transmits a frame. If another station sends data at the same time, a collision is detected and both (or all if more than two stations were sending data) stations back off for a random period of time. After this random back-off period, the transmission is attempted again.
CSMA/CD worked great for low-bandwidth applications in the 1980s and can still be found in use today in some LANs. However, because of the increasing popularity of Ethernet and pressing demands on its performance, Ethernet networks were segmented and connected through bridges. These bridges forward the traffic on a given LAN if and only if the desired destination resides on a given LAN (or if the current station location is unknown; in that case, the frame is broadcast as appearing on all LANs).
The other advance in Ethernet that accompanied the development of bridge technology is micro-segmentation (where one station and switch on a given LAN or LANs are connected to form a catenet) to the point that stations are now mostly connected only through bridges/switches. The station and bridge connected on both ends of a transmission facility form an Ethernet LAN segment. Both the devices operate in full-duplex mode.
This mode of operation is different from the half-duplex mode of operation previously described, as both the simultaneous receipt and transmission of data traffic are possible. A flow control mechanism is provided if the station is swamped by more data it can handle. The station then sends a pause frame to the transmitter requesting that it stop transmission for a specified amount of time (included as one of the fields of the pause frame).
1. Full-duplex and half-duplex Ethernet.
Link Aggregation and Controller Protocol (LACP)
With the introduction of the full-duplex specification, Ethernet is now used to support higher bandwidth applications than could previously be deployed on half-duplex networks. By the early 1990s, Ethernet was widely deployed in LANs and the fastest speed available was 100 Mbps. This was not sufficient to support the traffic needs of the core switches used in campus backbones or to support multiple streaming multimedia applications from a server (such as a video server).
To meet this demand, link aggregation was developed. This protocol allows two or more bundled Ethernet links to appear as one single link to higher layers. The link aggregation protocol is completely specified in the 802.3ad link aggregation standard and is backward-compatible to the existing Ethernet. The link aggregation protocol maintains a soft state of the bundled links and can be reconfigured under configuration control. The link aggregation protocol can aggregate N full-duplex, 100 Mbps links to create an N*100 Mbps data pipe.
LACP mandates that all bundled links must operate at the same speed. The data-link layer mandates the link layer devices to maintain traffic ordering at a given priority level. Thus, traffic distribution across links must be carefully managed, especially during transitions (since LACP maintains a soft state). The traffic is distributed by specifying the use of a single Ethernet link (selected from among those bundled) for source address or destination address (SA or DA) pairs at a given priority level. This solves the problem for switches. If the link aggregation is with a server, then DA/SA pairs or higher layer protocol fields can distribute traffic among the links involved.
Architecturally, the LACP application is a client to the media access control (MAC) layer. The LACP defines frame collection and distribution along with an LACP agent. The LACP defines two modes to re-distribute traffic among links. First is with the use of special packets called markers. The frame collector at either end of the link parses special marker packets from the incoming stream. These packets are then passed to the LACP agent. The LACP agent can also instruct the distributor to generate marker response packets.
To redistribute traffic among Ethernet links, the higher layer sends a special marker packet at the point where it wishes to re-distribute traffic and waits for the marker response packet. The marker response packet is generated by the receiving agent only after all frames have been passed to the higher layer applications.
Another mode of operation is to wait for a period called a collector max delay, which is determined during the link aggregation phase. After sending an Ethernet frame, no further frames are sent on the link. It is assumed that the frame has been passed to higher layers within the collector max delay, so now link reconfiguration can happen safely.
As mentioned previously, the link aggregation maintains a soft state and can be reconfigured under configuration management. This reconfiguration may be required in normal operation to better manage the ongoing load and according to application demands. The link bundling happens through the exchange of the LACP protocol data unit (PDU). The LACP uses the concept of actor and partner, with the actor initiating the link distribution process. PDUs exchange information about their own Ethernet links (speed, priority, key) and what information the link has about its partners' links. Links at a particular speed that have the same key can be grouped together to form a single link, with priority acting as a breaker in case of a tie. The LACP allows a mode for users to override the soft state determined by the protocol.
2. Link aggregation between switches and a switch server.
Apart from increased bandwidth and incremental link bandwidth, there are various other advantages of link aggregation. Link aggregation provides higher availability, automatic load sharing and rapid reconfiguration in case of outage. Link aggregation is also completely backward-compatible to existing Ethernet and higher layer protocols. Fig 2 depicts link aggregation and its application.