Lattice Semiconductor and Flexibilis have released a Gigabit Ethernet Switch IP core that is scalable, non-blocking, and extensible…
Underlying functionality of modern Ethernet switches
Modern Ethernet switches have added significant new functionality to Ethernet while decreasing port prices, becoming the ubiquitous building block of any intelligent network:
- Ethernet switches support auto-configuration; i.e. plug-and-play operation, and also automatic reconfiguration if there are network changes.
- Ethernet switches learn about the network by discovering network addresses (Address Learning). Switches segment users into logical groups to allow efficient provisioning of services (Broadcast and Collision Domains).
- Switches decide how to treat the traffic that is moving through them from the various ports, based on their knowledge of the network (Forwarding and Filtering).
- Ethernet switches also provide the means to handle Quality of Service (QoS). As the type of applications and the data that they generate proliferate, there is a growing need to ensure the condition and reliability of services. For example, real-time applications such as video, voice and critical control data can be assigned a higher priority than web browsing and other non-critical data operations. QoS functionality in Ethernet switches provides a mechanism to classify these various traffic types and treat them appropriately.
The functionality that Ethernet switches support is evolving, not only in current applications like enterprise and data centers, but also in telecommunications providers’ transport networks, industrial networking such as factory automation, and energy utility networks like Smart Grids, as described below.Datacenter Ethernet
Ethernet technology is deployed in virtually every organization’s data center. This broad adoption is driven by the robust Ethernet ecosystem: standardization, multiple vendors and cost-effectiveness. Cloud Computing (Private and Public Clouds) is a scalable, granular and seamless (virtualized) apportioning of data center resources, including compute cycles, storage bytes and network bandwidth, to the various applications. This implies that there is a need for a “unified switching fabric.” This fabric would need to consider the needs of new traffic types – low latency for storage area networking and deterministic delivery of processor to memory access.Carrier Ethernet
Long haul or remote communications has been provided by a SONET or SDH communications infrastructure. Though these legacy systems have been voice-centric, they have been reliable and well understood and define the requirements for future systems and services. These systems have also provided synchronization that is used, for example, in mobile network base stations. The main issues with these systems are that they are less flexible, difficult to scale and are not as well optimized for packet data transmissions. Wide Area Networking (WAN) and Metropolitan Area Networking (MAN) providers need to provide their customers with Ethernet services that leverage the volume and cost advantages of Ethernet technologies for the capacity needs of high-definition video storage and transport. But these Ethernet services also need to provide robust synchronization and reliability guarantees: transfer certainty (determinism) and low latency of the legacy systems. Industrial Automation and Control Networks
Ethernet has also become the de facto networking technology in industrial automation. Switches support the features needed to provide reliable and time synchronized service for real-time control applications. Redundancy is provided by supporting network topologies such as ring, double star and mesh, and automatic re-configuration in case of network failure. Switches also support nanosecond-level accurate time synchronization of all the devices connected to Ethernet.Energy Utility Networks (Smart Grid)
A Smart Grid includes a monitoring system that keeps track of the electricity used in a network, which makes it possible to more efficiently utilize renewable energy sources such as solar and wind power. At short peak times it is possible to let the Smart Grid turn off selected instruments; for example, electric heating or air conditioning, to reduce demand and minimize the need for reserve power. Users are also able to consume the power they need when electricity is least expensive. For example, electric cars can be charged at night when power consumption is lowest and energy least expensive.
Utilities have begun to quickly replace many legacy and expensive proprietary technologies with Ethernet for mission-critical substation local networks due to the flexibility, cost effectiveness and high performance of Ethernet technology. Substation device vendors are incorporating Ethernet support in protective relaying devices. In addition, IEC 61850, a substation communication standard that uses Ethernet as its core technology, has gained significant traction as an international standard for network communications in substations. Also, Ethernet lends itself to fiber-based connectivity, which is vital in electrically noisy industrial environments. Ethernet also supports ring, star and mesh topologies that are highly resilient to network faults, which improves operational reliability.
The applications discussed above reveal certain critical and common requirements for a real-time Ethernet switch. These requirements are discussed next.Real-time Ethernet switch: Requirements and standards
Deterministic and Synchronized
A real-time system implies an “immediate” response to an input. Since the definition of “immediate” can change for various applications, it is best to think of real-time as a “deterministic” response to an input. A response is deterministic if it has a consistent and predictable time delay to a given input. So, by definition, a shared transmission medium cannot be a real-time system.
To develop a deterministic system, special traffic engineering is needed to ensure that non-critical traffic does not adversely impact critical real-time data traffic. The network must support low latency, QoS and time synchronization to perform in a real-time environment. Low latency can be provided by supporting Ethernet switching in hardware. QoS can be used to provide shorter and more deterministic latency for real-time traffic. Time synchronization is needed in real-time networks to enable accurate time stamping of measurements in different devices and also of scheduled control, where a control is taken into use at the same time in every device.
Time synchronization is a critical feature in real-time networks and also in telecom networks like SDH and SONET. In mobile base stations, drifting outside the specified frequency range results in mobile handoff issues, call interference, call drops and call rejection: this reduced service level increases the operator’s Operational Expense (OPEX). Effective synchronization enables post mortem analysis that is used to locate network faults quickly and remotely without emergency on-site calls.
Accurate time synchronization among substation devices and substations is required in a power grid for fault analysis, time coordinated control and wide area situational awareness for improved grid monitoring and reliability. Frequent (many times a second) and precise time synchronized measurement of voltage and current from various locations across the power grid using devices called Phasor Measurement Units (PMUs) provide an image of the power system. This wide area situational awareness capability, which eases congestion and bottlenecks in power systems and mitigates blackouts, is driving the need for accurate time synchronization in substations. Sub-microsecond time synchronization accuracy is needed for Phasor Measurement Units and is desirable for protective relay devices.
The source of the synchronization can be external, like GPS (which adds cost and is single sourced from the US Army). The de facto industry standard for time synchronization of electric utility equipment in substations has been IRIG-B with GPS clocks as the time source. However, IRIG-B requires a dedicated copper network that is different from a data network connecting devices such as protective relays, fault recorders and PMUs.
The source can also be from within the communications network. The Precision Time Protocol (PTP) as defined in IEEE standard 1588-2008 can provide similar robust synchronization and allow the service provider to leverage the cost and scalability of Ethernet services.
There is a growing trend in substation automation to utilize an Ethernet-based communication architecture with the goal of enabling interoperability among various substation devices. The move towards the IEC 61850 standard, presence of Ethernet in substations, higher time synchronization accuracy requirements and challenges associated with the IRIG-B standard have created an opportunity for Precision Time Protocol (PTP)-based time synchronization among substation devices.
In addition, the same requirements for real-time measurement and control arise in other critical infrastructure networks: water supply systems, gas and oil distribution, road networks, railway networks and other transportation.IEEE-1588-2008 (v2) Precision Timing Protocol
The IEEE 1588 "Precision Clock Synchronization Protocol for Networked Measurement and Control Systems," or "Precision Time Protocol" (PTP), was developed to quickly synchronize networked clocks with differing precision, resolution and stability to nearly nanosecond-level accuracy. PTP was designed for low cost implementation, simple installation and maintenance in Ethernet networks.
Figure 1. Precision timing protocol synchronization
IEEE-1588 utilizes a master-slave hierarchy (Figure 1). Time information is fed to a network by a master, and slave clocks automatically synchronize their clocks to the master clock. Several masters can be in the network, and one of them is selected to be used automatically. Other master clocks activate immediately if the previously selected master is removed from the network. For the network to maintain robust synchronization, Ethernet switches are key elements in the network that should support this protocol.Redundancy
In mission critical networks, breaks in communication are not tolerated. To achieve the required availability there should be redundancy in the network. One solution is to use the Rapid Spanning Tree (RSTP) protocol to enable Ethernet communication in a network where there are redundant links. When RSTP detects a link failure, it tries to use the redundant link (failover). In this way, RSTP provides redundancy, but communication can suffer noticeably during failover, especially at higher port speeds. High-Availability Seamless Redundancy (IEC 62439-3)
The currently developed standard IEC 62439-3 defines new redundancy protocols. The High Availability Seamless Redundancy (HSR) protocol provides totally lossless communication over Ethernet: in failover there is no break in communication. This is achieved by sending every data twice (in Figure 2, the red and green arrows) and the receiver removes possible duplicates. In case of failure, only the duplicate is lost. Normal Ethernet devices can be connected to HSR devices via Redundancy Box (RedBox). HSR supports ring and double star topologies.
Figure 2. High-availability seamless redundancy (HSR)
As established protocols like IEEE-1588 migrate into new markets or new standards like HSR emerge, FPGAs are the best option to develop the right solution with the additional ability to manage risk. There are other benefits to using FPGAs that are discussed below. FPGA-based real-time Ethernet switch
An FPGA is programmable hardware, a technology that combines features from software and hardware. From the software world comes the programmability: the functionality of the device can be changed after manufacturing is completed. For example, if a new version of an existing standard emerges, support for the new version can be uploaded to the device in the field or easily changed in production. From the hardware world comes the device performance. So, an FPGA has the flexibility of software and the performance of hardware.
Advantages such as large logic resources, flexibility of use, low NRE (Non-Recurring Engineering) costs, fast design cycle, life cycle extension, factory or field upgradeable, standard IP cores and a good design flow to build custom functional blocks allow for the efficient use and leverage of FPGA technologies.
An FPGA-based Ethernet Switch can have one or more of the following cost saving capabilities: an arbitrary number of ports instead of the fixed 8, 16 or 24 ports in Application Specific Standard Products (ASSPs), and an arbitrary mix of ports like 3 10/100Mbps ports, 5 10/100/1000Mbps and 2 10Gbps ports, unlike the fixed configurations in most Ethernet Switch ASSPs. There is also significant potential to integrated adjacent functionalities.
Recently, Lattice Semiconductor and Flexibilis have released a Gigabit Ethernet Switch intellectual property (IP) core that is scalable (configurable number and type of ports), non-blocking (forwarding at wire-speed) and extensible (adds emerging capabilities not available in most ASSPs). The Switch (Figure 3) conforms to IEEE standards 802.3 and 802.1D and supports Ethernet packet filtering and prioritization on each port. The switch also supports the IEEE1588-2008 end-to-end one-step transparent switch processing in hardware. For redundancy, both RSTP and HSR are also supported.
Figure 3. Flexibilis Ethernet switch
The LatticeECP3 FPGA family provides a very cost effective and low power solution to implement such an Ethernet Switch. The LatticeECP3 FPGA supports various devices, ranging from 4 SerDes channels to 16 SerDes channels. Each of these SerDes channels can be configured independently to be a Tri-Speed Ethernet (10/100/1000Mbps) port. The core logic can be configured to enable any appropriate feature subset to right size the switch for the target application.Summary: An FPGA-based real-time Ethernet switch
Real-time Ethernet Switches are becoming increasingly valuable in many applications such as industrial automation, smart grid, electricity and other utility networks; Data Center Ethernet for convergence of separate compute, storage and connectivity networks; and Carrier Ethernet for both wired broadband access (e.g., Passive Optical Networks) and wireless broadband access (e.g. Microwave Backhaul).
The standards to achieve deterministic delivery with appropriate QoS, reliability and highly accurate time synchronization are evolving rapidly as operators and system providers understand the right deployment trade-offs.
It is evident that new, winning solutions based on Ethernet technology will be the ones that can adopt new standards cost-efficiently and preserve time to market.
Programmable FPGAs are the ideal foundation for these solutions Additionally, FPGAs provide a migration path to newer semiconductor geometries to preserve design investment. FPGAs help not only accelerate time to market, but also extend time in
market.About the authors
Lalit Merani is a Senior Product Marketing Manager at Lattice Semiconductor. Prior to joining Lattice, Lalit was a Senior HW Engineering Manager and System Architect for Nokia and prior to that he held senior technical leadership positions at Intel. Lalit holds a M.S. from Oregon State University.
Timo Koskiahde is the Chief Technology Officer of Flexibilis, Oy. Prior to joining Flexibilis, Timo was a senior design engineer for Nokia and has been in engineering for various telecommunications and software companies. He holds a M.Sc from Tampere University of Technology, Finland.
If you found this article to be of interest, visit Programmable Logic Designline
where you will find the latest and greatest design, technology, product, and news articles with regard to programmable logic devices of every flavor and size (FPGAs, CPLDs, CSSPs, PSoCs...).
Also, you can obtain a highlights update delivered directly to your inbox by signing up for my weekly newsletter – just Click Here
to request this newsletter using the Manage Newsletters tab (if you aren't already a member you'll be asked to register, but it's free and painless so don't let that stop you [grin]).