Networking Memories: Intelligence for 400G app acceleration and host offload—Part III

Networking Memories--Part III, Page 3.

Implementation of a 4x100G Two-Rate/Three-Color Meter (TRTCM)

Metering network traffic using a token bucket is one of the most challenging operations to accomplish at high performance line rates.  While other applications examined in this series of articles can attempt to utilize architectural tricks to enhance performance of low performance and legacy memory devices, the metering function is unique in that the records need to reside in a single location and by nature of the high speed comparisons and updates, it cannot be easily cached. Metering operations at the access point of the network can take place at low line rates, but once the flows are aggregated onto higher speed links at the edge of the network metering becomes a significant challenge.


Figure 6 illustrates a Two-Rate/Three Color Meter (TRTCM) with the first bucket representing the Commit Rate and the second bucket the Excess Rate.  Tokens are credited to the buckets at the Input Rates (CIR and EIR) and the buckets are allowed to fill to the bucket size limit (CBS and EBS). Each flow has a unique bandwidth profile describing the input rates and bucket size which is assigned and stored in the metering record when the flow is established.  Each packet associated to a flow debits tokens first from the Commit Bucket and when that is depleted from the Excess Bucket. The packet is colored Green or Yellow depending on which bucket the consumed tokens come from, or if both buckets are empty colored Red. This function requires multiple operations at line rate for each metering instruction. Take for example a 4x100GE line card with a packet arrival rate of 6.67ns and a metering record of 128b:

1. Upon a packet arrival the flow ID is used to select the metering record.
2. The meter calculates the token credit for the commit rate and the excess rate buckets based on the time since the last update and the bandwidth allocation.  This credit is added to the existing bucket values.
3. The packet size is debited from each of the buckets as defined by the metering algorithm   
4. If there are insufficient tokens the packet would be recolored.
5. The Meter function passes the updated color and bucket value information along to the Marker and Dropper to be used for Policing and/or Shaping.

The MoSys Bandwidth Engine Macro device, with its onboard accelerator is capable of atomic fire-forward metering operations which can process records wholly internal to the device, reducing the number of memory bus transactions from six down to one as well as relieving the host of the computations required for the update.  It can return the result and retire the entire operation in under 30ns, far quicker and at substantially lower power than any other solution. Furthermore, the metering macros on the Bandwidth Engine can be saturated using only 8 SerDes lanes, further reducing the power, pincount and host resources. A comparison of different implementations of a 4x100GE TRTCM servicing 4M flows can be seen in Table 2.

The Bandwidth Engine, with the high performance interface is capable of unified memory applications where portions of the device can be allocated for flexible combinations of Statistics, Metering, Buffering and Lookup applications as needed.

Conclusion

As network performance continues to scale far faster than the improvements realized by Moore’s Law, architectural improvements are necessary to keep up.  Networking equipment has transitioned to highly parallel, multi-threaded processing System-on-Chip complexes which require an insatiable amount of memory bandwidth.  MoSys Bandwidth Engine provides a threefold solution to the growing needs of this market; first, the highest performance single chip buffer device, capable of up to 384 Gbps, second the highest performance memory access device, capable of up to 4.5 billion dual ported accesses per second, and finally an offload accelerator for statistics counters and TRTCM metering; a device unique to the industry in its capabilities. These solutions are supported by the open GigiChip Interface; a 90% efficient transport protocol for low latency chip to chip, CRC protected communication.  These innovations will enable next generation networking equipment to deliver unparalleled performance with lower power, smaller footprint and lower cost than other solutions.

 

About the Author
Michael Sporer brings over 20 years of marketing, sales and engineering experience to MoSys. Prior to joining MoSys, he was a Technology Strategist for Micron Technology, an industry leader in semiconductor memory products. Previously, he was Director of Technical Marketing at LG Semicon and with Hewlett Packard in the Memory Technology Center. Mr. Sporer holds a Masters of Science in Engineering from Stanford University, and a Bachelor’s degree from the University of Michigan.