Network switch device equipment balances performance, cost and power in the cloud

Much like microprocessors – which several years ago hit a scalability ceiling in terms of single-core processing throughput – switch chip architectures now face aggregate processing bandwidth requirements that favor a multi-core approach in order to meet data center performance, cost, and power requirements.  Yet adopting a multi-pipeline design creates MMU partitioning challenges that demand careful consideration. For instance, an architecture that splits MMU resources per switching core may deliver poor many-to-one burst absorption based on static allocation of each buffer on a per-port or per-port-group basis. Delivering high performance in a multi-pipeline switch mandates an optimum, centralized MMU architecture, in which packet buffering is dynamically and adaptively shareable across all ports. 

Centralized, Shared, Intelligent MMU is the Solution

Network designers have an alternative to design approaches previously limited by statically allocated, per-port packet buffering schemes. By implementing a centralized, fully shareable, dynamically allocated, and adaptive packet memory management architecture, network operators can achieve the holistic approach essential to managing high-performance cloud applications.  For example, Broadcom’s StrataXGS switch architecture featuring Smart-Buffer technology incorporates a scalable multi-pipeline design interconnected through a centralized MMU architecture; further, its packet buffer is right-sized and dynamically shared across all ports for excellent burst absorption. Its combined architecture enables global admission control, queuing, policing and shaping functions. These intelligent switching capabilities deliver optimal buffer utilization and burst absorption for data center workloads by taking a holistic approach to buffer management – using real-life data center traffic scenarios to maximize overall throughput and lossless behavior.

Excellent burst absorption is the priority, particularly in today’s networks characterized by transient congestion.  Due to the random nature of network traffic, multiple packets can simultaneously contend for the same egress port. Since egress port bandwidth is limited, some of the packets would have to be enqueued in the system buffers to be transmitted at a later time. The number of packets that may need to be enqueued depends on the nature of the incoming traffic. As a result, the buffer management policy must provide enough resources to enqueue packets during times of congestion, for a wide variety of traffic scenarios. This is essential, as a buffer management policy offering good burst absorption leads to fewer frame drops.

Further, the switch must ensure that during times of congestion, uncongested ports do not get starved out of access to the shared buffer pool. The buffer management policy must also ensure that congestion on some ports does not unfairly throttle link utilization on uncongested ports. Ultimately, the choice of parameters for a buffer management policy should not depend heavily on the nature of the traffic. A single parameter setting must be able to provide optimal (or near optimal) performance for a wide range of traffic scenarios.

Figure 1. Congestion is localized to a subset of egress ports and realistically never happens on ports simultaneously.  This enables a centralized on-chip buffer to be right-sized for overall cost and power and shareable and weighted towards congested ports exactly when needed.

Figure 2. Broadcom's Smart-Buffer delivers up to 5x better packet buffer utilization.

Ideally, dynamic sharing and self-tuning is transparently enabled across all ports, regardless of the processing pipeline or switching core to which the ports belong. In addition, the centralized buffer should be allocated based on class of service or priority group.  Available buffer resources can therefore be partitioned into separate, virtual buffer pools and assigned to special traffic classes. This is especially useful in converged I/O scenarios where some traffic classes (such as storage) may require guaranteed lossless behavior.

Intelligent Switch Technology Meets Cloud Metrics of Cost, Power, and Performance

Data center workloads demand high throughput and robust, consistent performance from Ethernet switches; these performance features are required in order to handle characteristic traffic patterns in their networks. Cloud-centric workloads such as Hadoop/MapReduce require network switches with excellent burst absorption capabilities in order to avoid TCP incast problems.  With the current transition of server interfaces from GbE to 10GbE performance, demands in server access infrastructure necessitate highly integrated network switch devices that utilize multiple switching cores and pipelines. At the same time, cost and power metrics in the cloud drive the need for fully integrated buffers and sophistication in switch MMU design.

Today, innovative and proven switch device technology enables cloud infrastructures based on a more intelligent approach – incorporating centralized, fully shareable, dynamically allocated, and adaptive packet memory management architecture. Using cost-effective, integrated packet buffering, intelligent switches maximize burst absorption capability through full resource sharing and dynamic port allocation schemes. This holistic approach is proving ideal for cloud network operators, facing daunting challenges in scaling their network infrastructure to tomorrow’s workloads and selecting appropriate switching equipment with a greater understanding of underlying switch architectures and their required tradeoffs between performance, cost, and power.

About the Author

Sujal Das serves as Director of Product Marketing for Broadcom Corporation’s Infrastructure and Networking Group (ING).  In this role, Das is responsible for driving Broadcom’s Ethernet switch business in the data center and enterprise LAN market segments, and the development of product, ecosystem and strategy based on technology trends and application workloads.

Das has extensive experience in semiconductors, networking software, data center network architectures, virtualization, and server systems. Prior to Broadcom, Das served in senior product development and marketing roles at AMD, Marvell Semiconductors and Mellanox Technologies. 

Das earned a BS EEE degree from the Birla Institute of Technology and Science in Pilani, India and an MBA from Santa Clara University. He has published and presented on numerous data center networking and virtualization related topics in well-known industry publications and events.  Das has been active in the open source community, driving multiple initiatives that have helped proliferate the adoption of high-performance networking in data center applications.