Building a network performance analysis test system with Linux, Tcl/TK, SQL & extremeDB

A router may be designed to interact comfortably with 10,000 other devices. Must its producer maintain a test network with all the servers, gateways, switches and other gear needed to reproduce the normal as well as stressful conditions that the device is likely to encounter?

Obviously, this could get costly, not to mention time-consuming. And if it's hard to test a garden variety router, how can a manufacturer put the largest telecom and networking devices - such as 10GB Ethernet switches - through their paces, or gauge how different system interactions and demand scenarios affect the delivery of complex services, such as the "triple play" - the merging of voice, data and video - offered by carriers?

That's where Spirent Communications and our Spirent TestCenter, come in. We designed it as a comprehensive network performance analysis system, providing software and hardware to emulate virtually any imaginable network traffic scenario, to simplify and accelerate testing.

Functional and performance testing are centralized through the use of a single application that generates control and data plane traffic and collects test results. Performance is such that users can accurately measure devices' latency in the single-digit microsecond range.

What enables Spirent TestCenter to keep up with network packet traffic flowing at line rate is a unique, highly distributed Linux-based architecture -so distributed that a single installation can amount to hundreds or even thousands of individual embedded Linux systems, operating at the edge of the device.

This processing location - at Spirent TestCenter's ports, where the device handles torrents of inbound and outbound packets - is a key to system performance, because it leverages the large number of CPUs positioned there and ensures that only test results, and not raw data, are transmitted to a centralized control application.

For example, Spirent TestCenter's largest chassis has 15.75 inches of horizontal rack space with 12 slots for plugging in test modules. Each of these cards provides as many as 12 ports, and each port has its own instance of MontaVista Professional Edition embedded Linux, running on a MIPS processor, with a copy of the eXtremeDB in-memory embedded database (Figure 1, below).

Figure 1. Application and database at Spirent TestCenter Port

An open source, SQL-based relational database is also used for back-end storage of test results. Also positioned at the ports are Field Programmable Gate Arrays (FPGAs), typically from Xilinx, which handle the most performance-intensive tasks: generating, capturing and dissecting packets.