EDA Platform

EDA Takes Advantage of an NT Compute Farm Compaq is migrating to a Windows NT-based EDA environment. The first step, building an NT compute farm using its own machines and several carefully selected support applications, is paying off. by Wendy Stresau and Ben Buzonas

In 1997, the need to expand EDA resources at Compaq prompted the company to consider a crucial question: Should we adopt more Sparc-based hardware for EDA or turn to the Intel-based hardware that we manufacture? It appeared that the key EDA vendors were now supporting NT as a standard operating system platform, but moving to Compaq hardware and Windows NT too soon could lower EDA productivity and increase time to market for the company's products. Nonetheless, the company decided to migrate EDA resources to Compaq systems running under Windows NT, including an NT-based compute farm.

For a successful migration, the hardware must provide sufficient EDA performance and the operating system must be stable enough to continue to deliver designs on schedule. Additionally, because designs are becoming increasingly complex, the hardware and operating system must support sufficient memory. In mid-1997, Compaq began shipping the Professional Workstation line of NT workstations, which offers performance comparable to that of Sparc-based hardware. The Compaq machines also offer large memory complements, so Compaq moved forward with the process of migrating to an NT-based EDA environment.

The migration of our Unix compute farm to an NT farm--now more than half complete--required a series of steps. First, we had to make sure that the currently available tools for NT could actually perform the EDA tasks our designers required of them. Then we began to move our desktop computing environments to NT, maintaining enough Unix systems to perform the tasks not yet covered by NT tools. To make the transition as seamless as possible, we gathered up a few key system support applications, most of which mimic the functions of the Unix support applications used to control our original farm. Finally, we attended to issues of interoperability, including file sharing and X terminal emulation software.

Motivated migration

At the beginning of the NT migration process, Compaq needed to ensure that vendors would offer NT applications that meet a wide range of EDA requirements: logic simulation, verification and regression, synthesis, design capture (schematic and Verilog code, for instance), FPGA design, high-speed system design (signal integrity analysis), PCB placement and routing, and design kits from ASIC vendors. The PCB layout tool vendors were early adopters of NT, and many of the other vendors have followed suit. Currently only a few of the required pieces of the ASIC/IC design flow are missing, and we are working with EDA vendors to ensure that all of their tools will run on NT.

Given the available tools and with expectations of others coming in due course, we began by implementing a Compaq-NT compute farm. Since the compute farm came into being in 1997, Compaq has operated a mixed SunOS and NT environment for EDA (see Figure 1).

The next goal was to move 75 percent of the EDA desktop compute environment to NT, an activity well under way. A significant number of designers are now running only NT on the desktop, using X terminal emulation software whenever they require access to Unix. Unix workstations will constitute 25 percent of the computing environment, functioning as X-term hosts or running the EDA applications that are not yet available on NT. The ultimate goal is to migrate Compaq's entire EDA computing environment to NT PCs by the end of 2000.

We're seeing improved performance on EDA tasks, partly because our Unix compute farm included some fairly slow legacy systems (Solbourne 906 servers). Replacing the old servers with our own hardware has boosted performance significantly.

We also wanted to move to NT to standardize on a single desktop system in the engineers' offices. In addition to saving space, the move makes it easier to use office productivity tools such as Excel and Word alongside the EDA tools, since it simplifies the sharing of data.

Finally, Compaq expects to get these benefits for less than it would have cost to upgrade and maintain EDA resources with more Unix hardware. The initial purchase price of the Compaq hardware is significantly lower (even for users outside of Compaq), and the PCs will also have a longer productive life. Unlike an old Sun workstation that must be scrapped because no one has a use for it, a Professional Workstation retired after 18 months or so still has a life as a high-performance PC for people who run office productivity applications. The Compaq hardware also costs less to maintain. Savings on maintenance are high enough to more than offset the expense of retiring a great deal of Unix hardware.

Figure 1	The Unix-NT combo
During the migration from a Unix EDA environment to NT-based systems, Compaq is operating a mixed environment. The first step was the installation of a compute farm based on Compaq hardware running under Windows NT. Both it and the Unix compute farm are isolated from the rest of the corporate network.

Maintenance savings from the NT environment will come as a surprise to some people, but we're seeing these savings now, largely through centralized support for NT. We had worked hard to develop centralized support models for Unix, and we wanted to maintain similar models with NT. Fortunately, leading EDA vendors such as Cadence Design Systems and Synopsys do not charge extra for a license that floats across Unix and NT systems. Thus users can ignore the underlying hardware.

A reliability issue--for now

Some people complain that NT is less reliable than Unix and costs more to support. The reliability issue may currently favor Unix because NT is not as mature, though NT's stability continues to improve. Years ago, when Compaq was making the transition to Unix systems, the support staff dealt with stability issues similar to the ones they see today with NT.

More importantly, NT users can manage the environment to minimize problems and costs. Users can take advantage of Platform Computing's LSF Suite, for example. The tool's automatic fail-over feature ensures job completion as long as another CPU is available to take up the spooled job.

One way to address concerns about support for NT systems is to use a compute farm. Centralizing the resource and having one group control it eliminates some of the elements that reduce NT's reliability or make it hard to maintain.

Additionally, the compute farm isolates the EDA applications from other types of applications that an engineer might run on a desktop system. Debugging the system is thus easier because the support staff knows exactly what software is running on the compute farm, and the engineer can still share data among all the applications and use them from a single desktop system.

We have always carefully controlled our Unix environments to minimize problems with mission-critical design applications (users didn't control the root even on the desktop workstations). We can't afford not to do the same for the NT environments. By separating most of the EDA tools into an NT compute farm, we can maintain the same computing model at a lower cost.

We've been using a Unix compute farm for about five years with great success. Approximately 60 percent of the jobs on the Unix compute farm at Compaq were Verilog regressions. (We therefore targeted Verilog-XL as the first application for the new NT compute farm, which is now mainly running regressions.) The Unix compute farm consists of centralized compute servers (Sun and Solbourne systems) that reside in a computer room on the same subnet as the NFS file servers (Auspex systems). At the end of 1998, the compute farm incorporated 102 CPUs running more than 500,000 jobs.

In addition to the support benefits, the compute farm model offered a number of performance and utilization enhancements. Specifically, a compute farm makes efficient use of computing resources and gives each engineer access to more CPU cycles. Reliability and availability improve compared with desktop systems or servers, because if one system in the compute farm goes down, many other resources can take up the load. The failure of one or even a few systems in the compute farm probably goes unnoticed by most users. Similarly, networking instabilities among the members of an enterprise-wide design team disappear, because all users of the compute farm are on the same subnet. Engineers also gain access to more memory, while the farm utilizes the memory more cost-effectively than if it were spread across a number of workstations or servers.

The utilization differences between a compute farm and a work group server are dramatic. In a multiprocessor server that we've dedicated to a single project, CPU utilization averages about 20 to 30 percent over the 28 days of the graph, with the peaks and valleys typical of a project cycle (see Figure 2a). During some periods, the engineers use a lot of the server's resources to perform such tasks as simulation, but when analyzing the results, they use hardly any of the server's resources.

Figure 2	Server utilization
Several engineers working on the same design project utilize a server only about 20 to 30 percent of the time, leaving plenty of dead processor time (a). Running the EDA tasks for multiple projects on a compute farm achieves much higher utilization of computing resources (b).

During a similar time period on a compute farm, CPU utilization increases to 80 to 90 percent (see Figure 2b). The difference is that all of the design groups at Compaq share the compute farm, so the usage cycles of each project average out. Nor do engineers need to shop around for available CPU cycles. Under the work group server approach, engineers were wasting some of their design time bartering for capacity on other groups' servers. Given a centrally controlled compute farm that automatically schedules jobs, the engineers make better use of the hardware and of their own time. They simply submit jobs to the queue, and the scheduling software finds the resources for them.

The ability to see a utilization curve also simplifies decisions concerning when to add more CPUs, memory, or other resources to the compute farm. When resources are spread around the company in work group servers, it's difficult to decide when peak demand on any given server really justifies a capacity increase.

Configuring the farm

The most important software for setting up a compute farm--regardless of platform--is the job scheduling utility. When we implemented the Unix compute farm six years ago, we could find no suitable third-party job scheduler. We therefore had to develop the software internally, and we've employed several full-time programmers to support, maintain, and enhance it ever since.

For the NT compute farm, however, Compaq is using Platform Computing's Load Sharing Facility (LSF). Popular for building Unix compute farms for EDA, LSF has recently been ported to NT. The package handles workload balancing as well as license management, automatically sending jobs to the CPUs that possess the right licenses for those jobs and meet the many other criteria for CPU assignment. LSF offers more features--and costs far less--than the scheduling software that we developed internally.

Scripts also serve as powerful tools for setting up both NT and Unix environments. We're using Perl scripts with NT, just as we do with Unix. Compaq standardized on Perl as a scripting language several years ago, a move that greatly simplified the transition to NT. Scripts launch many of the EDA jobs via batch mode into the Unix and NT compute farms. They monitor the status of jobs in the farm and notify designers when their jobs are complete. They also monitor the health of the compute farm and send messages to system administrators when parameters begin to go awry.

Other software also simplifies the NT migration: Hummingbird Exceed, which provides the capability to display X Windows on NT; the Cygnus tcsh shell; Microsoft's Windows Terminal Server, which handles distributed graphics and interactive job control; Installshield and Microsoft Systems Management Server (SMS), for software installation and distribution; our free system monitor, the Compaq Insight Manager (CIM); Vector Networks' PCDUO, for remote administration; and Legato and Diskeeper, for backup and disk defragmentation.

The compute farm hardware

The initial implementation of the compute farm included 24 Compaq Professional Workstation 8000 systems, each of which has three Pentium Pro CPUs, 3 Gbytes of memory, and a 4-Gbyte hard disk drive. We keep one of these compute servers on standby and use three for test purposes. Compaq is currently adding 10 Proliant servers as compute servers, each with four CPUs and 4 Gbytes of memory, bringing the compute farm's total to 100 CPUs capable of running 100 Verilog jobs simultaneously. A 100-Mbit/s FDDI collapsed ring concentrator links the compute servers and a Compaq Proliant 5000 application server containing two Pentium Pro CPUs, 128 Mbytes of memory, and a 16-Gbyte disk.

Another Proliant 5000, with a 52-Gbyte disk, acts as a project data server. Compaq is adding redundant application and data servers to eliminate single points of failure. Though the compute farm includes both application and project data servers, we keep the executables (binaries) separate from the project data, storing executables and configuration files on the application server. The applications are executed via a UNC path and run on the compute servers.

The separation of executables and data offers a number of advantages. For administration purposes, for instance, different people have write access to the two sets of files and the separation simplifies access enforcement. The separation also helps to ensure reliability. Further, using a separate file server for project data keeps the smaller disks on each system from having to act as file servers and avoids disk-mounting issues. For simplicity, system administrators even set up the paths on the NT server as directories with the same names as on the Unix compute farm server.

The NT compute farm also follows the same memory guidelines that Compaq established for the Unix compute farm. Both types of compute farms have a shared memory model, but the target for each machine is 1 Gbyte of memory per CPU. For example, a three-CPU machine has 3 Gbytes of memory that one CPU can access or all three can share.

The guideline of 1 Gbyte of memory for each CPU is based on statistics that the EDA group collects on the average memory requirement for EDA jobs. For a long time the average was closer to 200 to 300 Mbytes per job, but average memory usage has now crept up to about 800 Mbytes per job. With a gigabyte of memory per processor, the system can run three average jobs simultaneously. However, trying to run a larger-than-average job plus two other jobs on a three-CPU system would only result in lots of swapping to disk and slow performance for all the jobs. Good scheduling software is of course essential for allocating resources based on memory usage, CPU requirements, software licenses, and many other factors.

Few engineering organizations can instantly switch from a pure Unix EDA environment to a pure NT environment. Changing from Unix to NT--or from NT to Unix, for that matter--therefore requires a migration plan that involves Unix and NT interoperability.

In our migration to NT, we're dealing with the complex interoperability issues between Unix and NT systems on the desktop and in compute farms. Our biggest interoperability issue is file sharing. So far, no single solution in the NFS-CIFS arena has met all our requirements, although the solutions are improving over time and already meet the requirements of users who employ simpler EDA systems. For us, the file-sharing issue simply places some limits on flexibility in the migration. Although the EDA group would like to be able to run any application on any platform, right now some of the design flows must remain entirely on either NT or Unix.

For a number of years, we've been using NT workstations running Hummingbird Exceed, an application that allows Windows users to access X applications transparently and provides such desktop features as the ability to copy and paste from one window to another. Exceed's tools for centralized management allow administrators to deploy, configure, and maintain the desktop environment remotely. Exceed's Jconfig tool, for example, provides administrative services such as application management and desktop configuration from any Java-enabled desktop on the network, and the NFS Maestro client allows network file and print sharing.

With our EDA migration to Windows NT still progressing, so far the project is clearly a success. We've successfully deployed an NT compute farm and made the transition from an all-Unix development staff to a staff of Unix and NT developers. Finally, cost-effective hardware has replaced existing, more expensive hardware, increasing computing capacity with fewer dollars.

For the past four years, Wendy Stresau was the systems development manager in the corporate CAD group at Compaq Computer Corp.'s Server Division in Houston. She was responsible for migrating Compaq's ASIC design environment from Unix (Solaris) to Windows NT. In January, she transferred to the Workstation Division, where she is a software development manager.

Ben Buzonas, a 10-year veteran at Compaq, manages the EDA organization for the enterprise computing group. He has more than 20 years' experience in electronic design and development and has held various responsibilities in ASIC design automation and design verification at Compaq.

To voice an opinion on this or any Integrated System Design article, please email your message to miker@isdmag.com.

integrated system design  April 1999