special section

The technology that 21st Century Computer Corp. is developing is designed for mission-critical applications in computing and networking environments--in particular, military and commercial applications where fault-tolerant computing and networking capabilities are essential.

Extensive verification is necessary to test the clustering and internode communication capabilities of the system, as well as compliance with the specifications for the processor I/O bus (such as PCI or VME). This verification was accomplished on generic brand, Windows NT-based PCs using Verilog and the Synopsys VCS for Windows simulator.

Specifically, we're designing a hardware/software product that will let us place an ASIC in a server and cluster the central resources so that, in the event of a server or CPU breakdown, a redundant system can assume networking and computing responsibilities with no break in service. In our first application, we are developing a low-latency, high-bandwidth, reliable interconnect--which we call Scalable Fault-Tolerant Interconnect (SFTI)--for networking PC computing resources (see Figure 1). In this configuration, multiple PC systems will be clustered together to take advantage of key features such as internode communications, shared memory, and messaging. In the product, code-named Pegasus, the nodes will work together to provide the redundancy features required of a hardware fault-tolerant system.

Figure 1	The Scalable Fault-Tolerant Interconnect
The Scalable Fault-Tolerant Interconnect (SFTI) is used for reliable internode communication and messaging.

Application overview

Today, the most prevalent and pervasive examples of distributed enterprise computing are high-performance application servers with clustering software. However, they don't provide the same level of data integrity and high availability offered by proprietary fault-tolerant servers. Unfortunately, fault-tolerant servers are prohibitively expensive for most applications and thus play a limited role in distributed enterprise computing.

Pegasus will offer low-cost fault-tolerant technology. Consisting of an ASIC, dual cables, and software, it uses high-bandwidth, low-latency, synchronous communication between off-the-shelf hardware components to offer system-level fault tolerance. The ASIC uses SFTI hardware to provide on-demand, flexible application-specific clustering configurations to suit the customer's preference and budget.

Pegasus uses a standard processor I/O bus-based implementation of fault tolerance, making it independent of processor and operating system and transparent to the application and I/O module. In addition, SFTI provides fault-tolerant communications. SFTI can co-exist or be merged with emerging industry clustering standards and other hardware interconnects, as well as with clustering software from several vendors, including Microsoft's Wolfpack.

Dynamic reconfiguration

A unique feature of Pegasus technology not offered in traditional fault-tolerant solutions is its adaptive nature, which permits self-repair or dynamic reconfiguration. The latter allows load-sensitive configurations that include a cluster of single nodes, dual nodes, three or more nodes, or a dynamic combination of all three. As a result, it offers on-demand, application-specific, fault-tolerant capabilities for a range of applications from no hardware redundancy (maximum performance) to full fault tolerance (maximum availability), with other dynamic configurations in between.

Figure 2	Single-node test configuration
The single-node test configuration is a simulation vehicle for verifying some of the stand-alone core ASIC functions.

At the heart of the system is a 900,000-gate ASIC that will control the fault-tolerant capabilities for two to n servers. The ASIC design is based on an SFTI core that can accommodate any processor I/O bus in Unix and other OS-based computers, as required. To fully test the complex interactions of many servers, we are relying heavily on verification at the RT and gate level.

The verification environment

We looked at several simulators from different EDA vendors. We chose the VCS for Windows simulator because from a technical perspective we felt that it delivered acceptable performance with low to medium risk. Indeed, for us, as an economically challenged start-up, the business decision was a no-brainer. In addition, the lure of the successful development of a complex ASIC on an NT development platform had significant appeal. VCS for Windows was the only simulator that could run under NT and handle our large design. For the bulk of the RTL and gate level simulation, we used a 266-MHz Pentium II with 256 Mbytes of memory and other NT platforms that we had at various locations.

The design and verification team shared six VCS network licenses, served by a PC running under Windows NT 4.0 Server. Our Microsoft Visual Sourcesafe source-code-control repository resided on the same server. This setup allowed the design team to work not only at the office, but also at home. Files were check-in/out files from Sourcesafe, and simulations started by checking out a VCS network license over remote connections.

The remote connections consisted of a pool of modems and the Internet connectivity. To provide security for modem access, we used a remote-access server for user authentication and modem call-back. In addition, access was provided through the Internet using a firewall to prevent unauthorized access from the Internet and a Virtual Private Network (VPN) server to authenticate remote users and restrict user access from the Internet.

To allow for verification scripts, we used the MKS Toolkit for Windows NT. Combined with a log-in daemon, the design and verification team could log on to their work PCs and submit verification jobs remotely.

Figure 3	Processor I/O bus simulation configuration
A chip that bridges processor I/O buses provides two system bus interfaces for exercising the ASIC. Separate Verilog modules are used on each bus.

In addition to the NT systems, we run synthesis using Synopsys Design Compiler, as well as static timing analysis using Synopsys 's Motive, on a Unix platform. For connectivity with our Unix environment, we use Hummingbird Exceed and NFS Maestro Solo.

Simulation flow

Our simulation flow was to simulate using VCS for Windows from the command line, generate a VCD file, and then view the results using a waveform viewer. As required, we would also use the Fusion graphical user interface for interactive debugging.

Our first version of VCS for Windows was version 2.4.6, which, at the time, was a full revision behind the Unix version. We quickly migrated to VCS for Windows 3.1.4 beta to gain access to all the fixes and features previously added to the Unix product. More importantly, we also gained access to the vastly improved Fusion 2.0 GUI and the new Vwaves waveform viewer. A few months later, we migrated to VCS for Windows 4.0 beta and the Fusion/Vwaves 2.1 beta GUI to take advantage of further improvements in speed, capacity, and usability. These improvements became necessary as the design progressed because, when we started the project, the ASIC was going to contain only about 500,000 gates. After some market research and the subsequent addition of features, we topped at around 900,000 gates.

Compilation speed was the biggest issue with VCS for Windows, as native code generation isn't currently available. However, the availability of reliable incremental compilation minimized the impact on everyday simulations: VCS for Windows includes the command-line version of Microsoft's C/C++ compiler/linker, which translates the Verilog code into a simulation executable.

The initial PC workstations consisted of several 200-MHz Pentium Pro machines with 128 Mbytes of memory running under Windows NT 4.0 Workstation. Using those workstations in the later stages of development was exceptionally painful, as the simulation times grew as functionality approached the equivalent of 900,000 gates. That made it necessary to change the simulation workstations to 266-MHz Pentium II machines with 256 Mbytes of memory. Typical simulation runs dropped by a factor of three, greatly improving the design and verification cycles.

After we developed the Verilog code, we partitioned the major features in order to ease the burden on our verification resources. We separated them into the processor I/O bus interface and SFTI cores and clustering, fault-tolerant, and miscellaneous features. We also synthesized the RTL into gates as we verified each partition. That way we could verify at the gate level as we moved deeper into the design flow. We proceeded to use these partitions in three separate scenarios.

Test configuration

One test configuration used a single physical node to ensure that we had minimally correct behavior (see Figure 2), and we created a processor I/O bus functional simulation environment to validate the ASIC from the processor I/O bus interface (see Figure 3). Next, we verified a dual-node system to allow data packets to move from one node to the other and vice versa. The final and most complicated verification task was for an n-node configuration. Originally, we were going to verify a triple-node configuration, but we found that adding more nodes wasn't really a problem in terms of verification but more a function of computational power. Eventually we verified up to 15 nodes (at 900,000 gates each) at the RT level. The final verification task ensured compliance with the processor I/O bus specification.

We ran over 65 regression tests with the various configurations. In the two-node configuration, the simulation took about 3 hours. When we went up to the 15-node system, the simulation time grew to about 30 hours.

In the 15-node configuration, we were able to find latency problems that simply wouldn't have shown up in a two- or three-node system. At this time, we've run some verification at the gate level, but because of the increased complexity, we've only performed preliminary gate-level simulations of up to two nodes.

Though most of the design for the first product is complete, a fair amount of verification is pending. In addition to design verification, we wanted to prove that it was possible to run large, complex simulations on NT-based PCs. Some of the functions that are coming out in future generations of NT are directed to networking applications, which only makes the choice of NT easier. For start-ups with tight budgets or even established companies, we believe that the NT-based EDA systems and software are a great way to accomplish simple and complex design development and verification tasks.

Martin Blitz is the vice president for advanced development and system architect at 21st Century Computer Corp. in Ashland, Mass. He has 13 years' experience in advanced hardware, software, and system development. Previously, he worked at Digital Equipment and Kendall Square Research.

Arthur Harris is a consulting engineer and design engineer for 21st Century Computer. He has over 19 years of research and development experience, including work at Digital Equipment and Hewlett-Packard.

Pedro Mora is the ASIC implementation manager at 21st Century Computer. He has over 13 years of experience in ASIC technology, design, and testability. Previously, he worked at Viewlogic Systems and Digital Equipment.

Leo Salazar is the company's ASIC verification manager, technology strategist, and system engineer. His more than 16 years in advanced product development includes work at Tandem Computer, Digital Equipment, and Hughes Aircraft.

To voice an opinion on this or any Integrated System Design article, please email your message to miker@isdmag.com.

integrated system design  May 1998