Editorial

Do Dueling EDA Tools Benefit Designers? Design contests provide a demonstratable value in real-world applications. by Jonah McLeod

What's the point of contests and benchmarks of design tools? Integrated System Design continues to stage these events, and I decided to question the wisdom of doing them. To answer the question, I asked myself, what is the purpose of tools? A design tool provides the user intellectual leverage by replacing one or more designers with a software program.

The next obvious question is how does a potential purchaser of this tool evaluate its worthiness? Today, buyers mostly take the tool on faith or the word of others who have experience using the tool. However, it seems to me that if a tool is to replace a real designer, it should meet some of the same expectations we have of the engineers we hire.

Engineers are expected to show some proficiency of their skills. New engineers entering the work-force have passed a rigorous course of instruction that involved testing in every engineering class. In each of these tests, students were judged against their peers in their ability to solve a set of problems.

It seems to me design contests and benchmarks that pit a number of different tools against one another is a trial for tools. The trial provides those observing the contest some sense of the tool's ability in a "real" design situation. Integrated System Design and Hewlett-Packard Co. (Palo Alto, CA) sponsored just such a contest at the Design Supercon on January 30th at the Santa Clara Convention Center (Santa Clara, CA).

In John Cooley's account of the contest in the April 1996 issue of Integrated System Design , the reader receives a blow by blow description of major happenings during the 2-hour contest (see our Internet site "www.isdmag.com" for a copy of the article "The Great ESDA Shootout").

The value of such a competition for design automation tool users is they see tools perform in a stressed situation similar to their working environment. Furthermore, in a world where time to evaluate individual tools is limited, such a venue allows users to evaluate a number of tools at once.

As you might imagine, the contest results were criticized by those who did not do well and extolled by those who excelled. In the aftermath of the contest, articles appearing in other publications called into question the results of the original contest.

Even contest judge John Cooley authored one of these articles, which appeared to reverse the outcome of the contest. He reran designs created by several of the tools that initially did poorly. He wrote that when he reran the designs with the Finite State Machine Compiler from Synopsys .com/isdweb/&lf=isd-sendtolog"> Synopsys Inc. (Mountain View, CA), they produced faster designs. In the SuperCon contest, only the first and third place winners used FSM.

The point is that the original contest results cannot be rewritten after the fact; to do so is a supreme act of marketing.

Here's the bottom line: designers must meet design windows of three to six months. They must deliver the fastest, densest integrated designs. Thus, they need tools that provide demonstratable value in real-world applications. Contests that stress tools ring more true than any vendor's marketing hype.

If you agree, let me know (e-mail jonah@asic.com), so we can sponsor more of these events. *

Jonah McLeod is editor-in-chief of Integrated System Design .

To voice an opinion on this or any Integrated System Design article, please e-mail your message to: michael@asic.com.

integrated system design  July 1996