As the old axiom states, "you cannot improve what you cannot measure." A practical methodology for improving productivity requires first identifying actionable and relevant metrics to measure that do not burden the design process or the design team with excessive overhead. Furthermore, comparing multiple projects requires some type of normalization methodology to account for differences in complexity and scope. Factors to include when normalizing project analysis are design size, application area, process technology, frequency, and IP usage.

Possible measurements
Productivity metrics can be divided into two basic categories: design characteristics and resource utilization. Design characteristics provide insight into both the status of the IC design throughout the design process and the overall complexity of the chip. Resource utilization refers to computer and EDA tool resources as well as information about the personnel effort used during the project. To the extent possible, both basic categories of metrics should be captured in an automated fashion throughout the development cycle to minimize the impact on the design team. However, the recording of personnel effort typically poses the biggest challenge for most companies (the criticality of this information to meaningful productivity analysis will be discussed later in this article).

Automatic capture of many metrics is best done within the design environment. Figure 1 models how the data flow for metrics related to the implementation process could be conceptualized within a design environment. At each major step of the flow, the data for the metrics are extracted. Each time a task is invoked, metrics about task execution and the state of the design are put into a database. Each project has an isolated database, and the environment includes a mechanism for extracting information from each project database and storing a summary in a global database. Similar instrumentation of the flow can be established for verification metrics. Tables 1 and 2 respectively list some examples of metrics to be captured for both the implementation and the verification process.

In addition to compute and EDA tool resource utilization, the other major aspect of resource utilization that must be measured is the effort of the people working on the project. This information can provide some of the most interesting insights, yet is often overlooked. Few companies keep detailed records of the amount of time spent by each member of their development teams, aside from overall headcount records. (General records of project headcount typically provide imprecise measurements at best due to variances in work schedules and the fact that people usually enter and leave projects based on fluctuating, undocumented requirements.) However, with only a small amount of overhead, it is possible to capture detailed information about how design teams spend their time

To be most useful, staffing metrics must record the amount of time spent on the design project as well as the type of work completed by each person. This data can be used to understand issues on the current project and for planning future projects. To support use of the data across multiple projects, it is important to use a consistent time-capture process and template.

It is also important to capture the proper context for analyzing staffing data. Specifically, a set of standard project milestones (for example, the completion of the final netlist) should be used. Figure 2 shows an example of a post-project assessment with categorized effort data and a standard set of milestones. The height of each bar represents the amount of effort expended during a particular time period (in this case, each bar represents one week.) Each differently shaded segment of the vertical bars represents a task completed during the week. Project milestones are identified under the time axis to provide the proper context for analysis.

Reporting methods
Reporting methods Consulting organizations and development teams for defense and government applications often have business requirements for tracking the time spent by their engineers. In some cases, these requirements extend to the need to categorize that time by design task. Although many companies do not have these kinds of reporting requirements and consequently may not have easy access to this effort data, capturing this resource utilization data is vital to understanding and improving productivity. Three factors are critical to capturing this data successfully:

1. Define a low-overhead process. Design team members should be able to complete their weekly timesheet in less than five minutes.
2. Provide top-down support from management. This program must be viewed as a required and important task in the performance of the designer's job.
3. Clearly explain the rationale for collecting the data (for example, to analyze project bottlenecks and systemic business issues using only aggregated reports) to establish trust that the data will not be used to evaluate an individual's job performance.

Once metrics are captured in a database, project managers can use them in many ways. For example:

• To provide a snapshot of the health and status of the design (e.g., size, utilization, power dissipation, WNS, conditional coverage, etc.)
• To provide compute and tool resource utilization reports (e.g., number of CPU hours spent using a tool.)
• To show trends of the project's key indicators (e.g., TNS) over the life of the project (see example below.)
• To help identify bottlenecks (e.g., excessive route times, linkage of problems to a specific host CPU, non-convergence of coverage)

Investment in systems to measure and improve IC design productivity is often viewed as one of those "important but not urgent" activities that never seem to get started in many companies. Shorter product lifecycles place an increasing burden on design teams to focus 100% of their efforts on product development, with little or no time allotted for infrastructure improvements. Smaller companies that produce relatively few designs may find it particularly challenging to allocate time or resources, however small, to process improvement due to the uncertain ROI and because achieving tapeout on the immediate project is required for survival. Ironically, the economic drivers that compel companies to examine IC design productivity (e.g., time to market) can become impediments for embarking on such an initiative. For this reason, a productivity-improvement process such as the one described here requires both executive and design team commitment. The discipline that comes with a strong program management philosophy is necessary for consistent execution of the metrics capture process and analysis of the data. Finally, since the goal of productivity measurement is productivity improvement (not just reporting), it is vital that the metrics and analyses chosen provide pointers to actionable improvement plans.

Author's Biography:

Michael Solka is Director of Physical Design Methodology for Synopsys Professional Services. Michael has over 20 years of experience in the semiconductor industry in various engineering, marketing, and management roles. Michael holds a BSEE from The University of Texas and an MBA from The Wharton School.