Can you measure success prior to completion?
I'm in the habit now of asking this question. And I get all kinds of reactions. I see some perplexed faces. Sometimes I get silence. Often, I'm laughed at. But I plod on because, you see, I'm going somewhere with this. I even go as far as telling engineering managers they can make more accurate predictions about project completion months before tape-out.
Who do I think I am?
I'm the senior VP of worldwide sales at Verisity Design and I'm not crazy. I've simply had the good fortune of seeing numerous customers successfully escape their predictability predicaments.
Oh sure, some engineering teams have a good high-level grasp of the fundamentals of budgeting and scheduling. But delivering on a spec in a measured, methodical way? That's a totally different story. Most are forced to cite success stories only in the past tense, if at all. I propose a sea change.
You must have your metrics.
Most IC testing teams have a good deal of experience and skill. And in fact, many project managers also have great confidence in their team, based in most cases, on decent track records.
What they don't have is a technology allowing them to more accurately predict and plan the process steps that lead toward closure. There is hope to significantly improve their experience. The solution lies in metrics: simple, succinct measures, or indicators, of a project's progress.
Overwhelmingly our customers have found that coverage is a good indicator of the quality of verification. Total coverage is a combination of functional, code, and assertion coverage, and when combined, monitored, and measured, it is a powerful metric by which to manage projects.
Guess what?
A customer of ours chose to try a metric-driven methodology when they began using our products. Immediately, their verification processes contributed to greater product quality and they were nailing their schedule goals with a much greater consistency.
Over an executive dinner recently, my engineering peer and I were discussing his most recent project, a multi-million-gate SoC. He told me the project was wrapping up.
He spoke with great enthusiasm of what had so far transpired in design and verification, but what knocked me out was what he had to say about the future. He said, "Our chip will come back from fabrication in three weeks and it will be right." It wasn't a guess. It was pretty much a promise he was very confident.
A measure of success.
Ironically, my customer was selling me on the benefits of a metric driven process. He explained how he learned to read metrics throughout the life of a project and how it enabled him to better anticipate the outcome.
He spoke about how his team, with the right set of tools, developed skills that enabled them to bring problematic issues to management early. He went as far as saying that creating a metrics-driven approach at his company had increased every contributor's ability to collaborate effectively.
Somewhere between our plates of pasta and a couple great cappuccinos, he also told me their weekly project review meeting was abstracted to a list of projects by division, with red, yellow, or green indicators. I challenged him by saying that it sounded overly simplified. His comeback: simplification was the biggest benefit of it all.
It's always exciting to hear a customer volunteer a success story so eagerly. This was a true gem. His company, I was told, is institutionalizing metric-based processes enterprise-wide. I was told it's now far easier than ever to report progress at the executive committee level. I was told the company now manages multiple divisions based on metrics. Then I was told that he'd get the check.
We finally disagreed.
Coby Hanoch is senior VP of worldwide sales at Verisity Design.