R&D ROI
Comment
zeeglen
Yeah, I know, meaningless when applied to silicon design. But the managers ...
zeeglen
Tell your managers to tally "capacitors per hour". Use 0.001uf instead of 0.01 ...
Productivity and pornography
Ron Collett
10/29/2010 3:36 PM EDT
"We must increase our IC development productivity!" is the persistent invocation from semiconductor industry executives, and it's getting louder by the day. It's not surprising. Yet when I ask the question, "What do you mean by productivity?," executives and R&D managers often give vague answers. Few seem to have a firm grasp of the definition, other than saying "we need to finish projects on schedule, reduce development cycle times and use fewer engineers." As a characterization of the benefits of boosting productivity, it's not a bad start.
It reminds me of U.S. Supreme Court Justice Potter Stewart's comment in a landmark First Amendment case in which he described the challenge of defining pornography: "Pornography is hard to define, but I know it when I see it." Maybe semiconductor executives are saying the same thing: "Increased productivity is hard to define, but I know it when I see it." Justice Stewart subsequently recanted, concluding that pornography can be indeed defined. So too can productivity.
As a starting point, consider the U.S. Department of Commerce (DoC)'s definition of manufacturing productivity: "Output divided by Labor-input." Output is the difference between the widget's selling price and the cost of materials comprising it. Labor-input is the effort expended on manufacturing it (in person-hours). The dimensions of manufacturing productivity are therefore: Dollars per person-hour.
We can use a similar approach to quantify IC development productivity: Output/Labor-input. However, Output must be a calculation of the design's complexity as opposed to value-add. That's because complexity is a measure of the development team's output. For the moment, let's assume we can calculate it—a challenge, but one that's definitely achievable.
Labor-input, the denominator, is the total effort the IC development team spends on the project from start to finish. The start milestone is "begin concept definition." The finish milestone is release-to-volume production. Consistent accounting from one project to the next is a must. Also, it's more reliable to use full-time equivalents (FTE's) instead of counting person-hours, where one person working full-time for a week is a an FTE, or a "person-week."
Adapting the DoC's mfg. productivity definition to IC development productivity yields a productivity metric whose dimensions are: Total IC development effort expended divided into the chip's design complexity (i.e. "Complexity/Effort"). The key question of course is how to accurately calculate design complexity?
Ronald Collett is president and CEO of Numetrics, which provides fact-based project planning and benchmarking software that improves IC development productivity and schedule predictability.
It reminds me of U.S. Supreme Court Justice Potter Stewart's comment in a landmark First Amendment case in which he described the challenge of defining pornography: "Pornography is hard to define, but I know it when I see it." Maybe semiconductor executives are saying the same thing: "Increased productivity is hard to define, but I know it when I see it." Justice Stewart subsequently recanted, concluding that pornography can be indeed defined. So too can productivity.
As a starting point, consider the U.S. Department of Commerce (DoC)'s definition of manufacturing productivity: "Output divided by Labor-input." Output is the difference between the widget's selling price and the cost of materials comprising it. Labor-input is the effort expended on manufacturing it (in person-hours). The dimensions of manufacturing productivity are therefore: Dollars per person-hour.
We can use a similar approach to quantify IC development productivity: Output/Labor-input. However, Output must be a calculation of the design's complexity as opposed to value-add. That's because complexity is a measure of the development team's output. For the moment, let's assume we can calculate it—a challenge, but one that's definitely achievable.
Labor-input, the denominator, is the total effort the IC development team spends on the project from start to finish. The start milestone is "begin concept definition." The finish milestone is release-to-volume production. Consistent accounting from one project to the next is a must. Also, it's more reliable to use full-time equivalents (FTE's) instead of counting person-hours, where one person working full-time for a week is a an FTE, or a "person-week."
Adapting the DoC's mfg. productivity definition to IC development productivity yields a productivity metric whose dimensions are: Total IC development effort expended divided into the chip's design complexity (i.e. "Complexity/Effort"). The key question of course is how to accurately calculate design complexity?
Ronald Collett is president and CEO of Numetrics, which provides fact-based project planning and benchmarking software that improves IC development productivity and schedule predictability.
|
|
|
|
|
Navigate to related information
Mark Wehrmeister
10/29/2010 5:38 PM EDT
Design complexity is less important than the value to the customer. Presumably, a more complex design will have greater value to the customer and fetch a higher initial price. Instead of trying to measure design complexity, use the market price for the result of that design as a proxy in the calculation of productivity.
Sign in to Reply
RCollett
10/30/2010 6:05 PM EDT
Mark,
No doubt that value to the customer is the most important metric in business. And using a product's revenue (or profit) as a measure of Output is very useful. However, in most cases it yields an incomplete picture because revenue is the result of the Output of the entire enterprise, not just the R&D organization. For example, high revenue could be the result of powerful marketing and sales, as opposed to superior engineering. This actually is very common -- a strong sales/mktg. organization masks a weak R&D organization. Likewise, the revenue for a particular product line might be very low because of poor mktg/sales, but the productivity of the R&D organization could be very high. Thanks for your comment. Ron
Sign in to Reply
Luis Sanchez
10/29/2010 10:25 PM EDT
On schedule, within budget and less engineers.
That's simple isn't it?
But... we engineers like numbers so an equation is required for IC productivity.
To measure complexity... I bet there ought to be some efforts being done already trying to measure that. In some top rated university in the US.
Sign in to Reply
RCollett
10/30/2010 6:13 PM EDT
Luis,
One approach is to use a model that calculates the amount of effort the average development team in the particular industry segment would expend on that project. Then compare that effort value with the amount that your team actually spent. If you spent less, then you were more productive. If you spent more, then you were less productive. Since this isn't a forum for commercials, I'll direct you to www.numetrics.com where there is further info on this. Thanks for your comment. Ron
Sign in to Reply
Frank Eory
10/29/2010 10:34 PM EDT
This reminds me of stupid management metrics like "lines of code per hour" for software engineers and "gates per hour" for hardware engineers. It also reminds me of an old Dilbert cartoon in which managment was paying bonuses for verification engineers to discover bugs -- which quickly lead to collaboration between design & verificaton and a cartoon panel in which one of Dilbert's colleagues was coding a minivan worth of bugs!
Sign in to Reply
RCollett
10/30/2010 6:16 PM EDT
Frank,
I completely agree with you -- "lines of code" and "gates" or transistors, etc. are wholly inaccurate measures of output, and to use them invites misleading results and misguided behaviors. Thanks for your comment. Ron
Sign in to Reply
zeeglen
4/25/2011 8:38 PM EDT
Tell your managers to tally "capacitors per hour". Use 0.001uf instead of 0.01 and 0.1...
Sign in to Reply
zeeglen
4/25/2011 8:41 PM EDT
Yeah, I know, meaningless when applied to silicon design. But the managers don't know that.
Sign in to Reply
wilber_xbox
10/30/2010 7:32 AM EDT
attention grabbing title Ron! productivity is easier to define in manufacturing units as one need to simply check the end results but during research and development, i do not think the productivity is as simple. One way to check productivity can be through the return on investment as everything is about money in the end.
Sign in to Reply
RCollett
10/30/2010 6:30 PM EDT
Himansuhu,
Definitely want to use revenue (and profit) generated from the product -- to calculate the return on R&D. However, what if you have a really poor sales & mktg. and very productive R&D organization? If revenue is low, it appears that the return on R&D is low. Although financial measures that use revenue and profit can be very valuable, they can give a distorted picture. Metrics that, to the greatest extent possible, directly reflect the output of a particular organization provide the most insight when analyzing a business. Thanks for your comment. Ron
Sign in to Reply
yalanand
10/31/2010 2:11 PM EDT
True RCollett, I agree with you that financial measures are very deceptive while measuring productivity.
I have specifict question for you.Since I belong to service based company I want to know how do you measure the productivity for service based companies ?
Sign in to Reply
RCollett
11/1/2010 2:56 AM EDT
Yalanand,
For a particular service engagement:
(Total dollars billed for the Service - Total fully-burdened cost of delivery of that Service)/(Total person-hours expended on that engagement)
The numerator is the your firm's value-add. The denominator of course is the effort expended to deliver that value-add to the client.
Thanks for the comment.
Ron
Sign in to Reply
phoenixdave
10/31/2010 3:58 PM EDT
@yalanand:If your are based on a hourly service rate, your billable hours (within the established budget) would be a start.
Sign in to Reply
iniewski
11/1/2010 12:36 PM EDT
Rob,I think we agree that the final value of IC (or any other product) is determined by a customer (market). We also agree that this is has to do with Sales & Marketing effort to the same extent as R&D. Trying to separate R&D from that total is tough, if not impossible. What if my team designs very complicated, sophisticated product in half the time, a product that nobody wants! Is that R&D productivity? Kris
Sign in to Reply
RCollett
11/2/2010 11:25 AM EDT
Kris,
If your R&D organization uses half the effort that another organization would expend to design an equivalent product, then your organization's productivity is twice as high. If as you say, nobody wants to purchase the product, that's the fault of marketing and perhaps other parts of the enterprise, and it is wholly unrelated to the development productivity of the R&D organization (unless the R&D organization had exceptionally heavy influence on the product definition or other marketing-related tasks).
Rgds,
Ron
Sign in to Reply
prabhakar_deosthali
3/7/2011 5:39 AM EST
Like someone has commented, to be able to measure productivity of a person or a team we must be able to measure or estimate correctly the amount of work done. We must also be able to measure the complexity of the work done and then add appropriate weight-age to the amount based upon the complexity. The same number of lines of code written for an offline application is less complex than those written for a time critial real time application with a tight memory budget. If such metrics are evolved then only we can truly measure the productivity for a given development project team.
Sign in to Reply
RCollett
4/25/2011 7:42 PM EDT
Prabhaker,
You are correct. Our customers use an empirical model we developed. It calcuates the amount of effort the average team in the industry would spend on that project. It's quite accurate -- coefficient of determination (R-squared) is over 0.9 (i.e. Predicted Effort vs. Actual Effort). For more information, you can go to the blog entitled "How Complex is Your Design," which ran on 11/5/10, or alternatively, you can find a lot more information on our website (www.numetrics.com).
Rgds,
Ron
Sign in to Reply
ATUL SRIVASTAVA
3/7/2011 7:53 AM EST
Instead of a ratio it can be defined more as a percentile . The formula will be :Productivity=
(Current project data/ What is the last best data you have for similar project ) X 100
Data can be internal or from the industry sources and you generally factor for any differences due to complexity , geography etc .
This has been practically used by some companies .In one company certain groups are star performers and others are expected to reach upto start groups productivity level .In that case the productivity of star performer team is 100 percentile. This approach is quite practical.
Sign in to Reply