Agile hardware development – nonsense or necessity?

Hardware developers tend to see software development as a foreign land with odd people, languages, tools and techniques. Agile development approaches seem just as odd to most of us even though, according to sources like Forrester Research, they are becoming mainstream in software development. While software developers have largely accepted the merits of agile development and commonly debate the value of one agile practice against another, there is no such acceptance nor debate in hardware circles.

Should there be debate when it comes to applying agile in hardware development? Might the values and principles that guide agile software teams similarly guide SoC teams; or are the differences between these two disciplines too great?

With it being relatively unknown in hardware development, it is difficult for the average hardware developer to offer an informed opinion on agile. Complicating matters is the fact that a true and tangible definition of agile can be difficult to pin down. Even in software circles where it’s been used for years, agile can have different meanings, bring varying degrees of success and carry with it different baggage. Fortunately, hardware developers can avoid boxing themselves into the opinions or misconceptions of others by first looking to the manifesto for agile software development, then to practices that have made agile software teams successful for more than a decade.

Software development is a creative process
The manifesto for agile software development was written by a group of software developers in 2001. At a weekend retreat in Utah, this diverse group of individuals advocating differing approaches to software development met to talk about the current state of software development. Despite their differences, the group agreed that the top-down, process centric, heavily documented, painfully deliberate style that many organizations followed was proving less and less successful. As Jim Highsmith writes on www.agilemanifesto.org, everyone agreed that “In order to succeed in the new economy… companies have to rid themselves of their Dilbert manifestations of make-work and arcane policies”.

Intense discussion from two days of meetings resulted in the creation of the manifesto for agile software development. To its signatories, the manifesto was a launching point from which they and other likeminded developers could unravel the damage done to software development by classic project management techniques and specifically, the waterfall development model.

The underlying premise of the waterfall model is that design and construction of some product can be defined in advance through construction of a detailed project plan. Development then becomes a largely mechanical process of staged execution according to the plan.

A fundamental flaw in applying the waterfall model to software development, say agile software practitioners, is that the act of designing and constructing software cannot be reliably defined in advance. Software development, they argue, is a creative process and due to its inherent uncertainty cannot be accurately planned in advance; product vision, customer need, target market, target technology and team dynamics among other things can vary considerably and unpredictably during development. For a creative process like software development, therefore, it is agility and adaptability that drives success rates to a greater degree than the intense upfront planning characteristic of waterfall development.

Is hardware development a creative or defined process?
Before jumping into an analysis of the applicability of agile in hardware development, it is important to first categorize developing hardware as either a creative or defined process. Presumably, hardware development as a creative process would be similarly crippled by traditional project management practices and similarly benefit through the application of agile development. If, on the other hand, one decides hardware development can truly be captured as a defined process in an accurate and reliable project plan, there is little point in going further because the problems agile is designed to address simply do not exist.

There are several questions that can help characterize hardware development as either a creative or defined process.

• Is it possible to build a detailed project plan that accurately captures a complete
• development cycle prior to development?
• Do detailed project plans remain stable over time?
• Is it possible to build a concise set of product requirements that will satisfy customer and
• market need 6, 12 or 18 months in the future?
• Do product requirements remain stable over time?
• Are products consistently delivered on schedule?
• Do products consistently meet market demand and customer need?
• Are target technologies thoroughly understood and unlikely to change?
• Would a hardware team design and build a product the same way twice?
• Are product architectures likely to remain stable over time?
• Are people and their respective skill-sets interchangeable?

If you find yourself answering Yes to these questions then, luckily, you have the stability necessary for defined process. Your job is a relatively easy one. Many others, however, will answer these questions with an emphatic No! Many teams repeatedly underachieve with intense up-front project because accurate planning for them is simply not possible. For those teams, it is their level of agility and their capacity to harness instability–not deflect it–that determines success.

Agile values in software development (The items on the left)
First and foremost, the manifesto for agile software development is comprised of four core values. These values are presented as a dichotomy where waterfall development legacy–known as the items on the right–is contrasted directly with the characteristics that have come to embody agile development–the items on the left.


Individuals and interactions
Software development is carried out by people complemented by process and tools, not the other way around. The writers of the manifesto agree that companies erroneously link success to efficiency brought by process and tools as opposed to the innovation and creativity brought by people and teams. The manifesto encourages organizations to see people and communication as the critical factors in building great teams and great software, with process and tools being complimentary.

Working software
A common saying in agile software development is I’ll know it when I see it meaning the insight that planning and product documentation offers will always pale in comparison to the insight gained by seeing the product itself. To that end, the manifesto emphasizes the creation of working software as soon as possible, then demonstrating it and incrementally adding features throughout development. Working software gives the team, stakeholders and customers regular opportunities to objectively measure progress and offer feedback based on a working product.

Customer collaboration
The primary goal of any for-profit organization to make a profit. To do that, one needs customers; happy, repeat customers are ideal. The manifesto encourages regular customer collaboration as the only practical way to gain and maintain a clear understanding of customer value as it inevitably evolves over time. Products are designed to please customers so it is customers that are best equipped to lead development.

Responding to change
For many, responding to change is the core value of an agile team. Software development is a creative process where it is impossible to predict, understand and plan with pinpoint accuracy over an extended period of time. In most cases, long range planning becomes a futile attempt to gain confidence and control over the development process. The authors of the manifesto argue absolute confidence and complete control are impossible to achieve. Customer needs, requirements, team dynamics and projects goals will change over time turning long range plans obsolete. Agile teams pride themselves on successfully absorbing change by combining detailed short team planning with long range estimates.

The items on the right
The final lines of the manifesto are as follows:

“…while there is value in the items on 
the right, we value the items on the left more.”
(www.agilemanifesto.org)

What are the items on the right? They are the characteristics that always seem to represent due diligence and professional practice but have repeatedly lead to project failure, dissatisfied customers and disgruntled employees in software development when allowed to dominate the project environment.

The items on the right are:

• processes and tools;
• comprehensive documentation;
• contract negotiation; and
• following a plan.

It is important to note that the manifesto does not ignore the items on the right; it instead suggests the role they play should be lessened. The authors believe the items on the right can positively impact development, but only as part of an agile approach that first emphasizes individuals and interactions, working software, customer collaboration and responding to change.

Bringing Agile values to hardware development
Arguably, there is a heavy preference for the items on the right in hardware development. Hardware development teams rely heavily on tools. We strive for process definition and comprehensive documentation–especially in ASIC development. Requirements are normally hammered out during contract negotiation and change to those requirements–aka. feature creep–is deflected whenever possible.

For anyone that views hardware development as being a creative process, it is hard to deny the values of the manifesto are directly applicable. But embracing a set of values is obviously not enough. At some point, abstract values have to translate into practice. Luckily, software teams have created a number of practices that embody agile values, many of which can be ported directly to hardware development.

Pair programming and co-located teams are very visible and effective ways agile teams emphasize the value individuals and interactions; neither are common in hardware development even though they are directly portable and would presumably produce similar benefits. Pair programming can be trialed by a team with very little risk or impact on current responsibilities. Similarly, removing cube walls and physically bringing developers together poses very little risk regardless of circumstance. The results of using either are tighter teamwork and more effective face-to-face communication.


Iterative development, where products are built and demonstrated over a number of regular intervals, is the primary means by which teams realize the value of working software. Hardware demonstrated through simulation, virtual models, prototypes or emulation platforms can be used to similarly demonstrate products, quantify progress and gather feedback. Test-driven development (TDD) is another practice that helps teams focus on building working hardware and less on simply writing code. While iterative development and TDD are potentially disruptive for hardware teams, the benefits of objectively viewing progress with tested code are hard to ignore. This is especially true early in development when the value of documentation and untested code can be difficult to quantify.

Iterative development and regular product demonstrations are also an effective way to involve customers thereby ensuring progress is indeed progress in the right direction. In a step further, some agile teams embed their customer within the actual development team so they are available at all times for questions and clarifications. For some, embedding a customer within a hardware development team would no doubt be controversial. But regardless of the perceived problems, increased collaboration and transparency with customers is a reliable way to secure trust and long term success in the vendor/customer partnership.

Continuous integration (CI) systems and automated test suites are key to responding to change. CI systems and test suites are put in place early and used immediately in development. Code is written in small batches, integrated regularly and regressed continuously. While automated testing is already prevalent in hardware development, automated CI systems and regular small batch integrations–every few hours–is less common. An automated CI system is a tool that can help hardware teams significantly decrease design and test cycles and maintain a working code base.

One other practice of particular note is the practice of early and continuous deployment to customers. For many agile software teams continuous deployment is absolutely critical to their success. Unfortunately, its applicability in hardware development–or lack thereof–tends to be used to discredit agile altogether. For ASIC development in particular, continuous deployment is obviously not realistic; but one practice not being realistic should not be used to dismiss the entire suite of practices. No agile software team perfectly characterizes agile development; no hardware team could be expected to do so either.

Nonsense or necessity?
Agile software practitioners have found that success depends on treating product software development as more than an impersonal, pre-defined, assembly line process where individuals toss code from cubicle to cubicle. The same holds true for hardware development. On a fundamental level, product development–software and hardware–relies on creative problem solving that cannot be shoehorned into a defined process. Successful products evolve over time and require real collaboration within a team that is constantly synchronized with customer need; developers must understand the context of their efforts and the business they are a part of.

While there are obvious differences between software development and hardware development, there are also significant similarities. The key for hardware developers is to resist getting caught up with the differences and to instead focus on the similarities. Doing so makes it hard to argue that the values of the agile manifesto and agile practices could not be used to achieve the same benefits that software developers have been realizing for years.

Change happens in hardware development. Regardless of whether its due to changing specifications, employee turnover, team dynamics or new technology there is no avoiding it. The teams that drop their “Dilbert manifestations of make-work and arcane policies” for a better rounded, refocused and more humane view of hardware development captured by the manifesto will turn that change into an advantage. Teams that don’t will be left to complete the futile exercise of fitting a square peg–defined process–into the round hole that is modern day hardware development.

Bio:
Neil Johnson has been working in ASIC and FPGA development for more than 10 years. He currently holds the position of Principal Consultant at XtremeEDA Corp, a design services firm specializing in all aspects of ASIC and FPGA development. Neil is also co-moderator for AgileSoC.com, a site dedicated to the introduction of Agile development methods to the world of hardware development.

---

If you found this article to be of interest, visit EDA Designline where you will find the latest and greatest design, technology, product, and news articles with regard to all aspects of Electronic Design Automation (EDA).

Also, you can obtain a highlights update delivered directly to your inbox by signing up for the EDA Designline weekly newsletter – just Click Here to request this newsletter using the Manage Newsletters tab (if you aren't already a member you'll be asked to register, but it's free and painless so don't let that stop you [grin]).