Design Article
How I write software
Jack Crenshaw
6/1/2010 12:00 AM EDT
In the two decades or so that I've been writing this column, we've covered a lot of topics, ranging from the best implementation of abs(x) to CRC algorithms to logic theory, Karnaugh maps, and electronic gates, to vector and matrix calculus to Fourier and z-transforms to control theory to quaternions, and lots more. I even wrote about whether or not the ancient Egyptians really built the values of π and Φ into the Great Pyramid (they did. And 
, too).
Almost all of the topics involved math, but they also required software, either for embedded systems or just to illustrate a given algorithm. But I've never talked much about the way I write software. I thought the issue of my own personal software development methodology would be, at best, a distraction from the topic at hand and, at worst, a subject of great hilarity.
I've always known that the way I write software is different from virtually everyone else on the planet. Is this true for everyone? I don't have a clue.
That's why I thought it would be fun to share with you the way I write software. If you do it the same way, that's great. It would be nice to know that there are other folks who share my approaches. On the other hand, if it's not the same way you do it, we have three possible outcomes. Either you'll learn from me, and decide that some of my approaches have merit; I'll learn from you, and decide that I've been doing it wrong all these years; or my revelations will serve as a subject of great hilarity. Any of those outcomes, I submit, would be good. It's a win-win-win situation.
The environment
Let's begin with the software development environment. In my time, I've used some really rotten development environments. I've worked with systems whose only I/O device was a Teletype ASR-33. We got so tired of emptying the chad-catcher on the paper tape punch, we just let it flutter to the floor. (At the end of the day, guess who got to sweep it up.)
I worked on one program where my only "compiler" was a line-by-line assembler (meaning that it didn't accept mnemonic names, only absolute addresses). The bulk storage device was an audio cassette. Just so we're clear, this wasn't a homebrew system I cobbled up out of old Intel 4004s. paper clips, and chewing gum. It was a system built up by serious, if boneheaded, managers in a very large corporation.
Perhaps because of these nightmare environments, I really appreciate a good one. I especially love using integrated development environments (IDEs), in which all the tools are interconnected, preferably by a lovely and fast GUI interface. I know, I know, real men don't use GUIs or IDEs. Some folks much prefer to use only command-line interfaces. and to write ever more complex and inscrutable makefiles.
If that's you, more power to you. Whatever works. But I can't hide my view that such an approach is an affectation, involved more with earning one's credentials as a software guru than on producing good code. I'm sure that there are good, valid reasons to write a makefile rather than let the IDE do it for you. I just can't think of one, at the moment.
So what do I do when I'm told that I must use the environment the project managers already picked out? First of all, I'd ask them (always discretely, of course, in my usual, ultra-tactful manner) why on earth they chose the development environment without discussing it with those of us who had to use it. Second, I'd campaign mightily for an environment more suited to my approaches.
As an aside, I used to teach a short course in the development process for embedded microprocessors. I told the management folks that they should plan to give their software folks every possible computer tool available--hang the expense. I pointed out that software development is very labor-intensive, so the front-end cost of good tools would be quickly made up by increased programmer productivity.
Needless to say, sometimes I'm successful in changing the managers' minds. More often, I'm not. So what do I do then? I make do as best I can, but I discreetly tweak the system and my approach to make the environment behave more like an IDE. For example, I might add script files, hot keys, etc., so that pressing a single button would invoke one tool from another.
|
|
|
|
|
ukzw
6/1/2010 8:33 AM EDT
Hi Jack,
I read your paper with smile: what a surprise, the way I write software is exactly the same as you. Possibly this is due to we have one thing in common. I'm a practitioner have my hands dirty for 20 years, start from 8051 assembly, later move on to C51, PIC and ARM.
Maybe our way of write software is a bit old fashioned. But customers won't care how we write, all they want is the final product reliable and bug-free. I'm happy I'm still get hired, this partly proves this "old" fashion is a good fashion.
Enjoy writing,
Zhe
Sign in to Reply
Lyeal
6/1/2010 1:40 PM EDT
Jack,
You are a guy after my own heart. Starting with assembly code for 8080's, 8085's, 6802's, and 6809's and later some C and C++ was the way I did embedded programming. I relate well to your comment on Turbo Pascal--it was wonderful. I did modules, not as small as yours but generally for specific functions, even when doing assembly language programs. There was a similarity to my modules and the units in Turbo Pascal. I did the top down, with some bottom up, and maybe your external-internal approaches. Each module was tested extensively, and therefore integrated well at the end. Maybe the fact that I was fundamentally a hardware designer warped my mind a little, as I expected each subsection (unit, function) to actually be known to work as early on as possible. This was my ego trip of "that's good, that works, now let's go on." My very first programming was on an IBM 1401 with 8K bytes of memory and done in SPS-2, basically assembly language and this was done in 1964, so you know how that dates me. Thanks for your great perception of things when you write your articles.
Sign in to Reply
JackCrens
6/1/2010 3:36 PM EDT
Lveal, I think you put your finger right on a couple of very good points. First, I think having written in assembly language is a big plus. I sure wouldn't want to force everyone to have to do the same, but I do think it the experience gives one a very specific view on what's going on inside the hardware, that's hard to get otherwise.
Even more importantly, I think understanding hardware design is also a plus. A hardware designer truly understands the concept of "black box," to the extent that it's an ingrained instinct.
When I was about 10, I got interested in ham radio, so decided to build my own receiver from a schematic I found. I went to the store, bought all the parts, and soldered them together (with acid-core solder, no less).
Guess what? It didn't work. That's where my project ended, because I had no idea what was wrong. I had found the schematic, no problem, but I had not the slightest clue what all those parts were supposed to be doing.
Fortunately, I got smarter with age. Today, I'd build a certain circuit, say an oscillator. Then I'd test it with scope, etc., to make sure it was osc-ing, and at the right frequency.
My pal Peter Stark designed a computer kit that worked that way. The first circuit elements consisted of the CPU, a clock crystal, and an LED indicator light (poor man's scope). He tied the data lines to ground, so as the CPU ran, it would cycle through the address space.
That's my kind of circuitry, and the concept can and should carry over into software.
Jack
Sign in to Reply
KarlS
6/2/2010 10:20 AM EDT
Really like the article. We need to apply similar approach to the entire design. Some of the things I see: Compile time for the hardware is too long to allow for iterations. Synthesis is too eager to throw logic away so that the design has to be essentially complete before anything can be simulated. Compilers optimize away code intended to control the hardware interface, There is no easy way to simulate/single step the hardware interface function. Context switching has to empty then refill the stack. I have some ways to improve some of this, but a one man band does not have much of a chance. By the way test driven design seems to have a similar approach to yours. Thanks, Karl
Sign in to Reply
slabs
6/3/2010 3:43 AM EDT
I suspect that most people of jacks vintage and experience do the same. It is all a question of what one has been exposed to. I certainly could have written a very similar story and I still work in the way he describes. I learnt my programming more than 20 years a go on an oil rig where we designed the system as we went along and testing was running the code written in the last hour for real. After we knew how it worked we wrote flow diagrams, and sub routine call trees by hand on bits of paper in order to convince our selves we knew what we were doing. I still use this type of code/design iteration technique today. Most certainly those of us that remember have to boot a system by programming the boot code by toggle switches really appreciate using an IDE. Why make programming harder. Long live KISS.
Sign in to Reply
rahul_lav
6/3/2010 4:25 AM EDT
Nice article,
I very much agree with you about the water fall or to say any process model. Typically when you are supposed to start your "designing" or "coding" many requirements are TBD(to be decided) and then you can either play poker or use a "Practical process model", you suggested.
Sign in to Reply
pteryx
6/3/2010 5:22 AM EDT
Really great article, thanks. I admit, that I don't like the idea of having everything "as modular as possible", it depends on the situation. When you write PLC-like state programs, then fragmenting something to blocks just for having it divided to blocks is possible, but the readability of such code suffers. Whatever is functional is beautiful.
Sign in to Reply
GordonScott
6/3/2010 5:36 AM EDT
Good article.
My methods are pretty similar, though I don't test quite so much so low. I do program fairly defensively, though. sizeof() used extensively, for example.
I too work `outside in', though I usually start with a pencil sketch (literally!) of what I think the system/dataflow/etc. is likely to look like. Then I'll move on to outline the boundaries .. the user interfaces and the hardware interfaces and so on. I'll choose the likely operating method .. event-driven or RTOS, then I'll start nailing up a top-level skeleton, which I'll use to start building and testing the low-level drivers. By the time I'm filling in the middle, usually my internal data structures are gelling nicely.
Small is good. If speed's an issue, well, that's what `inline' is for :-)
One place where I may differ from you is that since the early 80s, I've liked messaging as a means of intercomunication. I don't much mind if the messages are a parameter or a message string, but I do like to be able to take my modularity to hardware if I wish. As an example, in a new derivative of an existing product, we just ran out of I/O in a big way (for the scale of product, anyway). No problem .. I just pass the messages down to a new backend CPU that handles the new I/O and indeed takes over some of the old I/O. Job nearly done and everything outside the box looks pretty much the same as the previous version.
Sign in to Reply
KarlS
6/3/2010 10:32 AM EDT
These methods also worked for me back in the 80's and 70's, too. I have never understood the appeal of HDL and all the emphasis of synthesis/compiler optimization up front. Oh well, back to the method, An IDE that can use a block diagram for design entry so it is pretty easy to put together a data flow to keep a graphical image of what connects to what. It is easy to iterate as the design gels.
There can be a block diagram (flow diagram?) for the program, hardware, etc. Then if we look at either there is data manipulation which uses bit-wise operators that is controlled by logical and relational operators. Arithmetic expressions can be used to concisely define the data flow. Boolean expressions can be used to describe the control conditions.
Now it is a straight forward task to write a program to evaluate the expressions and in fact single step the entire function. There is no need to write, compile, synthesize HDL so the iteration time is short.
Now that there is a workable design methodology sketched out, what are the next steps?
Sign in to Reply
Weatherhead
6/3/2010 5:29 PM EDT
Jack you rule! Spot on buddy. I loved Turbo Pascal (along with the object oriented extensions) back in the day too (which was around '92 for me) although it has been a long time now I do recall that the Borland IDE and compiler running on my meager 33MHz 486 with an amber monochrome display was absolutely rocking. In fact I originally learned C in about 1 week from a little book called "C for Pascal Programmers" as I recall. Anyway, you are right on the software approach as most of these structured approaches to development actually don't work if they are followed precisely. Those of us who have been coding since before it was a fashionable profession ('85 for me) know that these systems setup to manage large teams of engineers almost never work correctly and that with the exception of what I would term "applications programming" (which I arrogantly classify as a 2nd tier profession) 10% of the players contribute 90% of the actual output. But don't tell anyone, while management is busy selling each other on the newest development acronym, those of us who actually know what we are doing can do 5x the work in 1/5th the time while we are waiting for the next phase gate or whatever. For the other 4 days a week there are pointless meetings, confused coworkers who somehow graduated from college without understanding how to write a complete sentence, the water cooler, 2 hour lunches and solitaire.
Sign in to Reply
DAH2136
6/4/2010 1:51 PM EDT
Excellent. Note that "lint" is now depreciated in the gcc world, instead you're supposed to just run the compiler with all warnings on.
Note also that you may be forced to flesh out the "drivers" because your hardware colleages are needing an Acceptance Test Plan to validate their latest spin of a PCB. The test code is often evolved into the driver. ATPs are needed for regulated applications (mine is medical) and for manufacturing control.
Sign in to Reply
antiquus
6/4/2010 7:47 PM EDT
This is a great description of a great method; I've used a similar approach for years.
In my last job we used Agile, and found that we had to alternate between SW and HDL development. You need SW to test HDL, and HDL to test SW, so we iterated in a big way.
If you like fast compiles, you should check out the Microsoft Visual tool sets. The compile (re)happens as you type, the list of error messages is constantly in flux, and error points are highlighted like spellcheck. What else should that 3GHz Pentium do between editor keystrokes? I'm waiting for those features to move into the embedded IDEs.
Sign in to Reply
KarlS
6/5/2010 10:18 AM EDT
This discussion is getting down to the real world, Thank Goodness! I use MS C# Express because I like the syntax, but C++ should work too.
Compile the C code on C#, get the error check, speed, and intellisense for free. Now you have clean code to compile on the embedded IDE.
The HDL compile time should be shortened by just generating the functional netlist without synthesis optimization, fitting, and placement. After all the objective at this point is to get the logic right. The physical build comes later. Another approach is to write an HDL simulator that does not require RTL. I did a prototype for Verilog so it is feasable, I just cannot sell it.
Would it be nice to have a tool that takes C code and HDL as input and has the speed of MS Visual Studio?
Sign in to Reply
willc2010
6/7/2010 11:18 PM EDT
Does the author really mean that his largest unit of program composition is a small function? Any project other than a very simple one must have higher levels than that in order to be manageable. A program with some hundreds of thousands of lines of code, say, is likely to be composed of a number of subsystems, each of which in turn is structured as a package hierarchy. So in a realistic sense a 'module' could easily consist of tens of thousands of lines of code. This is obviously not the same thing as saying that any particular blocks of code are tens of thousands of lines long.
When developing a system of significant complexity, it is necessary to decompose it at a high level and to define in reasonable detail the purposes and interfaces of the modules, so that engineers have some definite structure to work to. This requires some thought up front, before everybody starts to type in 'hello world' routines. Call it waterfall if you like, but having everybody just leap in and hack out code as the very first step is a recipe for chaos. There will generally need to be revision and iteration at all levels during the development of the system, and excessive rigidity will, I agree, make this difficult and inefficient. But it doesn't follow that we should adopt the attitudes of sophomores. I don't mean that the author recommends that approach, but some might read it that way. It's not clear to me at least whether he's advocating law 43 or discrediting it.
Sign in to Reply
Bakhruddin
6/8/2010 8:47 AM EDT
Great article; you described my style to a "Tee". Loved your comments about punched tape - remember the times when the tape missed the bucket and you would have to scoop it off the floor are refold it. On those occasions, I was really glad my coding had to fit into 2K EPROM!
I, too, write those one or two line general utility functions. Guess I'm lazy cause a lot of them are out there in libraries. But trying to find exactly the right one for the tasks at hand can be more work then rolling your own!
Maybe programming style really has a lot to do with your early applications. Like you, mine were low level hardware control programs, so I nearlly always started with the drivers and IO. Hearing relays clank away and watching led's flash during testing really gave me the warm fuzzies.
Ah, the good old days. Now I'm so far away from the hardware that it could crash and burn and my program wouldn't know it for days.
- Brian
Sign in to Reply
ESD editorial staff: SRambo
6/8/2010 1:47 PM EDT
More reader comments (originally posted on other sites)
Hi Jack,
I read your paper with smile: what a surprise, the way I write software is exactly the same as you. Possibly this is due to we have one thing in common. I'm a practitioner have my hands dirty for 20 years, start from 8051 assembly, later move on to C51, PIC and ARM.
Maybe our way of write software is a bit old fashioned. But customers won't care how we write, all they want is the final product reliable and bug-free. I'm happy I'm still get hired, this partly proves this "old" fashion is a good fashion.
Enjoy writing,
--Zhe (ukzw)
Jack,
You are a guy after my own heart. Starting with assembly code for 8080's, 8085's, 6802's, and 6809's and later some C and C++ was the way I did embedded programming. I relate well to your comment on Turbo Pascal--it was wonderful. I did modules, not as small as yours but generally for specific functions, even when doing assembly language programs. There was a similarity to my modules and the units in Turbo Pascal. I did the top down, with some bottom up, and maybe your external-internal approaches. Each module was tested extensively, and therefore integrated well at the end. Maybe the fact that I was fundamentally a hardware designer warped my mind a little, as I expected each subsection (unit, function) to actually be known to work as early on as possible. This was my ego trip of "that's good, that works, now let's go on." My very first programming was on an IBM 1401 with 8K bytes of memory and done in SPS-2, basically assembly language and this was done in 1964, so you know how that dates me. Thanks for your great perception of things when you write your articles.
--Lyeal
Lyeal, I think you put your finger right on a couple of very good points. First, I think having written in assembly language is a big plus. I sure wouldn't want to force everyone to have to do the same, but I do think it the experience gives one a very specific view on what's going on inside the hardware, that's hard to get otherwise.
Even more importantly, I think understanding hardware design is also a plus. A hardware designer truly understands the concept of "black box," to the extent that it's an ingrained instinct.
When I was about 10, I got interested in ham radio, so decided to build my own receiver from a schematic I found. I went to the store, bought all the parts, and soldered them together (with acid-core solder, no less).
Guess what? It didn't work. That's where my project ended, because I had no idea what was wrong. I had found the schematic, no problem, but I had not the slightest clue what all those parts were supposed to be doing.
Fortunately, I got smarter with age. Today, I'd build a certain circuit, say an oscillator. Then I'd test it with scope, etc., to make sure it was osc-ing, and at the right frequency.
My pal Peter Stark designed a computer kit that worked that way. The first circuit elements consisted of the CPU, a clock crystal, and an LED indicator light (poor man's scope). He tied the data lines to ground, so as the CPU ran, it would cycle through the address space.
That's my kind of circuitry, and the concept can and should carry over into software.
--Jack Crenshaw
Sign in to Reply
ESD editorial staff: SRambo
6/8/2010 1:48 PM EDT
More reader comments (originally posted on other sites)
Great article; you described my style to a "Tee". Loved your comments about punched tape - remember the times when the tape missed the bucket and you would have to scoop it off the floor are refold it. On those occasions, I was really glad my coding had to fit into 2K EPROM!
I, too, write those one or two line general utility functions. Guess I'm lazy cause a lot of them are out there in libraries. But trying to find exactly the right one for the tasks at hand can be more work then rolling your own!
Maybe programming style really has a lot to do with your early applications. Like you, mine were low level hardware control programs, so I nearlly always started with the drivers and IO. Hearing relays clank away and watching led's flash during testing really gave me the warm fuzzies.
Ah, the good old days. Now I'm so far away from the hardware that it could crash and burn and my program wouldn't know it for days.
--Brian (Bakhruddin)
I suspect that most people of jacks vintage and experience do the same. It is all a question of what one has been exposed to. I certainly could have written a very similar story and I still work in the way he describes. I learnt my programming more than 20 years a go on an oil rig where we designed the system as we went along and testing was running the code written in the last hour for real. After we knew how it worked we wrote flow diagrams, and sub routine call trees by hand on bits of paper in order to convince our selves we knew what we were doing. I still use this type of code/design iteration technique today. Most certainly those of us that remember have to boot a system by programming the boot code by toggle switches really appreciate using an IDE. Why make programming harder. Long live KISS.
--slabs
Really great article, thanks. I admit, that I don't like the idea of having everything "as modular as possible", it depends on the situation. When you write PLC-like state programs, then fragmenting something to blocks just for having it divided to blocks is possible, but the readability of such code suffers. Whatever is functional is beautiful.
--pteryx
Sign in to Reply
VeerYam
8/16/2010 7:35 AM EDT
A couple of comments with due apologies if they do not sound positive
1. Way back in the 1984's, atleast at the place where I worked, compiling was an expensive use of resource - firstly, due to licencing cost compilers were installed in only one build machine. Secondly, the compilation times were quite high, and the cycle time if one waited for compilers to throw up all the errors were pretty high. It did make sense to restrict programmers to run a compilation after some level of due diligence and high level of confidence in the code
2. The recommendation or rather practice to single step every line of instruction or statement doesn't go well with my line of thinking. A well written code should be able to be run over the programmers or the reviewers memory with same or higher reasonable speed without having to check them line by line. One gets to single step only if you don't understand your code, or if you find something different to what you comprehend to happen. For single liners such as check for comma, I would like them to be single stepped in the mind while they are being typed. I would think the approach suggested would just be unproductive in my kind of environment, and I would expect programmers as they gain experience to write or type code modularly or function by function that compiles for sure and works at first time without having to recompile or debug.
thanks for the rest of the stuff
Sign in to Reply