Zen and the Art of Protoype Debug

"'…at some point you've got to say, 'Wow, maybe the critics who say this is a really flawed design are right.' "

I don't think it is too much of a stretch to say that the "SPICE mentality" of modern college-produced engineers was a large contributing factor. By "SPICE mentality" I mean the tendency of some engineers to overly rely on computer simulations along with a commensurate reluctance to get their hands dirty. All this is compounded by a lack of respect for testing as a function, test engineering as a department and test engineers as fellow professionals.

We know that in Europe the general feeling is that anyone that picks up a soldering iron or bread-board is "blue-collar" and somehow a lesser person than all those high-tone elites that gave us communism, fascism and the Oil for Food scandal. What is so sad is that this same effete sit-behind-a-desk-in-front-of-a-computer culture seems to be infecting American engineering as well. I have to believe one forcing function in the intellectual castration of the engineering profession is the college professors that share the view that only shop-rats work with their hands and that really worthy people just sit around all day mulling over Fourier optics and Poisson Variable algorithms all day long. It goes back to the old philosophical debate that questioned whether the perfect mind could predict the entire operation of the universe just by theory-- with no physical inputs or testing of theories required.

It has taken me several decades to see the essential function good testing plays in good design. In fact, they are just inseparable sides of the same coin. I remember touring a GM design facility in the 70s. There was a fixture with big unwieldy air cylinders that was opening a truck door and then slamming it shut every few seconds. I remember being amazed that anyone would make such a big kludgey looking contraption. When I made some comment about how stupid it looked, the engineering manager giving me the tour said "But how else would we know how good the hinge and latch mechanisms are?" Indeed. I also remember being surprised while working at Ford Motor to see a bank of windshields set up with spray misters and windshield wipers slapping from side to side. This was when I learned that damp-dry glass causes the highest motor torque, higher than wet glass and higher than dry glass. That was also where I saw "calibrated dirt"-- a witch's brew of dust, dirt and particulates standardized by the ASTME that was used to grunge up the linkage so that is would wear the same as in real life.

I have always prided myself on being a hands-on engineer and I find I tend to respect other engineers that also have a little dirt under their fingernails. Good design is like sausage-- it tastes really good but you wouldn't what to see it being made. I was talking about this with a manager at a very successful analog chip company. He told me that he asks engineering candidates if they change the oil in their own car. If they never have even tried to change their own oil, he ends the interview early and tries to find someone that enjoys exploring the physical world as much as the intellectual world.

Being able to understand what it takes to get a paper design made real is critical to competent engineering. When I consulted at Teledyne a decade ago I meet a mechanical engineer that I really hit it off with. He had designed this really slick little billet housing shaped like a pie section. It was supposed to house one of my high-voltage power supplies in a missile. Since I knew a lot of machinists, he gave me a set of prints to get quotes. I gave the prints to my buddy Danny who had a shop a few doors down from my Office/warehouse/shop/consulting-place megaplex. After a day Danny came back and exclaimed: "Man that engineer really knows his stuff!" I asked him why he said this. Danny pointed out that a thin-sectioned hollow part like this in the shape of a piece of pie is normally a nightmare to hold and clamp. And holding and clamping is a very big part of what concerns machinists in their quest to make a good part. Danny said that every time you have to reposition the part in a vise you are sure to lose the inherent 0.0001 inch accuracy of a modern CNC.

There was also the risk a small chip would get caught in under the part and really ruin it. And funny-shaped parts like this always needed "soft-jaws"--- aluminum jaws cut out to a shape specifically made to hold this one specific part in one specific position. For some jobs like this the time spent in making the soft jaws exceeded the time need to machine the part itself. Danny was amazed at this part. It looked so complicated at first glance. (After all, this was rocket science.) What amazed Dan was how easy it was to machine. He said it went in the vice as a big square block. Then it was roughed out and entirely finish-machined with a 3/8th mill from one side. Then the part was flipped into a single set of soft-jaws and the other side was all machined, also with a single 3/8th mill. Where there were "breakouts", sections that cut through features previously made, these were all done on purpose and they were all properly documented on the print.

Danny had never seen anything like this in his entire career as a Silicon Valley inspector and machinist. When I went into Teledyne the next day and told the engineer what my friend Danny had said about his design he seemed a little embarrassed by all the praise but you could tell he really liked the fact other people appreciated his art. When I asked him how he got so good at mechanical design he had a simple answer: "Well, my dad was a machinist and I have a hand-crank Bridgeport in my garage so I know what it is like to actually have to make the chips as opposed to just scratching lines on paper. No kidding.

I have seen nightmares of engineering designed by people that had no time for the complaints of manufacturing. They figured those people in the back were peons and beneath their station. As one buddy said about one of these prima-donnas: "He's never been off the carpet." This referred to the fact that the cubicles housing us engineers were always carpeted but the test and manufacturing floors as well as the engineering lab was linoleum tile. I guess in this guy's perfect world it wasn't his fault if the peons could not hold a few thousandths tolerance on holes on opposite sides of a bend. It was the peons fault they could not bend metal to his whims. This is the kind of guy0 that makes square holes inside blocks of invar steel, not realizing a rotary tool has to carve out the material and that there needs to be a radius on inside corners unless he wants to pay for a broaching tool or an EDM operation.

Amateurs.

It has always been a particular blessing of electronics that there is less of a stigma for an EE to hold a soldering iron or hunch over a scope. What I fear is that the same effeteness that is causing the space shuttle to blow up is now infecting our profession. More and more we see young people out of college that think an EEs' job is to run a few SPICE runs and then toss the schematic over a wall to some poor schmuck that has to clean it up, add connectors, find real part numbers for all the components and then another toss to a semi-technical person that lays out the circuit board. As for test, well, hey, the CAD companies say we don't need to do that anymore so lets just ship it to the customers. The worse profession for this is software. If General Motors built cars the way Microsoft builds software the CEO would be in federal prison married to the man with the most cigarettes in less than a year. Over and over my co-workers and I find designs in the trade papers that could not possibly have been built for the very basic reason that the circuits won't work, even upon simple inspection. As long as American engineers feel that it is beneath them to sweat over a scope or test rig then there is little hope for retaining America's reputation as an inventive can-do society.

When I look at Bob Pease or hear old timers talk about how Bob Widlar would hover over the bench as soon as the silicon came back from the fab, I see a very different ethos and world-view than the recent grads our academic ivory-towers are producing. Neither of the Bobs needed documented PowerPoint presentations from the Test or Applications department to understand what his silicon was doing. Neither one would stall for a few months discounting negative results, blaming the test setup or the intellect of the test engineers. Widler and Pease were the test engineers. And being a test engineer made them a better design engineer. And being a design engineer made them a better test engineer. And this duality fed off itself until you have legends that are regarded as gods of electronics. If more of us young guys would get into the lab with a soldering iron we might have a chance at becoming legendary too. No one can sit at a SPICE workstation and predict the reality of a system based primarily on quantum mechanics for crying out loud, just like no mechanical engineer can sit at a CAD station and predict the behavior of foam on a mach 3 vehicle.

It is interesting to note that the ascendancy of Japan as a design, engineering and manufacturing powerhouse is directly related to the status of testing. In Japan there is no stigma in being a test engineer. Design does not feel it is some superior entity full of superior minds. Design and test work together to solve problems and they have progressed technology mightily with this mutually respectful relationship. I suspect that this developed because Japanese engineers at first had to just copy Western technology. When they did, or I should say, when they thought they were copying things right, they soon developed a reputation for cheap broken junk that didn't work very well even when it worked at all. Well, the Japanese character is not one to suffer humiliation well. So they started to test the copied designs. And they put the best-and-brightest on this job. The payouts were tremendous. Having good test engineers is far more important than having good design engineers. The test guy will catch the mistakes and teach the design guy what not to do. Most importantly, the consumers will never see the junk or the explosions. And after a while you will end up with a good design engineer to go along with the good test guy. Believe me, if you have $150 grand to spend on one test engineer and one design engineer, hire a $100k test person and a $50k kid out of college. The kid will learn a lot faster with a good test department and your customers will be a lot happier.