In his dotage, my old man (a chemistry Ph.D) used to say, as he slowly stirred his martini with a crooked finger, “the more we know, the less we know.”
As systems become increasingly complex because we overcome old technological hurdles, they also become more unpredictable.
One recent example is report that NHTSA and the NASA Engineering and Safety Center (NESC) published regarding unintended acceleration of Toyota automobiles. Michael Barr has an excellent report on it. In short, NASA said it couldn’t rule in but couldn’t rule out software problems as a culprit in the unintended acceleration problem.
And Stanford, via its Facebook page, has described how engineers are addressing the "aeroelastic flutter" problem, a complicated, unpredictable phenomenon. (P.S. don't watch this video if you happen to be on a plane with WiFi).
The more complex the software (and hardware), the harder it is to model and find corner cases. We seem to be falling behind in assessing the known unknowns and we’re completely in the dark about how to approach unknown unknowns.
We race up the abstraction ladder to try to keep our arms around design complexity, but that creates other issues. I attended the annual meeting of college engineering departments recently in Phoenix and one questioner from industry stood before a panel of academics shaking his head. It’s great to turn out really smart kids who know theory and can deal with abstraction, but if they struggle with basic engineering concepts, companies need to train (or retrain, perhaps) them.
How are we going to get better at anticipating the unknown unknowns? It is formal methods? It is impossible?
Better tools. That is how we have managed each technological step. As industry became dependent on electricity, no-one said 'Let's a'make all our manufacuring systems more fragile', they said 'Look how much more productive we are with this new machinery!' Then, by trial and error, with factories failing every now and then, tools were developed to make reliable electricity generation, monitoring, and usage.
Whether this attitude will work as we abstract our software-dependent systems still further from our caveman roots, remains to be seen. But I suspect that the best use of computer power will be found to be in software manufacture, in removing the human element from every part of the process that doesn't need it.
But that initiative won't come from the software community, will it?
I have a feeling that simple designs are not possible or allowed anymore. It has to be cramped with features, applications, extensions and connections that make them unnecessary complex. If I look to software most people rarely use 15% of the capabilities of f.i. word processors. I know that in the name of progress and profit a lot of the complexity is actually a need that is artificially created by clever campaigns from manufactures and not consumer demanded. And ther is also the learning curve for each product that becomes steeper when getting older. There is no relation anymore with the look and feel of the old product you replaced (f.i. Cell phones) A more baffling evolution is that simple products become more expensive due to there limited market potential (f.i. Cell phones for elderly people) The “Back to basics” although a good slogan has sometimes a strange side effect.
This thread takes on new meaning in the wake of the failure of the fuselage aboard the Southwest Airlines flight.
The USA Today article states the obvious: "Holes in planes rare but pose deadly risk"
I have to think, though, that as circuits shrink and get weaved into more and more materials, that our ability to be alerted to material stress and the like through sensors increases. That's a nice insurance policy when modeling falls short.
What isn't mentioned here is reasoning with uncertainty. Often models don't include it either. The simplest model in the world can be used if it can tell you the boundaries of its accuracy. This is one of the key issues with SPICE. It often uses approximations. If instead of reporting them as gospel, it would tell you the answer was between two limits and always be right on that account, you could then decide if that was an adequate answer or more investigation was needed.
And to NevadaDave, there are known limits to existing knowledge and models. When we go outside those, we know that we are in unknown territory.
I have not observed the oil or nuclear industries pursuing perfection of health and safety. Their fiduciary responsibility is to maximize shareholder profits, and as long as society is the de facto ultimate re-insurer, insurance costs for rare, but statistically inevitable catastrophes, will not be realistically high enough to give health and safety a good ROI. (Society pays later so that well connected industries won't have to pay now.)
And it's not just industry. Consider earthquake insurance in California. Why don't the majority of people buy it, and why don't mortgage companies require it? Partly wishful thinking, of course, but more importantly, they assume that if there's another SF earthquake, the government will pick up a huge part of the reconstruction tab.
@crwilliams and @rpcy... I would love it if you'd write up reviews of the books you referenced. They sound like worthy additions to the Engineer's Bookshelf section.
If you're game, just send me your review to my email and i'll post straight away: firstname.lastname@example.org.
Thanks for the comments!
Yes, we need backups of humans.
The initial comments coming out at the time indicated that there were a number of cases where instruments were warning of problems but were ignored by the higher-ups, management dictated short-cuts from normal petroleum engineering standards, etc.
Without models, you can't design extremely complex machinery in the first place. The trick is to know when to leave the model behind and confront reality directly. Don't blame the models nor the modeling when it's human judgment that's often the root cause of a problem.
On books, first read C. Perrow's book Normal Accidents, which points out the commonalities (and seemingly near-inevitability) of accidents such as nuclear plants, but then read Chiles' book, which also points out that for every Big Accident, there are ten near-misses that would have also been Big Accidents but for the timely intervention of some enlightened person who saw what was happening in time to take action.