Opinion: Computer overload threatens auto safety

2/4/2010 2:16 PM EST

The classic joke goes: if cars followed the same technology curve as computers, today you'd be able to buy a Rolls Royce for 200 dollars, it would get 200 miles per gallon, and would blow up every two months.

Suddenly that joke isn't funny anymore. This is a great commentary: while it can make sense to add auxiliary electronics to enhance performance and emmissions, the mechanical systems must always be in control.

2/4/2010 2:23 PM EST

Dean-

Great post on the embedded systems, and the technology becoming a little too invasive and unreliable.

I also find it a bit alarming how this new technology is supposed to learn my driving intent and then make adjustments to data stored about my driving.

It would be great to get your take on some of the technology I've posted on my blog about these "driver assist" systems:

http://www.tuneyfish.com/blog/toyota-driver-assisting-system-accelerator-reaction-force-control-unit/

Also, since you're with Honeywell can you explain whether Garrett is using any of this "driver intent" on the variable vane turbo systems, and if so where's the line drawn on being too "advanced"

Scott

2/4/2010 5:23 PM EST

The value that we have to take from this article is not that computer should not be used in automobile. Rather the computer engineers have to be more careful, make rigorous tests and ensure reliability before delivering it to the customers. There has to be a better process in computer-based development to ensure the same level of quality and safety, that you would get otherwise.

If computers and electronics can control spacecrafts, and pacemakers, why not cars? When internal combustion engines first came into the scene, it had a few issues. Then we did not decide not to abandon motor car and go back to horsebacks. Instead we figured out how to make it work. That's all what we have to do.

http://sunnyeves.blogspot.com/

2/4/2010 9:49 PM EST

The issue ought not to be whether embedded electronics are perfectly safe, or safe beyond a reasonable doubt even, but whether they are safer than humans. Spacecraft control software couldn't achieve that "one in a billion" threshold, but that doesn't make it riskier than manual reentry.

2/5/2010 11:38 AM EST

Well, I thought the commentary timely because my 2010 car just DIED and had to be trailed back to the dealer. Traffic on the web for people who have similar problems with this make/model range from hardware to software being the culprit. In fact, in CA here's a pointer to legal action being taken to address the software side of it:
http://www.consumeraffairs.com/news04/2009/12/ca_bmw.html

2/5/2010 3:01 PM EST

I have to agree with Dean on this one. Computer control in vehicles has gone too far, even in non safety related functions. My 97 van recently refused to start because of a cold solder joint on the connection to the instrument panel! I don't know what the instrument panel has to do with engine ignition today, but it certainly never had anything to do with it in the past. I recently drove a newer van with radar proximity capability that drops out of cruise and applies the brakes if, in the computers opinion and perception via sensors it thinks you're following too closely. I didn't know what was going on at first, but soon found the drivers seat to be way too crowded. The last thing we need are vehicles resulting from the "Vista syndrome" with unnecessary and unwanted features. If they want to impress me, then show me my flying car.

2/7/2010 9:40 PM EST

Computer control of critical systems as throttle, brake, steering in automobile industry need to be assured with robust safety systems on fault (tolerance, recovery, prevention) including redundancy similar to aviation/space standards. For such a new car price tag would have to be acceptable for the consumers.

2/8/2010 12:37 AM EST

By the way, even mid-1960s Lincoln convertibles
had the 'window down before door open' feature.
It was implemented with mechanical switches and relays and I can assure you that your Mustang's software implementation is far more reliable.

Concerning software, I think that many rich and convenient features can be provided at low cost via this approach. Reliability
is obtained via testing. Shortcuts taken
there will surely compromise the customer's
performance expectation.

Of course, language and development environments are important to keep costs low
and to quickly respond to system upgrades.

2/8/2010 4:01 AM EST

These days increasing impact of the computers is very prominent in every field, and this is changing the conventional engineering practices that are followed in that domain(medical, data communication),Automotive is not an exception. One way of looking at it is improve or groom the design with the help of this new concepts or technologies.
Errors has persisted all through the history of design from the very begining, rather they are very milestones for the best possible designs. Rare failures due to adoption of the technologies are inevitable. Better design methods and approaches in embedded doamin may stabilize the products and increse the "comfort to the user". This is the need which was existing and will continue to be so in future.

2/8/2010 1:14 PM EST

@devassocx: Re the mid-1960's Lincoln - really? On what basis can you claim that the SW implementation is "far more reliable"? If you're talking about quality of components I don't think you can compare 1960's era solenoids and switches to today's models. I would bet that the same implementation using modern components will be at least as reliable (and quite possibly more) than the SW implementation. The only reason SW would be more reliable is due to the simplicity of the window system.

Overall, I tend to agree with this article with respect to drive-by-wire (but I do love the entertainment and GPS systems in my car).

Having worked in Industrial Automation for a few years, I can tell you that we had plenty of mechanical and electro-mechanical safeguards built-in for the unlikely (but not impossible) event of software failure. These were automated systems such as luggage handling, and warehousing conveyor systems transporting boxes weighing 100's of lbs at multiple feet per second. These were not safetly-critical systems or manned vehicles, and yet because of the forces involved, wherever human met machine we had safety cages and Emergency stops (buttons that immediately disconnected all power to system) in case of failure to help prevent severe injury.
These safety systems were rarely used, but the important point is that there WERE times that they were used and customers deemed them a critical requirement for an installation.

We should not forget that the reason we have fly-by-wire in fighter jets and spacecraft is because it is _necessary_ not just convenient (in fighter jets it's humanly impossible to control flight surfaces with required precision). Also, those systems have very long and rigorous design and test cycles to help ensure reliability and even then, the systems are designed with multiple redundant control computers and fail-safe systems.

I'm fairly certain the auto industry doesn't have a comparable design process and until they do we should not place a computer in control of the critical control systems on an automobile.

I think entertainment, and possibly climate control, is fair game but leave a mechanical fail-safe in place for steering, brakes and throttle.

2/9/2010 3:38 AM EST

Yes, we have been using antilock brakes for decades, everybody appreciate it, including for its reliability.
But, this is software controlled, each electrovalve can suppress brake pressure on each wheel upon software request.
So, we can do it well with embedded computers.
But if a pedal is at full stroke, software does not know if it is due to a mechanical lock (Toyota gas pedal ?) or due to normal driver intention.
Yes, we could implement consistency checks (pedal stuck at same position for too long, brake pedal pressed while gas pedal fully pressed, etc.).
But I agree that more software is too much software, and may cause more troubles.
So, let's make good and simple job for mechanics, good and simple job for electronics, and keep the brake torque more powerful than traction torque in order to let the driver stop the car even at full gas.

2/9/2010 6:03 AM EST

I can't believe, that this is an article from Honeywell, which developed a special metholodogy to develop a safe software/system. And from which one are derived sevaral others.

I can't believe, if you look around yourself, how many things have in the a control loop some kind of SW: elevators, medical devices, etc. Have you ever thought about that?

I can belive if you want to drive a car with a maximum speed ~ 30km/h, than you don't need any electronic inside. But what about 200km/h?

2/9/2010 7:32 AM EST

I believe Mr. Psiropoulos is just trying to create some thought provoking discussion here. Embedded control is everywhere and here to stay. No matter how you cut it the advantages out weigh the risks.

Here is another software failure that is much worse: the human brain, which results in over 40,000 automotible deaths per year in the US. How do we fix that problem? With electronics and software!

2/9/2010 8:16 AM EST

This debate strikes me as baffling. As in non-automotive applications, one of the main reasons for computer control is that software is easier, cheaper, and quicker to fix than hardware.

2/9/2010 8:24 AM EST

The Prius problem in particular strikes me as a pretty rare occurrence. You have to be driving a 2010 model manufactured before January 2010, and it has to be a model with ESC. Even then the Prius extremely rarely uses friction braking; only for extremely hard braking, ABS or ESC events, and braking below 8MPH does the Prius use the friction brakes. Then, as I read it, on top of that, you have to be in transition from gasoline to electric power (or possibly the reverse) at the exact moment of an ESC event, and you then have to depress the brake pedal for less than a second.

Now that being said, yes, I certainly agree that Toyota deploy the fix to all affected owners, but gosh, I don't see any need to "freak out" here.

2/9/2010 9:09 AM EST

(To clarify, I was not specifically responding to Salid, but just to this issue in general.)

I'll also add that recent events with Toyota clearly illustrate that software control is superior to mechanical control: The vastly more dangerous and far more pervasive problem - stuck gas pedals on six or so Toyota models - is a mechanical hardware problem, whereas the Prius software problem affects a comparatively small number of vehicles under exceptionally rare circumstances.

2/9/2010 9:13 AM EST

The Toyota problem is somewhat old hat. We've had similar, albeit with less complicated systems. Not that I don't agree... I've had more than a few scary episodes with DBW systems on development cars: "traction control," electronic trannys, etc. that cause a car to behave abnormally. I.E. dangerously different than an old fashioned ride. My first impulse is to turn off every "system" that can be.

Now I develop EVs. Guess what? All drive is drive by wire. We've must learn to do it right! It would help if we quit coddling driver expectations - mine included. Specifically, we need to acclimate drivers to regen on the throttle, keep brakes brakes. Today's drivers aren't comfortable unless they're stomping one pedal or the other to the floor. I teach my kids that the the optimum driving position is feet off both pedals. The throttle controls your speed. The brake burns cash. Its a lesson the public needs to learn, regardless of fuel. Sadly, most "drivers" belong in a pod, on a rail. We have enough "infotainment" to lure them there. Just need to put it in the pod and not in the car! (and of course no one will ever hire me again after this comment...)

2/9/2010 9:15 AM EST

The problem is that people forget that Murphy lives. Problems will happen in the best design. There needs to be a simple override system to protect people.

A hot water heater has a pressure release valve that is separate from the control system. The car software may have code that detects problems, but this is not sufficient since it is not independent.

How about a switch that cuts off the fuel pump? Simple and independent from the ECC.

2/9/2010 10:02 AM EST

Do I understand correctly that the author is arguing for mechanical-only control systems in response to the Toyota problems? The throttle problems were caused by mechanical failure!

Toyota should be implementing MORE software to detect faults and control the car to safely when failures occur. For example, if brake and throttle are both pressed, assume the brake is the priority input and reduce throttle. The computer can detect the speed of the car to determine if it's decelerating properly. If not, perhaps the throttle is stuck, so it could affect the fuel injection to reduce power, etc.

Advocating mechanical-only control systems is like throwing the baby out with the bath water and is pastish and myopic. Sensors and software can provide safety that mechanical systems could never achieve, including correcting driver-error (as with ABS and ESC).

2/9/2010 10:27 AM EST

As someone who's worked on engine control software for the past 10 years, I have the following comments:

1. Automotive electronics make cars better (safer, more fuel efficient, less pollutant).

2. You can't find a car without electronic controls, because the electronics is driven in a large part by government regulation. Every car built today has electronic engine control due to emissions and fuel economy regulations. Other regulations mandate things like air bags, tire pressure monitoring, electronic stability control, etc. All of these require electronics and software.

3. The author works for Honeywell Areospace. How much manual control vs electronic control is there in a modern jet, space shuttle, or sattelite?

2/9/2010 3:06 PM EST

Let's be serious. While using electronics and software has its challenges (one of them is a lack of graceful degradation vs. mechanical parts wearing out), the issues with the pedals are mechanical. If the pedal is stuck at the bottom, also a carburator car will go full throttle.
This being said, safety is a system property, not only a software or electronics issue. The fundamental flaw of current cars is their architecture (central engine) that prevents really developing a SIL4 drive-by-wire car. The most we can achieve is SIL3 "fail-safe" (like limiting the engine to 1000 rpm). This is not safe enough. However a car with 4 e-motors, one in each wheel, can be made SIL4 and should be safer than any other car architecture. The problems come from the unnecessary complexity in the current architecture. Introducing drive-by-wire has consequences, one of them the necessity to change the architecture rather than screwing electronic boxes with software inside as straight replacements for mechanical, less efficient parts.

2/9/2010 6:06 PM EST

The problem with car makers is that they don't listen. Som far-east people always think they are the wisest creatures, but they lack common sense and logic. Full stop.
This is about other japan car:
I had Nissan car which nearly killed me by poor design of airflow meter, unstable idling RPM etc. The engine can stall in cold wind unexpectedly. I was thinking that my 9 year old car was a problem.
What was my surprise when I was driving new renault Clio soon after and had the same surprise. To my horror discovered under the hood the same electronics known from Nissan...
Now I have two old Mazda models, 3 years without any nasty surprise...
By the way I work full-time designing some car electronics and writing embedded
software... It can be done correctly and outperform ANY mechanical solution...

2/10/2010 2:02 AM EST

@tbb: for all his inexperience, Sundar has it right. It IS possible to make bug-free firmware, and even systems that are tolerant of the most probable electrical faults. Of course, that still doesn't provide a guarantee of zero failure, flipped bits caused by radiation aren't software bugs, nor are solder joints broken by excessive thermal stresses. The problem is that while it's possible, it's extremely expensive (the expense grows either quadratically or exponentially with complexity, depending on whether partitioning is used to isolate fault trees or not). In a free market, companies will tend to skimp on validation resulting in released bugs.

What to do? Well, society can decide that any failure is not acceptable. Financial penalties aren't likely to be effective, since the potential financial gains are so great. But detailed analysis of each failure followed by tarring-and-feathering the responsible engineer and jail time for everyone in the management chain would assure corners aren't cut. Or, society can purchasers vote and "caveat emptor". Different companies would surely implement varying levels of quality control and it would become a selling point just as crash survivability is. If I'm more likely on any given day to die of food poisoning than car failure with a cheap vehicle, I might reasonably choose not to spend $10000 extra for additional safety. Government need only regulate reporting of failures so that consumers can make an informed decision.

The problem with the second approach is that the cost of failure isn't solely borne by the owner, by potentially by innocent bystanders. Therefore there will almost certainly have to be minimum safety standards.

Spockahontas comes to the rescue with a seemingly reasonable and very achievable goal -- non-inferiority to the human driver who now cannot take control. With this mandated minimum, the market will surely provide several options with superior quality as well.

2/10/2010 5:38 AM EST

I agree with Dean, even believing that we can produce good embedded software. But rbv said it: "that still doesn´t provide a guarantee of zero failure". Current vital vehicle systems with appropriate maintenance in my humble guess are still more reliable nowadays: brakes, steering and throttle. I like the KISS paradigm. Keep It Simple, Stupid. As simple as possible. While complex but reliable systems are not economically viable, let it be used where they are to spare space, weight and non vital activity, so as stuff embedded in a spacecraft or the like, so people can focus on their tasks.
take care :-)

2/10/2010 7:27 AM EST

When we started with electronic controls, everyone was careful because it was clearly new ground; there were no standards to point to, just good engineering by a team that understood the problem.

Now writing software is a commodity. We have standards to show best practice, which is good, but inevitably this becomes something of a box ticking exercise; it helps, but ticking the boxes does not ensure a good quality product. The work gets done where it is cheapest which leads to teams spread over the world with the inevitable communications problems. Meanwhile the required reliability goes up because of the increasing market penetration and as the use of software becomes more commonplace – the example of the windows is a good one.

There’s no point in complaining but I hope that the realisation that engineering excellence is solely derived from standards returns soon.

2/10/2010 8:35 AM EST

I agree with mad_b. The KISS paradigm has been overlooked. I am even more concerned with regards to future consolidation of functions into multi-core controllers. For any high-availability system, single points of failure must be avoided at all cost. It's not optional and can't be considered for cost savings.

2/10/2010 10:11 AM EST

The idea of no mechanical connection to brakes or throttle is kind of scary. We have all heard of airplanes having trouble with fly by wire schemes and yet we still fly. We should develop fail safe modes to prevent sudden acceleration or loss of brakes. This should not be impossible or expensive to do. Manual control is great until you consider the greatest hazard, the person behind the wheel. Someone texting, stoned or drunk is more likely to cause injury or death than anything else.

2/10/2010 10:12 AM EST

At the cursory level, you may be correct. But given more thought, your statements are merely emotional and reactionary. One fault in a billion is unacceptable? Give me a break! Driving a car has all sorts of hazards associated with it that have much greater odds of occuring than 1 in a billion. As mentioned earlier, spacecraft and aircraft (Airbus) are ALL fly by wire. Wake up and smell the 21st century dudes!!

2/10/2010 10:35 AM EST

Disney a few years ago had an animated short about a monster that bought a fancy high tech car only to have it goes haywire. After the car runs amok does the guy say "I miss my old car". KISS is the way to go.

2/10/2010 10:57 AM EST

I would think the question is not "have we gone too far?", but rather "is drive-by-wire safer than the alternative?" In this case the alternative is typically a manual linkage, and right now I would agree that the answer is no. That being said, I also believe that it has the potential to be.

Hydraulic brake lines leak, steering linkages fail, and throttle valves get stuck open. These are all known failure modes in mechanical systems that do not exist in "by wire" systems. However, those systems have their own problems which, owing to the immaturity of the technology, have not yet been overcome to the same extent.

The problem with software/electrical failures is that the relationship between the symptom of a failure and the location of the failure is not always obvious. A stack overflow in the iPod controller might cause the A/C to go haywire (or, more sinister, an unprotected data read during a traction control event could cause the throttle to open up). I would argue that this does not mean electronic systems are intrinsically less safe than mechanical ones, we just do not understand them to the same extent and so our implementations are less robust.

Rebuttals welcome.

2/10/2010 4:24 PM EST

Love a good discussion! I started in automobiles with a Model A ford. Brakes were metal rods, clevis ends, pins and cotter keys. Reliability was known to be in question but it was easily inspected. Those guys weren't dummies though, the pedal moved all that but if that didn't work the hand brake was a totally seperate set of links to a totally seperate cam at the brake shoes (yes, drums) so if the pedal didn't work the lever sticking up from the floor would.

Some of you mentioned aircraft fly by wire. those folks use the same approach now called redundancy. We don't fly on one set of electronic controls. (Think the Concord had 4)

Give me well designed single redundancy and I have pretty much the same confidence I have in the single mechanical system. That is what I want for safety critical functions. Yes, two totally seperate control computers so you can completly remove one and the car still works.

Let the car put the seat back when I want to get out and move me back up when I close the door, That's fun and convienent. Just make sure that when it DOES fail I can still use the car with that fun thing not getting in the way of other fun things.

The basic things that will make the car run and be controlled need redundancy and the car must scream bloody murder when one of the two is down but keep running till I am stupid enough to let both fail.

Yell at me yes, but NEVER override me!

We have NOT gone to far, we have lots further to go, we just have to do it well and remember the least reliable part is still the nut behind the wheel.

2/11/2010 8:16 AM EST

Thanks twk! Finally someone actually employed some reason about this article. All these issues are manageable and engineering discipline is the key to achieving reliable and safe automobiles. You are also correct that the least reliable and least safe component in a car is the texting, eating, drinking, HVAC twiddling & (ad nauseum), dipstick behind the wheel!

2/11/2010 10:57 AM EST

In my opinion, electronic braking is going way too far. Electronic engine control, ok, as long as I have mechanical brakes to push on when the engine controller fails. Being an electrical engineer that builds high power energy converters(10MW+), I can never rely on a processor to perform human safety functions. Things like door switches, thermal switches, and other interlocks are not allowed to be controlled by a processor.

I also don't buy the human error argument either. In my short time driving an automobile (16 years) I have had 0 incidents. In that time I have owned 4 cars, countless computers and other electronic gadgets. 2 of the cars had electronic failures in their lifetime and all of the computers have had hardware failures of one kind or another that made them inoperable. Do I really want this lack of reliability in control of my brakes in an automobile? I don't think so. There are certain things that should not be replaced with electronics and brake control is one of them. I will take leaking brakes lines and worn linkages any day over sudden catastrophic failure that can't be prevented by regular inspection as is the case with mechanical systems. I can't have someone look at the processor and say,"ya I think you should have this replaced in the next 3,000 miles."

2/11/2010 11:32 AM EST

The window rolling down and back up isn't to seal properly (or only to seal) but to relieve pressure in the car so the door is sure to latch properly.
Vehicle safety isn't always a software bug, but improper thinking of when systems should become or remain active. I've got a mazda 3, and if the engine stalls the power steering goes away. It does have a mechanical backup which is far harder than a non-power system. This occurs even if the vehicle is in motion... it's an electric system so it can continue normally and should until the vehicle stops. But it's a lack of requirements.

2/17/2010 8:36 AM EST

The point is that car manufacturers have to go way beyond validation testing in critical areas such as braking and throttle control. Full system analysis and FMEA should have caught these issues- but everybody is in a rush to get to market. Individual components may have been tested to the Nth degree, but what happens when a critical sensor shorts and provides the wrong signal? The comments provided show an industry-wide failure to commit the necessary resources to FMEA at the system level. Are the bean counters going to take responsibility when the proverbial scat hits the fan? Nope- hang the engineers- again!

3/13/2010 6:50 AM EST

I strongly agree with the content of this article. Computer control should be confined to only "assist" the driver in carrying out his/her "willing" action. Software-dependent decision-making and/or wired electro-mechanical actuators for critical parts (as throttle and brakes are) should be accurately kept away from safe cars.

6/12/2010 2:07 AM EDT

One of the reasons for introducing the current levels of computerization is the reduction of cost & weight. With increasing electrically-powered features, the wiring loom for the car increased dramatically. It becomes much cheaper for a small MCU to act as a local communications hub, multiplexing the connections to (for example, window motor, mirror motors, mirror heater, window switches, mirror switches, locks, etc.)

Of course, once the ECU is in place, it's easy to see how the temptation to "add value" through new features can creep in.

With the appearance of ISO 26262 (a derivative of IEC 61508), automotive software development will have to step up to high quality development processes.

7/7/2010 11:56 AM EDT

I too sometimes long for the simple days of cars with breaker points, vacuum advance and carburetors. There is something cathartic about spending a few hours with wrenches and grease and saving a few hundred dollars in repair charges. Until I remember just how much time I used to have to spend working on them. Yes, the repairs were much less expensive, but the downtime was much more frequent and the fear of being stranded someplace was always in the recesses of my mind.

And, it wasn't just the maintenance items. Accelerator cable return springs could snap. Brake lines could leak. Steering linkage could break loose. All of those problems could be just as devastating as a firmware-induced problem, but back then, those problems happened much more often.

Yes, engineers must do everything in their power to produce the best and safest electronics and code, but I don't want to trade the reliability and safety of a modern car for something from the bailing wire and duct tape era.