Breaking News
News & Analysis

Toyota Case: Single Bit Flip That Killed

10/25/2013 03:35 PM EDT
Page 1 / 3 Next >
More Related Links
View Comments: Newest First | Oldest First | Threaded View
<<   <   Page 9 / 11   >   >>
User Rank
Re: Noise and probabilities
elctrnx_lyf   10/27/2013 4:19:32 AM
I think it is going to take a lot of effort from Toyota to dig deeper and identify all the buggy cars they have on the roads.

User Rank
Master throttle control
rode666   10/27/2013 4:05:21 AM
I don't understand how the obvious seems to be missing. No electronics can ever be 100% fail-safe because there will always be failures either in code or hardware.  We know that, and it shouldn't be difficult to provide an external mechanism that will return the accelerator to idle if the brake pedal is used.

No-one (sensible) will need to use acellerator and brake at the same time during highway driving, and it's normally expected that the same foot is used for both, and by design they are not suppoed to be used together.

An external or separate micro-controller can easily sense that road speed is above a preset threshold, engine revs likewise, and the brake is applied.  This can then be used as an override that forces the throttle back to idle, disconnecting the ECU if needs be.

This arrangement could quite easily still allow 'heel-and-toe' operation for hill starts with a manual transmission (does anyone still do that?).  At the same time, simple sensing would activate the separate micro if any of the defined criteria were met.

So, if road speed and/or engine revs are above preset limits, the throttle is open (or open beyond a 'reasonable' limit) and the brake is applied, the micro takes over and returns the throttle to idle or kills the engine completely.  Normal human reaction is all that's needed to get the car under control.

Normal driving is unlikely to trigger the event because most people only have one right foot.  Is this idea too simple?

User Rank
Re: Securing future for automobile electronics control
ANTONY ANDERSON   10/26/2013 8:47:03 PM
Bert22306 I think that your thoughts might be centering in the right area. Your valve controller  analogy is probably a fairly good functional fit to the electronic throttle control, except that I would imagine that a valve controller drives the valve both open and closed whereas my understanding is that in toyota's case the PWM driver for the H bridge motor is driven open and it is spring pressure that closes the throttle valve until the limp home position is reached, after which the H bridge reverses and drives the throttle to the fully closed position.

There is an interesting redacted statement in Appendix A of the NASA report which reads:

"A. Duty-Cycle Conversion The duty cycle conversion modifies scales the command based on the battery voltage and converts the signal to a duty cycle percentage. The duty cycle conversion operates at a rate of 16 ms"

So the H bridge controlling the motor voltage, instead of working from a constant voltage supply, as I think would normally be the case, switches the DC supply  voltage, which of course is far from constant, and the duty cycle is adjusted by the ECU to compensate for changes in the supply voltage. My personal view is that Toyota would have been well advised to regulate the voltage to the PWM with a standard voltage regulator and not try to combine the regulating function with the ECU software function controlling the throttle angle. It must surely add unnecessarily to the computing load on the ECU. Functionally the two configurations are effectively the same, but practically are very different.

The person who has done a lot of work on the implications of the duty cycle conversion is Dr Ron Belt who has written up several technical memos for circulation which you and others might find interesting as a stimulus to your own thinking. If you Google "Belt Hypothesis Toyota" you will find two memos on the subject which are hosted on my website.

Now there is another aspect to the toyota throttle mechanism itself that may be relevant and that is if you plot DC motor current against throttle angle you get a very wide hysteresis loop so that the current has to be greatly reduced before you get any  reduction in throttle angle. This stiction is  not mechanical stiction and appears to depend on the motor armature current.

Now this is with  DC excitation and it might be different with a 500 HZ pulsed DC voltage from the PWM because you might expect to get a certain amount of jitter which might overcome the stiction. What is notable about this "stiction" is that it is much greater than the normal mechanical stiction. I have yet to take a motor to pieces and check the design, but a possible explanation is electromagnetic cogging torque. This could be very dependent on manufacturing tolerances if the airgap is small.

So in reality I suspect that we may be seeing a combination of a whole variety of factors including electromagnetic design of the motor, the gearing, the design of the PWM the means for compensating for changes in battery voltage, timing errors, the software,not to mention electrical contact intermittencies, all of which very occasionally might combine together to cause the throttle to move to the wide open position and remain there but which under other circumstances might, for example, result in a sudden uncommanded deceleration. It will be interesting to see what comes out from under the Toyota all-weather floormat as a result of the Bookout case within the next few weeks.






User Rank
Noise and probabilities
DrQuine   10/26/2013 7:46:01 PM
With the numbers of miles cars are driven and the large number of engine cylinder operations per mile (say 12,000 for a 6 cylinder car being driven at 60 mph at 2,000 rpm), low probability problems are likely to surface. That said, in electrically noisy environments with connectors subjected to adverse conditions and wires flexing, the possibility that a bit might get flipped doesn't seem surprising. I guess the question becomes: what processes enable engines to quickly return to proper operating modes when errors are detected? I know for sure that problem hasn't yet been solved on my home computer.

User Rank
Re: Securing future for automobile electronics control
Bert22306   10/26/2013 6:15:40 PM
"There is in my opinion no way of 100% preventing an uncommanded wide open throttle condition occurring from time to time ..."

Yes, this is all about probabilities, and btw, exactly the same holds for mechanical throttles. I myself had this occur to me, in the pre-electronic car control era (well, not full throttle, but certainly open throttle).

It should not be too difficult to design throttle controls that only give the command for a short period of time, which is the way we tend to do this sort of control. E.g., you read the throttle or other command signal at, say, 10 Hz. If the signal is not consistent for n hits, you either go to an alternate source, or you fail safe. If updated commands are not received by the output process, again you fail safe. If the output process dies, again the throttle controller fails safe.

I've had one odd situation in which the valves we were controlling would slowly cycle open with only a single discrete command. So EVEN IF that single discrete command corrected itself after 100 msec, the valve would continue to cycle to full open, before laboriously beginning the close cycle. A very unfortunate combination of events. Since changing the valves appeared to be impossibly difficult, we ended up dramatically improving the error detection logic before closing any discrete signal to valves, which solved the problem (well, at least for several 10s of thousands of years, doing the statistical analysis).

I suspect that the Toyota throttle issue might be caused by a similar combination of unfortunate coincidences.

User Rank
Toyota's culpability here is the tip of the iceberg for everyone
JCreasey   10/26/2013 4:36:42 PM
One assumes that the hardware and software gaps identified in this ECU implementation is sobering for anyone implementing driver assistance systems, drive by wire controls as well as those pushing fully autonomous cars.

Most automobiles today have multiple cpu's communicating over non-redundant networks with a non-zero error rates; as the complexities of control increase in drivers assisted, drive by wire and autonomous vehicles the ability to fully test and verify the implementation reduces. Even though the software and hardware may be designed to meet strict standards, the standard themselves have limitations and are open to some interpretation.

My major concern is that automation systems built  by multiple organizations have an almost zero chance of reaching the same decision in the same time in any given emergency  (and faults are just another emergency) event. Putting lots of these systems in close proximity in high speed flowing traffic may be a classical Butterfly Effect when things do fowl up.

We have so many things to fix to make these systems safe, and it is not helped by the performance achieved by some vehicles. Who on earth needs 0-60 in under 4 seconds  where unintended acceleration means the vehicle may be pulling 0.7g from a standing start. With human perception/reaction times in the 0.5 seconds range at best (for a fully attentive driver to brake as here: ) the vehicle moved about 11ft before you had any chance to react. (Just as disconcerting and deadly would be un-intended braking at highway speeds on those around your vehicle).

Big red buttons for emergency disconnects, brakes expected to dissipate the full engine power or any other human activated device you care to describe will not help. The reaction time of the automation will always be far shorter than any human reaction, so putting reliance on the human to be the arbiter in case of emergencies IMO verges on asinine.

User Rank
is it worse than mechanical springs
jimlux   10/26/2013 4:19:07 PM
For all this talk of unuusal software failures, one should recall that broken throttle return springs on a mechanical linkage are by no means unheard of. I've had one, several of my friends have, back in the 70s and 80s.

All this talk of "sudden acceleration" and the ensuing potential carnage is really a comment on appropriate driver training.  A 70s muscle car going wide open throttle is going to accelerate pretty quickly, and you don't have the saving grace of a modern rev limiter, so the "shift to neutral" aspect is likely to wind up wiht engine damage.

Back then, the argument for neutral was "you'll lose your power steerng and power brakes" both of which are bogus.  You're not going to be parallel parking, you're steering to the side of the road, and the turning effort isn't all that high when the power steering pump is off: People do have the belt fail or the pump fail, and nobody leaps up to sue because the car was uncontrollable.  Likewise, power brakes are an assist, and vacuum operated.  With the engine at WOT, there's not much vacuum anyway.  In any case, you can still stomp on the brake and bring the car to a stop.

The real reason for the shift to tneutral advice is because people would turn the switch to the LOCK position and remove the key, and then be unable to turn the steering wheel. 

In any case, this is just a matter of appropriate training.  Everyone should have the instructor turn the engine off while driving, so they know what it's like.  My driving instructor did this, my father did it. 


Now lets talk about "I stood on the brake and it didn't stop".  There are lots of documented cases where drivers have believed they were on the brake, and were on the gas instead. But leaving that aside, there is no car made where the brakes cannot overpower the engine, IF and only IF, you apply the brakes hard.   Again this comes back to trianing. if you try to "ride the brake" to slow down (as opposed to stop), yes, you will overheat the brakes and they lose their effectiveness.




User Rank
Software Errors Can be Subtle
Gordon.Apple   10/26/2013 3:05:09 PM
I have analyzed systems many times, where a minor, unnoticed software error could prove catastrophic.  In one particular case, the probability of an incremental error happening was extremely small.  However, the cumulative effect over a long period of time eventually proved catastrophic to the system's operation. The company had spend hundreds of thousands of dollars testing, in an attempt to figure out what was happening.  I was brought in as an independent cousultant for another purpose, but needed to examine the relevant code to do my performance analysis. In my final report, I mention the errors I had found in the code, and their potential impact.  The rresponse was, "OMG, we've been trying for months to figure out what was going wrong."  I billed them heavily that month and they gratefully paid it.

Toyota could have used my expertise.

User Rank
Re: Securing future for automobile electronics control
ANTONY ANDERSON   10/26/2013 2:43:17 PM
There is in my opinion no way of 100% preventing an uncommanded wide open throttle condition occurring from time to time somewhere in a world population of over 1000 million vehicles. However the effects of a wide open throttle could be largely prevented by means of a totally independent fail-safe,  a kill switch for example, that reduces engine power in an emergency. The present situation in which drivers have to brake against full engine power is totally unnecessary and potentially very dangerous. From a functional safety point of view it is unacceptable to make the driver the fail-safe for the malfunctioning electronics.

 Hot News for 1897 – Preventing runaway electric taxis

 Improved electric hansom cabs introduced by the Electric Vehicle Company in New York powered by  two Westinghouse motors of 1.5 kW and 8oo RPM.

An emergency button operated by the driver's heel, could render the whole system powerless

Moms,Gijs: The Electric Vehicle –Technology and Expectations in the Automobile Age

John Hopkins University Press 2004 page 83

 So much for progress!

User Rank
Re: Securing future for automobile electronics control
wilber_xbox   10/26/2013 12:05:11 PM
it shows how dangerous an engineering or software engineer can be. Do not frustrate them and keep them happy (pun intented).

<<   <   Page 9 / 11   >   >> Parts Search

185 million searchable parts
(please enter a part number or hit search to begin)
Like Us on Facebook
EE Times on Twitter
EE Times Twitter Feed