Breaking News
News & Analysis

EE Times Community Weighs In on Toyota Case

11/7/2013 08:25 AM EST
27 comments
NO RATINGS
1 saves
< Previous Page 2 / 4 Next >
More Related Links
View Comments: Threaded | Newest First | Oldest First
junko.yoshida
User Rank
Blogger
community at its best
junko.yoshida   11/7/2013 9:08:42 AM
NO RATINGS
There are a ton of social media networks and public forums out there on the Web -- where many of us hang out and casually throw in our two cents.

But there are no others like EE Times' forum.

Here's the proof.

We compiled hundreds of messages posted -- scattered all over the places -- to a series of stories we wrote on Toyota's unintended acceleration case most recently tried in Oklahoma. These messages are smart, thoughtful and informative. And educational.

As a reporter, I must confess that every day, I learn just as much from reading our community members' comments. If you need to find out what engineers, responsible for making design choices every day, actually think about Toyota's unintended acceleration case, you can find it all in here. (and a lot more that we didn't have space to bring over to this comilation)

And please, don't hesitate to weigh in.

 

 

Tom Mahon
User Rank
Blogger
discussion among professional bodies?
Tom Mahon   11/7/2013 2:00:18 PM
NO RATINGS
Do discusssion among engineering standards bodies ever happen?  Say, between IEEE and SAE (Society of Automotive Engineers)?  As cars become computers on wheels, maybe it's time for such discussions.

junko.yoshida
User Rank
Blogger
Re: discussion among professional bodies?
junko.yoshida   11/7/2013 2:01:39 PM
NO RATINGS
That's something we definitely need to follow up, Tom!

antedeluvian
User Rank
Blogger
Re: discussion among professional bodies?
antedeluvian   11/8/2013 1:49:21 PM
NO RATINGS
Tom

I was at a TI seminar yesterday which included a session on their Hercules micros and their SafeTI program. The Hercules processors are intended to address the safety market with dual processors executing in lock step along with error correcting memory and more. The presenter did imply that if this processsor had been used then the Toyota acceleration issue may have been avoided.

As cars become computers on wheels, maybe it's time for such discussions.

TI also briefly discussed the general safety standard IEC61508 and the Automotive one ISO 26262. So there are certainly standards and there must have been discussions behind them. I don't know about the Automotive one, but I have seen bits about IEC61508 and it does seem to cover all the current approaches that should be used to create safe equipment.

Antony Anderson
User Rank
Rookie
Re: discussion among professional bodies?
Antony Anderson   11/9/2013 5:19:24 AM
NO RATINGS
The IET (Used to be the IEE) has done some useful work on Electromagnetic Compatibility for functional safety. They also have a Functional Safety Network that anyone can join.

http://www.theiet.org/factfiles/emc/

See also this fact file:

http://www.theiet.org/factfiles/emc/emc-factfile.cfm

See also this factfile:

http://www.theiet.org/factfiles/emc/emc-factfile.cfm

_hm
User Rank
CEO
Time to market pressure, less time for QA and lack of test engineers
_hm   11/7/2013 7:35:55 PM
NO RATINGS
Time to market pressure is very demanding driving most designer and supply chain people crazy. Even before QA getting final 1.0 version or less full version, product launch dates are fixed. QA has very little option but to sign-off product with few remarks.

Another aspect is that product needs lot many test engineers as compare to what they have now. As we did simultaneous engineering as a new concept, simltaneous test engineering should be new emerging branch.

 

Antony Anderson
User Rank
Rookie
Re: Time to market pressure, less time for QA and lack of test engineers
Antony Anderson   11/9/2013 5:59:00 AM
NO RATINGS
May I suggest having a look at Professor Nancy Leveson's Home page at

http://sunnyday.mit.edu/

and especially:

System Safety in Computer-Controlled Automotive Systems, by Nancy G. Leveson, SAE Congress, March, 2000. (Postscript), (PDF).

 An invited paper that summarizes the state of the art in software system safety and suggests some approaches possible for the automotive and other industries.

and what I think is a  brilliant read:

High-Pressure Steam Engines and Computer Software by Nancy Leveson. Presented as a keynote address at the International Conference Software Engineering in Melbourne Australia, 1992 and published in IEEE Computer, October 1994. (PostScript) (PDF ).

A comparison between the history of steam engine technology and software technology and what we can learn from the mistakes made with steam engines.

Quite a lot, as it so happens! May I wish everyone a happy reading weekend!

 

 

DrFPGA
User Rank
Blogger
What is covered in school these days
DrFPGA   11/8/2013 10:18:07 AM
NO RATINGS
It has been a while since my programming class in college. Anyone know what classes cover about safety and reliability these days? Should be a required subject...

Antony Anderson
User Rank
Rookie
Re: What is covered in school these days
Antony Anderson   11/9/2013 6:16:52 AM
NO RATINGS
What I would hope would be to encourage the next generation of engineers to read not just about engineering successes, but engineering failures. It is not the making of mistakes that matters, but the failure to learn from them.

Although an electrical engineer, I remember being shown a  film about the Tacoma narrows bridge disaster - Galloping Gertie  and it really stuck in my mind. It really illustrates dramatically system instability and eventual collapse.

There is now a wealth of material available via U tube that could set the scene for classes in safety and reliability. Otherwise Failure modes and effects analysis or component reliability classes will seem incredibly boring.

bbaudis021
User Rank
Rookie
Analogies with medical devices?
bbaudis021   11/8/2013 3:45:55 PM
NO RATINGS
My conclusion is that the auto industry with regards to the computerization is following the road taken previously by the medical devices and aircraft industry.
   Following to the letter, meaning that they kill a critical mass of people until they are forced to adopt the quality control processes  adequate to tasks the software is given.

    For years they used a limited set of very specialized software developed for the basic engine control. There are fewer engine design entities than the automakers, and the software came even from a smaller pool of design outfits. So there were just few packages out there, very well tested.
    Then the innovation pace in automotive picked up. They added ABS to the mix (but this still usually is a dual-control system: foot-to-hydrolic + computer assist).
The situation started to unravel with stability-control.
    Then came multimedia, GPS, infotaiment  all done by people who never practised and frequently had no concept of safety-critical software design.
In other words we will have cars driven by software designed by the same people who do the APPS ...



jperlick
User Rank
Rookie
What happened at NASA?
jperlick   11/8/2013 5:26:43 PM
NO RATINGS
Hmmm..NASA did a long and expensive study on the reliability of the Toyota throttle control and didn't find anything wrong!  Makes me wonder if NASA is no longer the bastion of high performance, high reliability HW/SW design.  How is it that a consultant can find the problem but NASA cannot? 

I am disappointed in NASA! 

 

junko.yoshida
User Rank
Blogger
Re: What happened at NASA?
junko.yoshida   11/8/2013 6:08:04 PM
NO RATINGS
To be fair to NASA, from what I understand it, they really didn't have as much time as Michael Barr did in looking into this problem. Besides, Barr said that he was able to build his work by picking up where NASA left off. So, NASA's work is not wasted. But at the same time, NASA also made it clear in its own report: "Absence of proof that ETCS-i has caused an unintended acceleration doesn't vindiate te system."

Antony Anderson
User Rank
Rookie
Re: What happened at NASA?
Antony Anderson   11/9/2013 9:40:24 AM
NO RATINGS
Quite a lot of the NASA report was redacted by NHTSA so that its impact was less than it might have been.

rick merritt
User Rank
Author
Kudos
rick merritt   11/10/2013 1:13:30 AM
NO RATINGS
Kudos for some great reporting followed by working the social networkng angle to the hilt!

B. Benjaminson
User Rank
Manager
Re: What happened at NASA?
B. Benjaminson   11/12/2013 8:15:12 AM
NO RATINGS
The problem was not NASA. The problem was NHTSA.

The inside story goes as follows. I have heard the same narrative from three people, each in a very good position to know what really happened.

*************************************************************

NASA's efforts were sabotaged by certain known individuals within NHTSA from the start.  After weeks of delay, the NASA scientists were given banker's boxes of random, unlabeled parts from Camrys that had not experienced any UA events. They were given '2 or 3' documents out of the tens of thousands Toyota produced for the govt. No engineering drawings. Then, just as they were getting started with their analysis and started finding questionable software design practices and tin whiskers, the guys from NHTSA seized the materials and told NASA the investigation was over.

There were witnesses to these events. Will they come forward publicly?

This is an issue that should be investigated by the U.S. Dept. of Transportation's Inspector General.  

*************************************************************

From this and from much other direct evidence that I have, it seems pretty clear that NHTSA and Toyota were way too cozy, to put it mildly, and the public has suffered.

Since then, the NHTSA official who presided over this affair, Ron Medford, has left NHTSA and is now in the "safety director" of the self-driving car project at Google that features a Prius. Hmmm.

Betsy

 

 

esrnetw
User Rank
Rookie
Toyota Failure
esrnetw   11/8/2013 5:48:44 PM
NO RATINGS
There has been a lot of ink used discussing the Toyota electronic throttle issue - however, no post so far has commented on the failure of Toyota to provide a kill button that kills the power to the system of everything else fails. It would seem that looking for a tow truck is much preferable to a runaway out of control vehicle with a panicked driver and pax!

junko.yoshida
User Rank
Blogger
Re: Toyota Failure
junko.yoshida   11/8/2013 6:16:25 PM
NO RATINGS
I actually saw a thread on that very issue. While it sounds good, it might be problematic if everything is killed when your car is going at a certain speed.

W1PK
User Rank
Rookie
Re: Toyota Failure
W1PK   11/8/2013 7:08:44 PM
NO RATINGS
I think much of the discussion may be missing the point.

My previous experience with safety-critical systems was with burner safety controls, at Fireye, Inc.  We designed and verified our products under UL 372.  The centerpiece of regulatory approval was the Failure Mode Effects Analysis, in which we had to enumerate each of the physically possible failure modes of each of the components, and show by analysis or experiment that each of those failures would lead to a safe condition; the unit would either close the fuel valve or operate to spec.  If it continued to operate, then we had to hold that fault and enumerate every other possible fault in combination with the non-shutdown fault.  I've seen FMEA tables that ran to 1500 pages.  We had to do it.  No excuses.

Similarly, if the product had internal programming, we had to show that it was physically impossible for a software failure or memory corruption to cause an unsafe failure.

For safety-critical aircraft software there is RTCA/DO-178, which requires an excruciatingly rigorous development and validation process to eliminate all possibility of a dangerous bug.  This is not pie in the sky, this is legally mandatory.  And then, formally correct software can't be relied on if the hardware platform is buggy; for that there is RTCA/DO-254 to ensure that the hardware is logically correct.

Of course, any hardware platform can be correct at the time of manufacture and suffer a component failure in the field.  So we assume that every physically possible failure will occur at some time in the future; it must be comprehensively proven that any failure of any component will result in a safe condition; ideally, the failure should be immediately obvious so that repairs will be performed before a second hardware failure appears.

In order that airtight proof of design correctness and comprehensive failure mode effects analysis be possible, the design must be kept simple, so that it can be completely understood by its human designers and independent reviewers.  I can't emphasize that enough; it requires a certain ruthlessness to exclude anything from the product specification that will result in complexity.  The safety-critical subsystems must be physically separated from the convenience functions, and from dangerous external influences such as high-level electromagnetic interference.

Personally, I find it unforgivable for a vehicle to be designed in such a way that it's physically possible to open the throttle without the driver's foot supplying the force to do it.  If any part of the *mechanical* linkage between the gas pedal and the throttle fails, not even God should be capable of preventing the return spring from closing the throttle.  Similarly, putting the transmission lever or its equivalent in an electric car into neutral should physically disconnect the wheels from the power source, and it should not be possible for any component failure to prevent it.  There ought be a law.  Literally.

esrnetw
User Rank
Rookie
Re: Toyota Failure
esrnetw   11/8/2013 8:52:56 PM
NO RATINGS
No need to kill everything, just the engine....leave the brakes and all safety systems intact. First rule of problem resolution - when you find yourself in a hole you can't get out of - stop digging then solve the problem!

W1PK
User Rank
Rookie
Re: Toyota Failure
W1PK   11/10/2013 6:13:15 PM
NO RATINGS
No need to kill everything, just the engine....leave the brakes and all safety systems intact.  --
 
Well.  This gets us into another problem.  It should be unlawful to market a vehicle that has not been rigorously proven to be physically incapable of malfunctioning in such as way as to defeat the driver's control of steering, braking and power.  Preventing loss of control of steering and braking is relatively easy: forbid steering or braking by wire.
Engine control is a different proposition, with engines that rely on complex control systems to minimize fuel consumption.  It must be recognized that simply shutting down in the event of a component failure is not necessarily a safe failure, because being stranded in some locations can be fatal to the vehicle occupants.  Thus, fail-safe principles can't be applied to engine controls; fault-tolerant design is required instead.  It should be legally mandatory that before a vehicle can be marketed, there must be peer-reviewed irrefutable proof than no design fault or hardware failure can defeat the driver's control of the engine, as long as the mechanical core itself is capable of operation.


David Ashton
User Rank
Blogger
Re: Toyota Failure
David Ashton   11/10/2013 8:02:05 PM
NO RATINGS
@W1PK... "....forbid steering or braking by wire."     The manufacturers will then cry foul because steering by wire makes it MUCH easier to make left-and right-hand drive vehicles....

alex_m1
User Rank
CEO
The non-embedded community's view
alex_m1   11/8/2013 6:46:29 PM
NO RATINGS
The response of the non-embedded software community is also interesting: They claim the software development processes were bad , the tools we're bad and too low level for such a complex critical software , and the education of electronics engineers is also part here - because they are'nt taught proper software engineering is  and it shows.

Here's are the relevant discussions:

https://news.ycombinator.com/item?id=6695790

https://news.ycombinator.com/item?id=6636811

MS243
User Rank
Manager
Brake and steer by wire
MS243   11/11/2013 7:43:57 AM
NO RATINGS
Fly by wire is done on aircraft -- and if you have flown on a 757,767,747-400,787,777, or any Airbus Airliner, you have depended on this technology from take-off to landing -- The best of these systems are Quadruple Redundant (typically three redundant actuators and dual sticks, plus redundant trim switch controls -- plus a disimilar backup system -- in these systems the power systems are triple redundant or quadruple redundant as well -- (MULTIPLE apu'S AND ENGINE mount Variable Frequency Generators -- with multiple batteries and ram air turbines for backup -- 767's have even glided in for a landing sucuessfully with no fuel on board(due to metric/english conversion error by crew) ---- Automakers have no trackrecord with redundant systems and safetys for doing electronics -- it will be a long steep curve if they wish to climb it)

W1PK
User Rank
Rookie
Re: Brake and steer by wire
W1PK   11/11/2013 11:10:43 PM
NO RATINGS
Fly by wire is done on aircraft -- and if you have flown on a 757,767,747-400,787,777, or any Airbus Airliner, you have depended on this technology from take-off to landing -- The best of these systems are Quadruple Redundant (typically three redundant actuators and dual sticks, plus redundant trim switch controls -- plus a disimilar backup system -- in these systems the power systems are triple redundant or quadruple redundant as well.

 

Exactly.  And that's affordable on a $300 million commercial airliner.  On a car, maybe a Lamborghini owner could pay for a fully fault-tolerant steer-by-wire system.  Nothing less should be allowed on a public road. 

FillG
User Rank
Rookie
Test SEU
FillG   11/12/2013 4:08:56 PM
NO RATINGS
After 15 years in flight controls software, having written code for 787, 747-8, Hawker Premier, Hawker Horizon, Bombardier Challenger, I have never ever seen a test for SEU. Not ever. How does one do it?  Expose the processor card to nuclear reactor and sit around waiting for it to happen? So not being able to deterministically test SEU, how can we ever now what will happen under SEU? We can't.  So we fall back on probability. Toyota's software is an unfortunate example of this state of affairs.

FillG
User Rank
Rookie
Re: Test SEU
FillG   11/12/2013 4:14:38 PM
NO RATINGS
Having shown that an SEU can kill a task does not prove it is actually the cause. But having no alternative explanation, we have a tendency to assume it must be cause. We couldn't be more incorrect. We would be 100% subjective. We are still in the state of not knowing for sure.  There is no less than a Nobel Prize here for the person that can figure out how to analyze an intractable problem.

W1PK
User Rank
Rookie
Re: Test SEU
W1PK   11/14/2013 1:13:20 PM
NO RATINGS
Your objection is well taken.  As I pointed out earlier, the fail-safe approach to safety-critical product design is to assume the Totalitarian Law of Physics, which says that whatever isn't forbidden by natural law is mandatory: it must occur.  If SEU is physically possible, then the product must be designed in such as way that it can be irrefutably proven that it will not cause a fatal accident when, not if, it occurs.

For products where no safe shutdown is possible, it must be irrefutably proven that the product will meet minimum functional requirements after SEU, component failure, external electromagnetic interference, or any other unfavorable circumstance: it must be fault-tolerant.

Probability is irrelevant in safety-critical product design.

Flash Poll
Radio
LATEST ARCHIVED BROADCAST
Join our online Radio Show on Friday 11th July starting at 2:00pm Eastern, when EETimes editor of all things fun and interesting, Max Maxfield, and embedded systems expert, Jack Ganssle, will debate as to just what is, and is not, and embedded system.
Like Us on Facebook

Datasheets.com Parts Search

185 million searchable parts
(please enter a part number or hit search to begin)
EE Times on Twitter
EE Times Twitter Feed
Top Comments of the Week