There's been a lot of reporting on the testimony of Michael Barr who testified for the plaintiff. Did Toyota field their own expert witnesses ? If they did, it would be interesting to learn what their rebuttal was.
The undoubted fact that all of us who drive make mistakes should not be used by the automobile companies as a cover for ignoring functional safety issues and making the driver the fail safe for malfunctioning electronics. What has come out of the woodwork through Dr Barrs testimony is that the EDR results may get corrupted and fault codes disappear when Task X dies - so the software has perfected a way of exonerating itself and the automakers and allowing drivers to shoulder the blame. This might not matter so much if it was not for the fact that people are getting injured and killed and blamed for it and insome cases, if they survive, ending up with lengthy prison sentences. At the end of the day there are moral issues here and public safety.
No matter if the Toyota Camry problem was due to penny-pinching, bad design, shoddy design, inability to anticipate where problems might probably occur, the fact is modern automobiles are systems far more complex than the Model T or Model A and even the best design will have interactions than cannot be anticipated, even assuming a rational driver who is knowledgeable about driving and about the tool – his or her vehicle – and what to do when the unanticipated occurs.
But in most countries, except maybe in Europe, there is little or no education. Unlike the situation in the latter part of the last century, In the US, there is no requirement for driver education in school and the driver's test you take to get a driver's lisence is micky mouse. And as for knowledge about the tool – the inherently dangerous tool under most normal driving conditions – forget it.
I have long expected that as automakers 'relaxed' their testing by not vetting such complex issues as software, and by accelerating the introduction of more and more technology at lower cost, that a decision would have to be made:
Do we increase cost and slow innovation by testing to what we suggest here to be FAA standards?
Increase cost and weight with backup/redundant systems?
Do we ask buyers to sign a disclaimer?
Do we confine rapid innovation to non-critical systems and fall back to robust, proven mechanical systems for acceleration/braking?
I do like Colin's suggesetion of a big red "STOP" button. If we can have that on a treadmill, why not a car?
We are talking of ensuring the stability of of interacting vehicles each with its own power supply. So ensuring stability is bound to be a major problem - where and how is the multi-vehicle system to be kept stable?
@Tom: as a former Intel employee my thots at the conoclusion of a meeting are - what's the follow up action to be taken?
Great question. Is anybody from the auto industry listening? How do we get them to understand that electronic systems relability is important and they need to pay attention? I thikn they are due to the Toyota case, but are they looking at the right things (or are they focused on "software" as the only problem/solution)?
@ dabebermanI think a more fundamental problem with autonomous driving may be proving the algorithms won't suddenly make a life-ending decision incorrectly, but not because of a software bug, just the way the algorithms work.
Good point. It doesn't have to be an official "bug" to kill, just an algorithm that didn't take a particular set of circumstances into account!
I think a more fundamental problem with autonomous driving may be proving the algorithms won't suddenly make a life-ending decision incorrectly, but not because of a software bug, just the way the algorithms work.
But what are you certifying against? I would say that certification should be that you have demonstrated that your systems are immune to radiation hazards to some level; that you have mitigated metstability failures to some small level; and so on... but if we don't know what those possible failure modes are, and/or if we don't analyze them, we can't define the standard in the first place.
If the auto industry wants to take the lead, here, they could simply start doing the analysis and then they can write the standards! Yes, they can! And I'd be happy with that if they prove they have done the analysis and that they understand the results and that the standards they set forth are reasonable. Oh, and there needs to be a public review of all of this.
there is an over-arching issue here. have our systems - in cars, health care, financial markets... - become so complex they are unmanageable except by even more complex systems which in turn require...? where does creative destruction end?
I suspect that cars ARE more reliable. But, a failure may now cause more damage (or death) so the numbers of deaths/injuries per failure may be higher. We need to mitigate failures so they are not catastrophic.
this exchange has been a real eye-opener for me in terms of the universality of this issue. we all have our favorite stories of problems resulting when 'the system crashed.' but when working with a desktop or laptop that was an inconvenience. system crashes at 60 mph are a whole other universe of pain.
@Antony Anderson -- the industrial automation space is going to Profinet. Doesn't look like the automotive space is going to move away from CAN -- two wire system, too easy to use, even though it is not secure.
Third-party safety certification is not being done for autos. Really it must be done.
Otherwise, a company like Toyota can trumpet its "redundancy" in PR and marketing, but that may actually mean they put two sensors on one chip. Which was the case here, in the gas pedal position sensors.
Certifications and regualtions are all "after-the-fact" solutions. We need to understand the fundamental issues before we can regulate them or cause people to be certified to meet some bar. I've seen little discussion of the underlying problems and that's what is bothering me. All of this talk about software makes it sound like there is an easy fix (for everyone knows that software is easy... of course it's not, but that's the conception). I suspect there is no easy fix; the systems are simply too complex so all of the handwaving that used to work will now fail. We need to get back to fundamentals and understand (then mitigate) failure modes.
It might be a good start if the automobile industry recognized that Murphy's Law - whatever can go wrong will - applied to electronics. At present their method of dealing with electronic failures seems to be to blame the driver. Witness the effort to "prove" every incident of sudden acceleration to be due to "pedal error". If they had put in a totally independent fail safe when introducing the electronic throttle we would not be having this discussion now.
I'm not worried about Ford MS Sync. If it failed, my car should still run (well, maybe I'm distracted by it, but that's another issue). I'm concernd about the 50 or so procesors controlling everying from my engine timing to the breaks to the airbags to the steering. Oh, and they all communicate so I'm worried about that, too...
Key word in bold. While engineers think of reasonable limits, the probability of an occurrance, lawyers don't. It only takes one instance to cast doubt.
Agreed! And if the laywers get invovled, that's good in this case! It will force the manufactures to do the analysis both to improve reliabilty AND to guard against lawsuits. "Your honor, we did everything humanly possible to prevent such an event." If thay cannot say that, I would hope they would change their ways so they could.
A company called Quality Control Systems in Md. is keeping track of current Toyota SUA complaints to NHTSA. These complaints are posted on the company's website. The numbers have come down somewhat, but still quite significant, even in new model years. This jives with one of the Toyota documents in which an executive expresses concern in 2010 that the poor quality practices are actually "proliferating." So they are not over.
@junko.yoshida althought I'm for safety-critical standards. There is a tremendous cost associated with the FAA standards, and it significantly slows down development. Think 2x to 4x at least in extended time. The question will come up, does the automobile need this level of certification.
There are reliability engineers working at every auto manufacturer. But they don't have the data they need to do the job. What can fail? What's the probability of failure? What are the modes of failure? There are tools to do this analysis, but I don't know of any auto manufacturers using them. Things like I mentioned, before: radiation, metastability and other forms of failure can be characterized, but are they? I think not.
Since the NASA study, NHTSA has established an electronics department and has hired some engineers there. The Senate recognizes the problem of lack of expertise at NHtSA and they are slowly working to solve it.
Well NHTSA has provided the 1989 NHTSA sudden acceleration report and the redaction of the NASA report and Secretary of State La Hood has exonerated Toyota's software and the car companies take shelter behind NHTSA so to that extent they do need to be called to account
For the engineers out there (I am one) we need to understand the underlying issues that can cause failures and work to reduce them to reasonable limits. This is what the aerospace industry does; it's what medical device companies do... it's not new teritory... except maybe in automotive electronics/systems.
>Let's blame the auto industry, the component manuafactures and the engineers who don't do the proper analysis.
Systems used to be tested to be safe in all possible scenarios, but as they got more complex, engineers have fallen all the way back to just getting it to work one way, then doing testing until its time to deliver.
I take it the kitchen sink approach is no longer something Toyota does. Obviously standards are happening organically in the auto industry...but I think embedded systems software experts should work with automotive engineers from all companies and come up with recommendations. Maybe it should happen through IEEE.
how many more issues like this will come up when auto-parking, collision avoidance, autonomous driving, etc. are more prevalent? Very complex software sitting on top of very complex hardware all being used in a harsh environment.
This is not a NHTSA problem. Assume for a moment that all of our cars will have more electronics in them next year than today... will NHTSA ban them all from our roads? The problem is what's been said, before... there has not been adequate attention to safety and reliability analysis on the systems in the car. Let's not blame NHTSA. Let's blame the auto industry, the component manuafactures and the engineers who don't do the proper analysis.
Yes, Michael, I do believe Japanese cultural issues have been at play all along. Quite a number of the documents put across an "us vs. them" attitude towards Americans, for one. A kind of samurai view of things.
my wife drives a Nissan Pathfinder, my daughter a Jeep, I rent Toyota Camry's. Would hate to lose someone to a correctable software bug, as did happen, and could happen to any loved one. Doesn't that speak for itself regarding getting NHTSA involved?
Talk about languages--with a global supply chain in the auto industry, I sometimes wonder about the human language translation of the documentation of all the software and hardware. Certainly that is a tower of babel that could lead to many mismatches between hardware and software components. Among Toyota's internal documents that I have seen, one introspective one by a senior quality executive mentioned the terrible communication difficulties with overseas suppliers.
Some say self-driving cars will actually be safer by removing "driver error"--the most common cause of accidents--however, for my dollar I want a "death proof" cage around my seat in a self-driving car!
What comes to my mind with the question of software for both Toyota and self-driving cars is both architecture/design and understanding safety/failure modes. Although the details are vague, it sounds like there were both architecture/design issues and safety issues.
Can't wait for the self driving car bugs to show up. I for one won't be trying them out in the first 5 years or so... And google will be checking the code very carefully too. Maybe a search engine for code bugs is their next big project...
So how does Toyota's software and practices compare to that of other automakers? We are seeing more and more problems here - Nissan had a software related recall, and so did Honda, Hyundia/Kia, GM and Chrysler - all this year!
I think there are lots of things that can fail in the hardware: electrical noise (standard stuff), radiation, metastability, etc. It's not just a bad speed signal... it's a valid speed signal but the software reading inconsistent states based on these errors, or having values change when they shouldn't, etc.
What nobody seems to have done is to look at the throttle control system from a control systems point of view. How stable is it? This could be done in the normal way that you might assess the stability of a system. This perhaps is something that could be discussed later.
@Susan Rambo there are some software standards from MISRA. Toyota did use these. However they are not formal methods based and wouldn't pick up things like unprotected variables that I believe Michael Barr wrote about
That's kinda what an FMEA does for HW, and what Barr investigated for SW. Not sure how you'd convert his findings to a definite probability, but I wouldn't want that SW in a word processor, let alone a car!
Quite right not to focus just on the software. For example, electrical intermittencies can cause false speed signals and it sounds as if a false speed signal might under some circumstances trigger a task death.
Software failed, in this case, because the hardware under it failed in some way. Don't we need to fix both? Software cannot overcome all hardware failures. What where the underlying hardware failures? Were there any details? I have some guesses...
The problem is that the auto industry is nowhere near as advanced as other industries insofar as standards of electronic functional safety is concerned.
This is perhaps an issue that could be picked up later. I have sent Junko an article written by Keith Armstrong, Brian Kirk and myself on the subject which I could perhaps be circulated to anybody interested
Jacob and Rich... YES! I understand that underlying issues (that I'm unsure were adequately discussed/investigated) caused the software to fail. That is, hardware failures (bit-flips) caused the software to fail. Yes, the software could/should be improved, but what about the underlying issues that "strange things happen" that we need to understand? All of this focus on the software is fine, but I posit it's impossible to write software that can account for every possible random bit error the system might encounter (some of which are permenent, some are transient). Don't we need to understand the fundamental issues of what caused the software to fail in the first place to even begin to understand how to minimize the risks associated with those failures?
Don't know how relevant this is. We work on the standards for safety-critical software adopted by the FAA and similar agencies in other countries. A large body of work exists on how to develop and test such software.
Someone commented that it is shoddy software engineering ..... is it bad engineering or have these systems become so complex that validating the interactions of these software and hardware systems is pushing the boundries of the processes and test methods that have been traditionally used?
Great questions about NHTSA's response to Barr's findings. I've been to DC twice to push this issue, and have observed and learned a lot about NHTSA (although still not an expert—the Center for Auto Safety has the world's best expertise about NHTSA).
First off, NHTSA responses are normally measured in months or years, not days or weeks. Second, as a policy, NHTSA does not accept materials from outside the agency unless they have a "docket" open on a particular subject, to which the public can contribute materials. They were recently asked to open a docket on SUA and they refused.
I think they will re-do their Toyota investigation only by order of Congress, and so far, Congress has not reacted in any way, AFAIK. The best thing you can do is to contact your own representative and Senators and press them to propose or support a new investigation. The main committees in charge of NHTSA oversight are the Energy & Commerce Committee in the House and the Commerce Committee in the Senate.
AntonyAnderson who has been working as an independent electrical consultant specializing in electrical machine and control system failure investigations and expert witness work. He has been very active in our forum where our readers have been discussing Toyota cases recently.
I also invited
Betsy Benjaminson, who worked as an official Japanese-to-English translator at Toyota; who has now become a public safety advocate
The cross-examination of the expert witness by Toyota was really weak.
This could indicate that Dr Barr was proving a very credible witness under cross examination. Pressing matters further on the points that you suggest might have been counter-productive and reinforced Dr Barr's credibility with the Jury rather than undermining it, as the defence might hope. So they would move on and attack from a slightly different direction.
The cross-examination of the expert witness by Toyota was really weak. They did not contest the main points of the deposition, that is possibility of stack overflow, race conditions among some of the 1000+ global variables, "kitchen-sink" design of the 1-milliseond task, etc. I am really interested to hear why Toyota allowed such unprofessional software and what they are doing now to improve their software development practice.
Hi Crusty 1 and Bert_home: I hope you cna join us for the live chat tomorrow on Toyota unintended acceleration cases. Junko Yoshida and Michael Dunn (who wrote EDN's in-depth story) will be there to discuss. Wed. Nov 6 -- 10amPT, 1pm ET...right here.
I dont have a Camry but I know people who do. I'd be very interested to hear about how common this sort of crappy design is used through out the automotive industry or if Toyota is the worst one. Any info on Nissan, Hyundai and VW designs would be appreciated.
Join EE Times editors for a live chat on Toyota's unintended acceleration cases an Wednesday Nov. 6 at 10am PT/1pm ET. Editor Junko Yoshida will discuss aspects of the Toyota Oklahoma case and what it means to both the engineer and the average consumer.
A Book For All Reasons Bernard Cole1 Comment Robert Oshana's recent book "Software Engineering for Embedded Systems (Newnes/Elsevier)," written and edited with Mark Kraeling, is a 'book for all reasons.' At almost 1,200 pages, it ...