Breaking News
Comments
Newest First | Oldest First | Threaded View
<<   <   Page 2 / 7   >   >>
Wobbly
User Rank
CEO
Re: software and hardware stability
Wobbly   11/5/2013 9:06:01 AM
NO RATINGS
@Junko, Thank you for that response. I am surprised that they did not employ ECC given the spectacularly noisy electrical environment that is present in a typical automobile. Ignition noise itself has always been a problem in cars, but even current diesel engines, with their Direct Injection systems, are electrically noisy beasts.

Is anyone using hardware controlled access to device space or memory? This is fairly common in cellular handsets, both for security and runtime stablility. It also makes errors readily observable since out of bounds accesses drive immediate hardware faults instead of leaky data errors that may or may not be observed in testing.

junko.yoshida
User Rank
Blogger
Re: software and hardware stability
junko.yoshida   11/5/2013 8:31:43 AM
NO RATINGS
@Wobbly, according to the expert witness, "Toyota claimed the 2005 Camry's main CPU had error detecting and correcting (EDAC) RAM. It didn't." As you accurately pointed out, the expert witness also agrees that EDAC, or at least parity RAM, is relatively easy and low-cost insurance for safety-critical systems.

Wobbly
User Rank
CEO
Re: software and hardware stability
Wobbly   11/5/2013 7:46:09 AM
NO RATINGS
You immediately addresses the topic of threads in safety critical products, but you did not address the two points that I raised, and those were ECC on memory and hardware memory region protection on client devices.

In thirty years of delivering core network equipment in telecom, including sixteen years within a Network Systems associated division of Bell Labs, I have had to deal with high reliability requirements. Not safety critical systems such as aerospace or medical, but still equipment that was intended to run unattended in locked vaults buried under ground in very remote locations, and perform its own diagnostics and fault reporting and mitigation. So I am not completety out of touch on those issues.

Even in single threaded systems with well defined task definitions, you can gain stability and safety through having well defined hardware access control limiting tasks to only those devices and memory regions that are associated with that particular task.

Having shipped systems that were expected to operate non-stop with five-nines of uptime in deployment, I have had the opportunity to observe things, such as the fact that any significant amount of RAM is going to show correctable single bit errors through a year of continuous operation, so bit flips do happen. Around mid 2005, we had appoximately six hundred router blades in a single distributed network, each blade had 1GB of DDR2 RAM, and over a year of collecting fault data, each blade expeienced about six or seven correctable ECC events. We were able to swap out two or three blades before they failed by having ECC event thresholds that flagged the cards for replacement.

Now admittedly, these where blades with 1GB of memory, running 24x7. But if you count the total installed RAM in all the cars on the highway, times the total run hours, there have to be distributed single bit error events occuring.

So do they use ECC? Or do they not?

Bert22306
User Rank
CEO
Re: Three-part series based on trial transcript
Bert22306   11/5/2013 4:43:36 AM
NO RATINGS
That's what I was referring to, Junko. That testimony was misleading. The "brake override" he was referring to was only the feature where applying the brakes simultaneously cuts the throttle. The implication was that the brakes didn't work at all, which isn't the case. And the throttle override feature does work, except in cases where task X dies while the driver is braking. So it's not as bad as I thought.

Specifically, this quote here:

"Q Where is the function for that brake override? Where is the task located, as you understand it?

"A Yes. So the brake override that is supposed to save the day when there is an unintended acceleration is in task X, of course, because it is the kitchen sink."

Don't you get the impression from this that the brake override won't work when task X dies? And is it made clear that the brakes do work, even if the throttle isn't cut in worst-case scenarios? The brakes would STILL "save the day," if the driver can overcome his or her moment of astonishment.

My approach would have been to make the whole situation clearer from the start, especially in view of the fact that the attorney doing the questioning did not seem well versed in these matters.

junko.yoshida
User Rank
Blogger
Re: Three-part series based on trial transcript
junko.yoshida   11/5/2013 1:55:37 AM
NO RATINGS
@Antoney Anderson, you have been absolutely critical in our Toyota discussions on this EE Times Forum. Thank you so much for chiming in often, offering pointed guidances and bringing clarity to the issues.

junko.yoshida
User Rank
Blogger
Re: Three-part series based on trial transcript
junko.yoshida   11/5/2013 1:18:40 AM
NO RATINGS
@Bert, you wrote:

Now, if somehow that task X death had affected ABS in such a way that the brakes didn't work, the situation would have been a whole lot more dire. In the early reports, this "small" detail was never brought out.


Because we only have a redacted versin of the transcript in which exact functions of Task X were not disclosed, it's hard to tell. But I would like to call your attention to the following part of the court transcript: http://www.eetimes.com/document.asp?doc_id=1319936&page_number=4

This may give us some clues.

Here are Michael Barr's answers to the questions by the plaintiffs' lawyer:

Q Let me ask about that then. The jury heard testimony about a brake override system. Are you familiar with that?

A Yes.

Q Wherein the accelerator is in certain condition, if you press the brake it will automatically cut the throttle. Are you familiar with that?

A I am. There is not one in the 2005 Camry, to be clear.

Q Right. Do you have an understanding of the system that Toyota has since used?

A Yes. I reviewed the one that they put into the 2010 Camry.

Q Where is the function for that brake override? Where is the task located, as you understand it?

A Yes. So the brake override that is supposed to save the day when there is an unintended acceleration is in task X, of course, because it is the kitchen sink.

 

junko.yoshida
User Rank
Blogger
Re: Petrochemical standards in 1980
junko.yoshida   11/4/2013 11:55:36 PM
NO RATINGS
@JIMAshby, what the root cause is for a single bit flip is apparently hard to find.

As the expert witness Michael Barr noted, among dozens of tasks, there are16 million different ways those tasks can die. The experts group was able to demonstrate at least one way for the software to cause unintended acceleration, but there are so many other ways that could have happened.

You may not conisder it as a conclusive evidence. But in a trial like this, it raised enough reasonsable doubt to convince a jury to deliver a verdict against Toyota.

junko.yoshida
User Rank
Blogger
Re: software and hardware stability
junko.yoshida   11/4/2013 11:42:39 PM
NO RATINGS
@MS243, you wrote:

Ground bounce can cause logic corruption in MCU's, DSP's, CPU's and FPGA's.


You are absolutely right about this, hence, the expert witness was talking that corruptions could happen "on certain road conditions on certain days." That makes it imperative to have a built-in selft test of the hardwrae by software, as you point out.

Bert22306
User Rank
CEO
Re: Three-part series based on trial transcript
Bert22306   11/4/2013 8:31:23 PM
NO RATINGS
Sure, it's better to have the engine throttled back in these emergemcies. But it's also true that the brakes can overpower the engine, even at full throttle.

I too was somewhat relieved to discover that the brakes did work throughout these instances of task X death (on the third article or so of the series - it was definitely not clear before that). The power from the engine is just not that huge of a concern, if you plant your foot firmly on the brakes, because, as the Audi tests showed in the mid 1980s, the stopping distances do not change by much, power on or power off. That means, there is not a big difference in the amount of energy the brakes need to dissipate as heat. It's more important to catch the problem before the car really speeds up.

It is true that the vacuum assist will go away if the engine is at full throttle, but that would only occur if task X died while brakes were being applied. Otherwise, if task X died while the brakes were NOT being applied, the throttle would shut down, and you'd have vacuum assist. And here's the really interesting part, even in the worst-case scenario (task X death while brakes are being applied), if the driver HAD pumped the brakes, as the Toyota is programmed, she would NOT have lost power assist! Because apparently, you need to release the brakes for a couple of tenths of a second and then reapply, in order for the throttle to be shut down, even in this worst case.

Now, if somehow that task X death had affected ABS in such a way that the brakes didn't work, the situation would have been a whole lot more dire. In the early reports, this "small" detail was never brought out.

Antony Anderson
User Rank
Rookie
Re: Three-part series based on trial transcript
Antony Anderson   11/4/2013 7:41:37 PM
NO RATINGS
From a functional safety point of view,  the most effective way of stopping a runaway vehicle is first to remove the source of energy causing the acceleration and then to apply the brakes.  It is inappropriate in my view to treat the driver exercising the brakes as the fail-safe for an engine that is out of control.This presumably is why Toyota now fit brake override software.

here are some of the factors that make it inadvisable to rely on the brakes as a fail-safe:
  • Brakes only have a limited capacity for absorbing heat. If the temperature of the brake cylinders rise too far the hydraulic fluid will boil and cause vapour locks which greatly reduce braking efficiency. The temperature at which the hydraulic fluid boils is dependent on the moisture content of the hydraulic fluid and drops as this rises. Hydraulic fluid readily absorbs moisture- hence the importance of changing it on a regular basis.
  • With a racing engine, there is no vacuum produced and hence if you pump the brakes you will rapidly lose vacuum brake assist
  • with a racing engine there may well be sufficient slip in the torque converter to give somewhere between a 2 and 2.5 times torque multiplication factor, which means that you have to press twice to two and a half times as hard to get the necessary braking force at the wheels.

I for one think that the three part series based on the trial transcript has provided an extremely useful and helpful insight into the evidence presented by Dr Barr to the jury. The the resultant discussion has been wide ranging, constructive and fruitful. I certainly have learnt a great deal. Many thanks Junko!

 

<<   <   Page 2 / 7   >   >>


Flash Poll
EE Life
Frankenstein's Fix, Teardowns, Sideshows, Design Contests, Reader Content & More
Engineer's Bookshelf
Caleb Kraft

The Martian: A Delightful Exploration of Math, Mars & Feces
Caleb Kraft
6 comments
To say that Andy Weir's The Martian is an exploration of math, Mars, and feces is a slight simplification. I doubt that the author would have any complaints, though.

The Engineering Life - Around the Web
Caleb Kraft

Surprise TOQ Teardown at EELive!
Caleb Kraft
1 Comment
This year, for EELive! I had a little surprise that I was quite eager to share. Qualcomm had given us a TOQ smart watch in order to award someone a prize. We were given complete freedom to ...

latest comment elctrnx_lyf congrats to rajeev prasad !!!
Design Contests & Competitions
Caleb Kraft

Join The Balancing Act With April's Caption Contest
Caleb Kraft
58 comments
Sometimes it can feel like you're really performing in the big tent when presenting your hardware. This month's caption contest exemplifies this wonderfully.

Engineering Investigations
Caleb Kraft

Frankenstein's Fix: The Winners Announced!
Caleb Kraft
8 comments
The Frankenstein's Fix contest for the Tektronix Scope has finally officially come to an end. We had an incredibly amusing live chat earlier today to announce the winners. However, we ...

Top Comments of the Week
Like Us on Facebook
EE Times on Twitter
EE Times Twitter Feed

Datasheets.com Parts Search

185 million searchable parts
(please enter a part number or hit search to begin)