It seems like most FDA stuff goes through a few rounds of clinical trials prior to full approval --- so the Requirements, Test Cases, Coding etc, goes through several itterations - trouble is it may take so long that one is replacing knowlege with each round of funding and round of trials
Well, that is my summary of the 60601-1. A copy is just above my desk. Whether manufacturers stick to it is perhaps a dfferent matter, but it will help cover their posteriors if things ever come to court.
@Barr -- Watson's real advantage, is one can feed it new medical literature (knowlege) all day long while it is working -- and it can graphically point the doctor to the items of interest given a set of medical records for a patient(most likely possibilities) -- kind of expensive for a common cold type visit, but really vital for the rare stump the doctor conditions
Using government mandates have it's own risks. FCC requires Faraday cage tests of phone handsets to get radiation certification. But I bet they don't test all possible combinations of Rx/Tx setups, when we have phones with NFC/Bluetooth/WIFI/GSM/... Not to mention emissions from the battery circuits and DC-DC converters (those inductors radiate "something").
This fact will worry you. IBM was looking at WATSON to deterime how they could use it to advance medical research capabilities. IBM survyed doctors to determine their reading habits. It determine that less than 5% of the doctors read published documents in their field on a continuing basis. Plug that into the fact that 50% of all doctors are below average.
Yes, engineers should be better trained as speakers. (I am pointing a big finger at myself.) But the thing is that engineers by nature tend to be folks who like building things - as well as possible - rather than getting involved in political fights. That is why Dilbert is so true.
Yes again. What price safety? What price certification? Somebody is going to make the implicit decision whater to keep the project alive without full certifation or take a huge loss on the development to date. Such is the real, dog-eat-dog world.
General statement: Engineers should be better trained as public speakers, so that they can more easily convey their concerns and how they should be addressed. And a bit of continuing education to keep up with evolving best practices should be in order as well.
The best-of-all-cases that I can forsee, particularly with complex (and potentially lethal) consumer products is to create consumer expectation (and education) for some sort of independent certification of roadworthiness. It troubles me that NTSB is spending way more money (and time) on vehicle crashes (which are valuable in of themselves), and hardly any proportional time on integration testing of complex control systems, such as the fuel throttles.
It is always a sad situtation to tell some client the IP they had developed for 500 on the cheap would likely take another 90K to turn into a well written set of requirements, and a testbed, and verifiy and validate the IP for the industry they were targeting -- Safe stuff that is complex costs quite a bit of money, and most of the errors that escape are traced indirectly to lack of time, tallent, or treasure
One mars mission I had something to do with involved a failure to activate the priority inversion feature of the RTOS prior to launch so that the lander became overwhelmed with the added cruise-mode photo (not in the original flight plan) clogging up the memory something fierce.
One particularly troublesome aspect of multiple failure detection in aircraft has to do with each alarm, being required to be distinctive, combining to increase the stress and workload so much that the emergency checklist could not be completed in time to prevent the crash. One other very infamous mission to mars failed because of a metric-imperial mismatch in dimensional units so the Mars Observer plowed the fields of mars instead of orbiting.
@Caleb A pilots culture inpacts their training. In America cockpit management is a major part of all pilots training. If you see somthing is not right speak up, regardless of your experience. In Asia the lower person never will speak up when they see a potential problem. That is the biggest problem American instructors have when teaching pilots outside of American carriers.
@garcia-lasheras Check out what happened to a squadron of test F-22 fighters when they crossed the International Date-Line for the first time: http://www.defenseindustrydaily.com/f22-squadron-shot-down-by-the-international-date-line-03087/
For the eyes one can warm a little dab of honey on a spoon on the stove burner till it is liquid and then add a few drops of milk -- put a bit of this in each eye and the conjunctivitis is cleared up shortly
Max -- on your allergies -- might be able to fix them with some locally farmed honey put over 1/4 teaspoon of black pepper -- down the hatch and the bee's resistance to the local pollen gets passed on to you
Arguably, the Air France South Atalantic crash was exacerbated by pilot interaction with a partially failed system. If they had recognised the cause of the anomaly (frozen air data probes) then they could easily have manged the situation. Instead the aircraft stalled with stick back and engines at idle all the way to the sea. Was this a software failure? Discuss.
In the case of the Swiss air crash it was a combination of a mis-set altimeter alignment, plus poorly trained controllers misdirecting pilots into mountainous terrain when an opposite turn to an alternate downwind approach was required. One other famous incident occurred when the automatic landing control was not disengaged, a corporate culture of requiring steep climbs out of this one airport for noise abatement, and an automatic misread of an updraft as if it was a landing flare stall, which cut out the engines during takeoff.
I have thousand of hour flying airplanes. Batteries have always been a problem. There are many aftermarked products that are designed for aviation. They range from better cables and controllers to the batteries. Too many times the cheaper part was used by the company and we the user need to buy a more expensive part to fix the potential problem.
@Caleb: I believe there was a chinese flight where the pilots changed the altitude warning...
I recall a Korean flight doing something like that -- and the junior pilots and engineer were trying to tell the pilot politely that he was doing the wrong thing, but they couldn't bring themselves to say it outright...
@mr_widget: I am truly feeling your pain Max, to within a gnat's crotchet.
I never used to have allergies in England -- they started up when I moved to Alabama (I came here for the night life LOL) -- but the last 3 days have been much worse than ever -- my left eye is completely closed and my right one is streaming -- I'm not wearing my happy face...
Except in the case for the 787, you can't unplug the APU's mid-flight, and so hence the fire hazard. Speaking about that just how robust (and openly discussed) are the Li+ charging circuitry in these huge power storage arrays? Is there any chance of independent review of these (The FAA could be the vehicle for certifying them for flightworthiness, except past experience with that process has been very problematic in detail).
@mexchip The solution, in my view, has three major components: Architecture, Process, and Culture. Architecture means that the system is designed so that when a software malfunction occurs the risks to people are minimized; but also that the software is designed so that malfunctions are rarer and more quickly detected. Process means that the procedures around software development have a logical flow that is designed to keep out and detect as many bugs as possible as quickly as possible. Just like the architecture, the pocesses should include multiple layers of defense. For example, both peer code review and static analysis should be performed (and more, of course). Culture means that the company helps the engineers make the correct architectural and process decisions and supports them in following through over time. Safety culture is broken if shipping by a certain date drives decisions that could negatively affect safety.
@Michael: I realize that Toyota could have done many, many things to improve their code. But I believe that the single thing that could have saved the day was liberal use of assertions. Would you agree?
@All: How many of you use assertions in the code and how many leave them enabled in the field?
By contrast, the system-level design of the Toyota ETCS allows a software malfunction, including a throttle held open against the driver's wishes, to put the vehicle as a whole into a hazardous condition. Under certain scenarios, this cannot be mitigated by the driver--even use of the brakes cannot always overcome a full throttle event.
I am sure that we have all beeen there where we wroted code that worked and was accepted. Then weeks or months later we thought that something outside our code could cause problems in a very rare case and wanted to correct out code to cover that rare occurane. Managements resposne was do not make the change, it will cost too much time and money.
Then months later the rare occurance happened and was going to cost much more in time and money to fix than an earlier fix. The fail to remember that they were warned. Then you were asked why you did not catch the potential problem.
@richardst Thanks for bringing that up. One of the critical FDA guidelines is that the system should be designed in such a way that even though the software can, and likely will eventually, go haywire, electrical and mechanical safeguards must be in place to protect the patient. If that is done very well, the quality of the software can be lower and yet the user can be safe.
In similiar vein, how was it that Underwriters got bootstrapped into the essential and widespread product certification businesses? Was there some sort of industry-wide assessment for funds to establish the lab? Can the extra costs currently seen in retail prices for cars be transferred into assessment funding for an independent testing laboratory, so that there is no net increase in the car price?
Here are some links that may be of interest to folks:
My blog post re: Toyota post-Oklahoma with other links to articles and testimony in it: http://embeddedgurus.com/barr-code/2013/10/an-update-on-toyota-and-unintended-acceleration/
A much earlier article about what could be learned from NASA's redacted public report (written before I was involved personally): http://www.embedded.com/electronics-blogs/barr-code/4214602/Unintended-acceleration-and-other-embedded-software-bugs
Much of the NASA report's review of Toyota's source code ("Appendix A") was redacted. Some entire pages were redacted. I got to see the unredacted NASA report, but only in an RF-secure room with no Internet and no phones (and no metal, belts, watches, or paper in/out). And I can't tell you what I saw there. Little of that is proprietary but lots of it would be embarrasing to Toyota.
@embeddedbarr: Curious from a legal perspective: if you were retained to review some code from firm "X" (e.g., GM, BMW, VW, Boston Scientific, Bayer...) - would that effectively make you ineligible as a witness on the plaintiff side in a lawsuit against said firm? I could imagine many firms retaining Barr Group for a "nibble" of work, simply as a defensive tactic to one day facing you on the other side of the table. Thoughts?
@TheOldTimer I think that's a strong possibility and I hope it's true. Fundamentally, Tesla knew from the outset it was going into the millions of lines of embedded software business. And hopefully hired and architected with that in mind.
Similarly, once the deadly Therac-25 bug was found, a researcher with a Therac-20 discovered that the same bug was the reason his institution had often seen blown fuses. The Therac-20 had better independent safety checks but the same underlying software defect.
I would believe that Tesla would be better. They started out with a clean slate and did not have as many electro-mechanical requirements as a major car company would need to support their old manfacturing processes
Having recently studied the Therac-25 and Patriot Missile Failure lethal software defects, I've noted that both have something in common with Toyota: safety systems and test protocols that didn't keep up with evolving/reused code. For example, Patriot was late adapted to targeting Scud missiles that traveled at 2.5x the speed of earlier aircraft/cruise missile targets. This exposed a bug that was always there.
@caleb Agreed that the UX of the Tesla code looks good, even in the face of danger. Does that also speak to the reliability? It is, unfortunately, so hard to say without actually reviewing the design/code.
One trend I've observed, perhaps relevant to the Tesla question, is that companies that started out as mechanical goods and/or electrical goods (say decades ago) and happened into becoming embedded software companies, generally seem to have a lot better mechanical/electrical practices than software processes.
@richardst, punitive lawsuits may be effective, but ultimately even those costs are passed along to the consumer, unless it targets executives personally rather than the corporation. But it still might help. I heard that with the Ford Pinto gas tank thing the execs compared the cost of the recall against the probable lawsuit costs, and chose to roll the dice on the lawsuits. Perhaps a history of punitive fines might tip the balance the other way next time.
@msamek275 Independent design and code consulting and reviews are key parts of our business. However, I cannot say that I have seen any noteworthy trends yet relative to pre-/post- Oklahoma jury verdict.
One noteworthy issue with voluntary standards is that one of the defenses automakers attempt to rely upon in litigation is that there is no public information about what their competitors are doing better, if anything.
@dunn and @rambo There are some volunatary automotive-industry standards that are international in nature. For example, ISO 26262 is a functional safety standard aimed squarely at automotive. This builds on the excellent work of MISRA (Motor Industry Software Reliability Association) and IEC 61508.
@bberg950 There are some APIs that are widely used in the automotive industry. For example, there is the OSEK RTOS API. This is not one RTOS product but rather a marketplace of RTOS products that all conform to a published OSEK standard API. However, this is not really at the hardware level like you are asking.
@mr_widget It's interesting to look at what the FDA is doing re: software that could kill device users. At a high-level, I understand that (1) They publish a set of software development guidelines. (2) They require device manufacturers to have similar processes in place and to provide paper trails. (3) They reserve the right to audit source code after an event and to pull the device off the market if their static analysis tools, e.g., can catch the flaw that caused the problem.
Unforunately, automotive industry is the wild west by comparison.
@richardst It is customary, when an expert is doing an accident root cause analysis, to reach an opinion that a particular cause was "more likely than not" a primary cause of the accident. This is because the full list of potential causes generally includes more than one that cannot be ruled out entirely. However, in this Oklahoma case, the opinion I reached and testified to was that a failure of the engine controller was the only plausible root cause. This was in large part because both side's accident reconstructions put the driver on her brake at the start of a 150 foot skid mark. Even if the driver had confused the pedals earlier in the drive, this skid mark (apparently from additional use of the parking brake) was way too long for a Camry at that speed unless the throttle was being held open.
@richardst - not so much. The largest cost of electronics in a tandem trailer (commercial) is the copper in the signal wiring. So, in order to minimize that cost the controller is being put in the tandem axle assembly more and more, with pressure sensors to measure the positive air pressure in the airbrake lines.
My own personal bugbear in embedded transportation devices is three-fold: 1) Power management, especially in the charging controllers for the huge Li+ arrays; 2) Avionics - that should be obvious; and 3) trucking - the average cost of a tandem trailer is approx $2500. Just how much scrutiny are the ABS brakes in the trailer wheels going to get when the cost margins are as thin as that?
Since an ever-increasing proportion of vehicles on the road today are using MCU-regulated control systems (ABS brakes, traction control, throttle, fuel mixture, to name a few), do you think it time to have some clearinghouse of experts to examine the software operating in these things? The Top Gear fan in me wants very much to assure myself that the next purchaser of a McLaren or Bugatti can be certain to be under safe operating conditions as much as possible, since these cars, especially, are running closer to the ragged edge than any other vehicles, other than professional racing cars. I can assume the racing teams are closely scrutinising the code as a natural production process :-)
On the Toyota case, it is my understanding that (i) the ECU code was both ugly and buggy and (ii) uninteded, continuing, accelleration would occur if the Task X control bit happened to flip while the vehicle was accelerating under cruise control. Please correct me if I am mistaken.
The jury decided that this must have happened. What do you think is the chance that this actually happened? Do we know if the plaintiff routinely used cruise control?
Have you been able to examine other manufacturer's and/or models' code to this level of detail? If so, have you found it better or worse than the Toytoa code?
Our next live online chat will commence on Friday 14 March 2014 at 10:00 a.m. Pacific Time (1:00 p.m. Eastern Time). You'll have to work out your local time from these clues (you can always use this handy-dandy Time Zone Converter).
Your host will be Max Maxfield, and our guest will be Michael Barr, CTO and Co-Founder of the Barr Group. As Michael says: "Embedded software can be dangerous, even lethal." In this week's live online chat, Michael will be available to answer questions on such topics as Therac-25 (a radiation therapy medical device that massively over-dosed 6 patients from 1985-87), the Patriot missile defense system's deadly failure to intercept a Scud on February 25, 1991, and Toyota's issues with unintended acceleration.
The Other Tesla David Blaza5 comments I find myself going to Kickstarter and Indiegogo on a regular basis these days because they have become real innovation marketplaces. As far as I'm concerned, this is where a lot of cool ...