DaveK’s Embedded Security Blog
Take your medicine, eat your SOUP
Dave Kleidermacher
8/2/2011 6:41 PM EDT
The FDA should scrap its panel for approving medical device and learn some lessons from the FAA.
This just in: surprise recommendation of Institute of Medicine panel--the FDA's medical device approval system should be scrapped because it offers little to no assurance of safety for patients.
I've been saying for a long time that the medical industry's certification process for life-critical devices, such as ventilators and pacemakers, is a joke because accreditors are not required to perform any kind of analysis of the device's software/firmware. A 510K FDA approval is essentially a rubber stamp: if the manufacturer can show that a product has essentially the same function as another device already on the market, then it simply doesn't matter if every single byte of code in the new product has been rewritten, relies on SOUP (software of uncertain provenance), or does not follow a quality development process. The government must rely on vendor self-management.
While my experience is that medical equipment developers often have very high internal quality standards and use suppliers with proven reliable products, this maturity is by no means universal, and some choices are downright idiotic. We may like Microsoft Windows for our desktop PCs, but do we want it controlling a digital radiography system that can fry you? The Therac-25 computerized X-ray system, deployed in the '80s, contained the equivalent of approximately 20,000 lines of C source code. Software flaws in the Therac-25 caused massive overdoses, maiming or killing six people. Did you know that some modern digital radiography systems contain 1 million lines of code?
Fixing the problem
The software regulatory environment for medical systems sits in stark contrast to some other life-critical embedded systems industries. The FAA has a rigorous certification process that imposes high assurance requirements on avionics systems contributing to aircraft safety. A flight control system must be certified to the stringent DO-178B standard at Level A, defined as systems whose failure can be catastrophic. The providers of Level A certified avionics have learned to develop and use high robustness components and systems.
Since Therac-25, hundreds of medical device recalls have been tracked directly back to software flaws. There has never been a commercial jet fatality directly attributed to software.
The response we'll hear from naysayers of the panel's findings is that increased safety assurance is bound to force costs up and profits down, which ultimately will defeat the purpose by reducing investment and stifling innovation. This reasoning is flawed.
First, we have the counterexample existence proof in avionics. Improving robustness of the electronics and its software content will actually raise the overall quality of suppliers. Those vendors who lack the experience and will to improve will fail. This will result in a smaller number of expert suppliers that will enjoy increased profits, market share, and long-term success.
Secondly, yet related to the first point: I have always asserted (based on experience) that it is possible to develop high robustness systems rapidly, when using the proper techniques and processes. Finally, modern CPU architectures and systems software technology enable medical device manufacturers to use Microsoft Windows (or Linux, Android, and other popular general-purpose software) alongside high-robustness software on the same embedded computer. Trends towards open source and powerful multimedia need not be sacrificed to achieve safety as long as system partitioning and control of these workloads is performed correctly. Here again is where the medical community can look to the commercial aircraft industry for an example: electronic flight bags (EFBs) are being designed that use Microsoft Windows for spreadsheet calculations yet safely interact with on-board avionics systems--all on the same computer and DO-178B certified.
The good news to take away from the panel's findings is that the sad status quo of medical device approval is now better exposed, and the organization that commissioned this study is none other than … the FDA. The first step to cure is admitting you have a problem.
Dave Kleidermacher is CTO of Green Hills Software. He writes about security issues, sharing his insights on techniques to improve the security of software for highly critical embedded systems.
This just in: surprise recommendation of Institute of Medicine panel--the FDA's medical device approval system should be scrapped because it offers little to no assurance of safety for patients.I've been saying for a long time that the medical industry's certification process for life-critical devices, such as ventilators and pacemakers, is a joke because accreditors are not required to perform any kind of analysis of the device's software/firmware. A 510K FDA approval is essentially a rubber stamp: if the manufacturer can show that a product has essentially the same function as another device already on the market, then it simply doesn't matter if every single byte of code in the new product has been rewritten, relies on SOUP (software of uncertain provenance), or does not follow a quality development process. The government must rely on vendor self-management.
While my experience is that medical equipment developers often have very high internal quality standards and use suppliers with proven reliable products, this maturity is by no means universal, and some choices are downright idiotic. We may like Microsoft Windows for our desktop PCs, but do we want it controlling a digital radiography system that can fry you? The Therac-25 computerized X-ray system, deployed in the '80s, contained the equivalent of approximately 20,000 lines of C source code. Software flaws in the Therac-25 caused massive overdoses, maiming or killing six people. Did you know that some modern digital radiography systems contain 1 million lines of code?
Fixing the problem
The software regulatory environment for medical systems sits in stark contrast to some other life-critical embedded systems industries. The FAA has a rigorous certification process that imposes high assurance requirements on avionics systems contributing to aircraft safety. A flight control system must be certified to the stringent DO-178B standard at Level A, defined as systems whose failure can be catastrophic. The providers of Level A certified avionics have learned to develop and use high robustness components and systems.
Since Therac-25, hundreds of medical device recalls have been tracked directly back to software flaws. There has never been a commercial jet fatality directly attributed to software.
The response we'll hear from naysayers of the panel's findings is that increased safety assurance is bound to force costs up and profits down, which ultimately will defeat the purpose by reducing investment and stifling innovation. This reasoning is flawed.
First, we have the counterexample existence proof in avionics. Improving robustness of the electronics and its software content will actually raise the overall quality of suppliers. Those vendors who lack the experience and will to improve will fail. This will result in a smaller number of expert suppliers that will enjoy increased profits, market share, and long-term success.
Secondly, yet related to the first point: I have always asserted (based on experience) that it is possible to develop high robustness systems rapidly, when using the proper techniques and processes. Finally, modern CPU architectures and systems software technology enable medical device manufacturers to use Microsoft Windows (or Linux, Android, and other popular general-purpose software) alongside high-robustness software on the same embedded computer. Trends towards open source and powerful multimedia need not be sacrificed to achieve safety as long as system partitioning and control of these workloads is performed correctly. Here again is where the medical community can look to the commercial aircraft industry for an example: electronic flight bags (EFBs) are being designed that use Microsoft Windows for spreadsheet calculations yet safely interact with on-board avionics systems--all on the same computer and DO-178B certified.
The good news to take away from the panel's findings is that the sad status quo of medical device approval is now better exposed, and the organization that commissioned this study is none other than … the FDA. The first step to cure is admitting you have a problem.
Dave Kleidermacher is CTO of Green Hills Software. He writes about security issues, sharing his insights on techniques to improve the security of software for highly critical embedded systems.
|
|
|
|
|
Navigate to related information
cdhmanning
8/2/2011 8:46 PM EDT
"There has never been a commercial jet fatality directly attributed to software."
In 1988 An Airbus 320 crashed at an air show. It seems that part of the problem was the software preventing proper throttle operation at low altitudes.
While Therac certainly nuked a few people over a two or three year period (of which apparently only thee died) you should also remember that it saved thousands of lives too. Delaying the project by three months to do extra checking might have saved those six people from being overexposed but it would have also deprived many hundreds of people of treatment.
I am not at all suggesting that we should take a cavalier attitude towards safethy, just that the pros and cons are not always very clear.
Sign in to Reply
repiret
8/30/2011 2:40 PM EDT
As a pilot and an engineer, such things interest me, so I did some research. According to the official report [1], software had nothing to do with the 1988 A320 crash. The pilot was flying too low and didn't add power soon enough. The pilot was even convicted of manslaughter.
[1]: http://aviation-safety.net/database/record.php?id=19880626-0
Sign in to Reply
ALey
8/4/2011 2:13 PM EDT
Curious timing, as such functions of the FAA are currently on furlough ...
Sign in to Reply
Steven Dean
8/11/2011 7:35 PM EDT
Hi Dave,
I’m glad you’re writing about this as it’s a hot topic not only for Medical OEMs, but also for their suppliers, which is where you and I live. The uncertainty and subjectivity of the current model does impact product development strategy and most certainly time to market.
My take on the subject of federal review;
*So long as the resultant changes come soon.
*So long as the resultant equivalent of the 510K process today provides positive patient outcomes, patient safety, while at the same time provides a predictable platform for the Medical OEM submission and review process.
*I’ll inject the idea of streamlining the approval process for companies whose baseline and track record in the field are exemplary; more regimented for those companies with known quality issues.
*AND… Medical OEMs routinely launch in the EU first as the CE process is always mentioned as much more objective, structured and more predictable. What about a model that more closely aligns with the CE process in Europe? It seems to work well.
Best Regards,
Steven Dean
Global Healthcare Segment Lead
Freescale Semiconductor
Sign in to Reply
Paul.Pacini
9/19/2011 4:42 PM EDT
I am so glad to see this! Thank you. I am an electronics engineer with a pre-med background and have over a decade’s experience in bio-medical engineering. I also have quite a bit of flight experience too, so this really caught my eye. Actually, the problem is much worse than you describe, but for different reasons. I hope I can express it well here.
Absolutely, the medical industry has problems with the FDA’s approval process, but in general healthcare in modern countries is exceptionally safe. The real problem with the FDA, their approval process, and the industry in general is not from complex, dangerous equipment possibly hurting people, but rather from useless, harmless, ineffective equipment allowed on the market under the guise of “safe,” simple, and “effective” equipment. You may ask why this would be a problem if it is harmless, but it comes down to two things: 1) hundreds of millions of dollars squandered on useless crap, and 2) people turning to ineffective treatment modalities instead of real medicine.
Quack medical devices go back in history with some hilarious and disturbing examples. It would seem that in today’s modern, Internet connected, tech-savvy world, people would see right through this junk. But that’s sadly not so.
“FDA approved” (more/less meaning grandfathered-in) (or maybe I should say NOT FDA DISapproved) equipment such as the Mac Beam infrared laser treatment system (http://www.devicewatch.org/reports/macbeam/macbeam.shtml ), electro-detox devices (http://www.devicewatch.org/reports/aquadetox.shtml ), and this gem, the Quantum Xrroid Interface System (QXCI) (http://www.quackwatch.org/01QuackeryRelatedTopics/Tests/xrroid.html ) are all disturbingly common quack devices and great examples of useless crap being promoted as effective. Take a good look at http://www.devicewatch.org/ for a huge list of dubious devices and their supporters the FDA should be chasing down, locking up, and fining the hell out of.
Sign in to Reply
Paul.Pacini
9/19/2011 4:43 PM EDT
How do I add spaces!?!
Continued:
Also check out Science-Based Medicine ( http://www.sciencebasedmedicine.org/ ) for some fantastic writing by a great group of doctors debunking all sorts of quackery in the medical field.
Even though some of the examples I mentioned are not truly FDA approved, just the fact the FDA is not arresting and prosecuting the purveyors of this crap speaks volumes. If companies were selling the aviation industry all sorts of magic bolt-ons for a smoother ride, miracle fuel saving devices, super-duper “bird detecting” add-ons using natural energy fields, etc., they would be prosecuted in a heart-beat. But medical devices? Seems scammers have a free and approving market to peddle their junk in and get rich.
Sign in to Reply
gayatrikumar_1
9/20/2011 8:25 AM EDT
This discussion started at human safety in medical equipment point of view and narrowed down to software flaws and its reliability.
Safety and reliability are system level concerns
Long back I had a technical discussion on two topics
1) Terrain awareness and warning system (TAWS)
2) Train activated warning device (TAWD)
Check wikipedia link where there is a citation regarding a crash of Polish flight in 2010.
http://en.wikipedia.org/wiki/Terrain_awareness_and_warning_system
TAWS as a system did not failed here. But The choice of piolt (Negligence) to turn-off (ignore)certain warnings has led to the crash.
If the warning is important to the piolt why there should be an option to turn it off. Whom to blame ? System level designer ? Approver of the system? piolt? Or the Hardware or Software engineers?
The function of TAWD is very simple.When there is a train on track Switch on a warning light at level crossing.
As per the safety standards, Railway signalling and warning systems have to be reliable and fail safe in any case. The people who inspect the system are FOOLS. My intention is not to
curse them. Thier intention is teh that system has to be fool proof and fail safe. They are responsible for the safety of thousands of passengers isnt it?
My opinions are...
1) Anticipate that software/hardware will malfunction at some point of time. Then what to do...?
2) Use fail-safe methods / features at hardware level and in procedures.
It is injustice to blame only software for such incedents.
BTW: I am a hardwrae engineer.
Sign in to Reply
ReneCardenas
9/30/2011 1:52 AM EDT
As flawed as it may be described, the present FDA system is a deterent more than a filter of bad/useless devices.
Many ideas have surfaced on how to improve the process but lobby and economic forces still trump real progress.
Sign in to Reply
Tazruth
10/5/2011 2:01 PM EDT
The information cited by Mr. Kleidermacher is outdated and incorrect. Mr. Kleidermacher states that "no analysis is performed on software/firmware". FDA authority to require Design Control was made possible by the Safe Medical Devices Act of 1990. The final ruling was issued in 1996 with full compliance to Design Controls required by June 1, 1998. Software (SW) and Firmware (FW) do require design Controls. In addition, FDA has very specific requirements for SW/FW with guidance documents issued as early as 1997. The requirements for 510(k) submission documentation for devices containing SW was released in 2005. To refer to a faulty product from 1980 to support claims of FDA review inadequacies in 2011 is unfair and inaccurate. In 1980, FDA did not have the authority to require design controls. Even the terminology used in the article is inaccurate. Devices reviewed through the 510(k) are not "Approved", but "Cleared". It is difficult to take seriously an article with so many inaccuracies. I hope other readers do not blindly accept the author's writing as fact and instead learn a little more about the FDA process before making condemnations.
Sign in to Reply
LSimone
1/5/2012 2:15 PM EST
Thank you, Tazruth. This article only scrapes the surface, and generalizes in a way that makes it incorrect.
IOM doesn't make FDA "admit it has a problem" - FDA had performed its own internal assessment in addition to commissioning the OIM report (link below), and FDA clearly reported its own shortcomings well ahead of the IOM report. Many praised FDA for voluntarily laying it out in public. Many shortcomings were echoed by the subsequent OIM report (e.g., funding, workload, turnover, transparency). However, the OIM report is less-than-useful in several regards, many recommendations were not specific enough. Well, stating the process should be scrapped is pretty specific, but FDA asked for concrete recommendations that the OIM decline to provide.
The OIM was requested to address two questions:
1 Does the current 510(k) clearance process optimally protect patients and promote innovation in support of public health?
2 If not, what legislative, regulatory, or administrative changes are recommended to optimally achieve the goals of the 510(k) clearance process?
The OIM reported answering these was difficult, and it declined to comment on some parts, such as the effect on innovation or specifics on a new pathway. Part focuses on post-market activities, but that's after the product has been released in the wild. 510(k) is premarket. Read the report.
FDA isn't perfect and admits it. FDA has ratcheted up transparency initiatives and has placed the IOM recommendations (among others) out for public comment and recently released more guidance documents on the 510(k) process.
http://www.fda.gov/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHReports/ucm221069.htm
But with regard to other aspects of this post, yes, it would be great to require more detailed transparency to a manufacturer's software, certifications, etc. In that area, the medical device field is sadly behind the ball.
Sign in to Reply