If we ask these same questions for commercial aircraft, we can pretty well answer these questions in the affirmative, at least from a software point of view. No life has ever been lost as far as we know on a commercial aircraft because of a software error. That’s an impressive safety record, so let’s see how we have achieved this. First, the software on board an aircraft is certainly complex; the Boeing 787 for instance has well over five million lines of critical code. For ordinary users of computers, and ordinary programmers using standard techniques, we don’t believe for a moment that software of this size or complexity can be reliable, and indeed we are used to our personal computers crashing frequently, not to mention our phones and other computer controlled devices. But the software on board planes is definitely not ordinary software created by ordinary programmers using standard techniques. On the contrary, the FAA requires extraordinarily rigorous procedures for creating this software (embodied in the DO-178B certification standard (See here), recently updated to DO-178C (See here). It’s painstaking work, and typically significantly increases the cost of producing software, but it’s remarkably effective in yielding software, that, if not perfect, is good enough in practice so that software is not the weak link in the chain.
Furthermore, we rely on an air traffic control system that is itself a highly complex set of reliable certified software components. And we also rely on simple principles of air traffic control, for example keeping aircraft well separated, operated by a small army of highly skilled air traffic controllers.
So can’t we just apply the same reliable procedures to drone software? I see two huge problems. First, there will be severe resistance to the extra costs and long time scales involved. Although we have been successful in enforcing good procedures for developing avionics software, we have failed in other areas. Voting machine software is still regarded as proprietary, and widely shown to be unreliable and insecure. Modern cars have huge amounts of software aboard (there are more lines of code on a Chevy Volt than aboard the Boeing 787), yet no significant certification of automotive software is required. Have people died as a result? We really don’t know. For example in the recent Toyota incidents of unintended acceleration, the final report was heavily redacted, and the government did not have full access to the software. Given the constraints on the study, the conclusion that the government team was unable to find a software cause for the problem was not really surprising. It should not be interpreted, as it sometimes was in the press, as saying that the software was not at fault, but rather that we simply don’t know. As an example from another safety-critical domain, medical instrumentation software is supposed to meet certain FDA certification standards, but a loophole allows modification of existing devices to be grandfathered in, and from the extent to which this loophole is used, you would think that no new advances had been made for a long time. In one ironic case a patent was invalidated on the grounds that the FDA application said that no new technology was involved. Have people died because of software failures in medical instrumentation? There we do know that the answer is yes (for example the Therac-25 accidents as summarized in http://www.ccnr.org/fatal_dose.html), and the recent uptick in incidents of software-related injuries and deaths from insulin pumps is causing the FDA to take another look at certification requirements for these devices.
The FAA is moving slowly on the issue of drone software certification because of concerns that we don’t know how to proceed safely, but as we see, the U.S. Congress (with police forces and national security agencies egging them on) has signaled that delays are not acceptable, and you have to worry that any attempt to require full certification of drone software will be met by heavy opposition from interests that want these things in the air yesterday.
To make things worse, the complexity of drone software is far higher in my view than standard avionics software. If we have hundreds or even thousands of these things buzzing around in our cities, we are not going to be able to have anything like a coordinated air traffic control system to make sure they don’t cause dangerous collisions, so we will have to depend on autonomous software of far greater intelligence than typical avionics software of today, or on the skills and diligence of the operators controlling them remotely. These people are essentially doing what teenagers do when they play video games. The big difference is that a mistake with a drone can kill real people, unlike a mistake in a video game, which resets and gives you another life.
Do I have a simple suggestion to fix this worrisome situation? Unfortunately no. I think it’s really a very difficult problem from a technical software point of view, from a regulatory point of view, and perhaps also from a political point of view in light of a popular current theory that we are over-regulated and that regulations kill jobs. I fear that Congress is steaming ahead here much too rapidly, without a sufficient appreciation of the risks involved. I understand that many may be worried about Fourth Amendment implications of drones, but surely we can all agree that the safety issue is paramount. We don’t want these things crashing into people and causing death and destruction even if they don’t have sidewinder missiles!
About the author:
Dr. Robert Dewar is co-founder, president and CEO of AdaCore and Emeritus Professor of Computer Science at New York University. With a focus on programming language design and implementation, Dr. Dewar has been a major contributor to Ada throughout its evolution and is a principal architect of AdaCore's GNAT Ada technology. He has co-authored compilers for SPITBOL (SNOBOL), Realia COBOL for the PC (now marketed by Computer Associates), and Alsys Ada, and has also written several real-time operating systems, for Honeywell Inc. Dr. Dewar has delivered papers and presentations on programming language issues and safety certification and, as an expert on computers and the law, he is frequently invited to conferences to speak on Open Source software, licensing issues, and related topics.
. High-integrity object-oriented programming with Ada
. High-integrity object-oriented programming with Ada - Part 1, Part 2 and Part 3.
Part 1 of this three-part article reviews the basics of object-oriented programming and summarizes the challenges it presents for high-integrity programming. Part 2 will provide a primer on the Ada programming language, and Part 3 will detail the tools Ada offers to help developers meet the OOP challenges.
If you found this article to be of interest, visit Military/Aerospace Designline
where you will find the latest and greatest design, technology,
product, and news articles with regard to all aspects of military,
defense and aerospace. And, to register to our weekly newsletter, click here.