Underwriters Laboratories has completed — on schedule — its first standard for Autonomous Vehicles. Called UL 4600, it is published and now available at ULstandards.com.
The UL group, which embarked on the ambitious task of drafting the standard only a year ago, finished the job in record time, driven by a sense of urgency among stakeholders, who see safety as paramount for the future of fully autonomous products.
Society is rapidly moving toward uncharted territory in which no rules define how safe is “safe enough” or where the ultimate responsibility lies when autonomous products with no human supervision run into accidents.
While no single standard can solve the world’s autonomous product problem, the authors of UL 4600 have fixed a starting point by asking autonomous product designers to make a safety argument.
Central to the goal of UL 4600 is to guide designers toward “the safety case” for their autonomous products. Phil Koopman, CTO of Edge Case Research and one of the principle authors of UL 4600, previously told EE Times, “If you can’t say what it means to be safe, and you can’t explain why you think the system is actually safe, then probably your system is not safe.” A safety case is “an important piece of designing safe systems,” he noted.
Recommended
UL 4600 Draft Puts Safety Onus on AV Hopefuls
Technologically neutral
UL 4600 remains technologically neutral. It does not prescribe hardware and software specifications. Instead, it offers “safety principles and processes for evaluating autonomous products.” Topics include “risk analysis and safety-relevant aspects of design process, testing, tool qualification, autonomy validation, data integrity and human-machine interaction for non-drivers,” according to Underwriters Laboratories.
UL 4600 breaks from conventional perceptions about international standards. Big global standards are often criticized by industry observers who complain that they are too slow to develop, too hard to update, and often too political. Indeed, vested promoting proprietary solutions tend to flood working groups, sometimes intentionally slowing the standardization process.
Followings are a few highlights of what UL 4600 is, what it is not, and how UL 4600 differs from other standards.
Stakeholders from all around
Underwriters Laboratories created a diverse body of international stakeholders on its Standards Technical Panel (STP) to develop the document. The STP consists of 32 members with voting rights, including representatives of government agencies, academia, autonomous vehicle developers, technology suppliers, testing & standards organizations and insurance companies. Members without voting rights have also contributed proposals and comments online.
Reviews and discussion of proposals was surprisingly smooth, according to Koopman. Sessions were only infrequently interrupted by members adamantly advocating a position or seeking to slow the process. “Everyone was there to get the job done,” Koopman said. “Although I am not one of those standards folks who go to every standard meeting throughout the world, I’ve heard from STP members say, ‘I’ve never seen anything like this before.’”
Those in the STP include: Uber, Nissan, Argo AI, Aurora Innovation, Locomotion, Zenuity, Intel, Infineon, Bosch, Renesas, Ansys, Liberty Mutural, AXA, US Department of Transportation, and others.
Feedback loop
In a fast-moving tech world, an oft-heard complaint about major standards is that the industry is too slow to come to consensus. Worse, there is a gap — sometimes three to four years — before updates and revisions can be published
UL 4600 dodges that bullet, claimed Koopman. “We have a built-in feedback system,” available on the online collaboration platform used by UL Standards.
“Hundreds” of proposals and comments have already come to the group since the UL 4600 draft standard was made public last fall, said Koopman. While not all proposals qualified for the first version of UL 4600, “We are already scheduling discussions, literally starting next week, to review comments and necessary revisions for the next version.” He noted that updates can be published within a year.
Some already using UL 4600
The last outcome desired by participants in the hard work of standardization work is to see the standards deemed impractical or left unused.
While declining to name names, Koopman said, “We’ve heard some companies are already using UL 4600 based on the draft standard we made available last fall.” This illustrates the industry’s craving for a technical standard that makes autonomous products safer.
Further, as more companies use the standard, they will devise their own best practices. Koopman said they can share these developments with the group to be included in updates.
Self-certification vs. Independent certification
The big controversy that broke out at UL 4600 STP was about who will certify final products. “That wasn’t a surprise, though,” said Koopman. He had all along anticipated this as a sticking point.
Obviously, there is a group of companies who advocate “self-certification.” Another faction insists on “a completely external, bullet-proof certification.”
The automotive industry, traditionally accustomed to self-certification, has balked at the prospect of subjecting designs and systems to external certification bodies.
However, many industry experts have been shaken by the Boeing 737 Max fiasco, which cast doubt on the FAA’s oversight of the certification process. Central to the question was the dubious notion of using company-paid employees to inspect their own aircraft for FAA certification. This example bolstered the argument within the UL 4600 STP for requiring an external, third-party body to certify the safety of autonomous products.
The certification issue could have been a deal breaker, but the group averted that risk by asking everyone to offer their own proposals. The STP debated pros and cons, and hashed it out, according to Koopman. A compromise was struck, opting for an “independent assessor” instead of requiring companies to use an external certification body. An independent assessor is not necessarily an external party. It could be a person working for the same organization designing autonomous products. UL 4600, however, stipulates that the company must make a credible argument for the independence of the assessor, by disclosing how and why that person is allowed to make assessments independent of the employing company.
Acknowledging the compromise, Koopman called it a “first step” that can be scrutinized in practice. “If it doesn’t work, we can fix it next year,” he added.
Play well with others
UL 4600 was not developed in a vacuum. It arrives with a full awareness of existing safety standards. Koopman stressed, “There are no conflicts with other safety standards. While it is not required, we expect UL 4600 to be used with other safety standards.”
On the safety standards landscape, the industry already has ISO 26262 (Functional Safety) and ISO 214448 (SOTIF), both designed for vehicles with a human driver responsible for safe operation. In contrast, UL 4600 deals head-on with full autonomy. The standard explains, that “complete removal of humans from performing aspects (including supervision) of autonomous item operation brings with it numerous additional concerns.” UL 4600 was developed to addresses these “additional concerns.”
How much does it cost?
UL 4600 is available for purchase.
The price, from $716.00 to $1,567.00, varies depending whether it is hardcopy, PDF or a combination of the two. The good news is that UL 4600 can also be viewed online free, using UL’s Digital View functionality. Koopman said, “As an educator, this is important. Most students have never seen standards because they can’t afford it. At least, they can view any pages of the UL 4600 now online.”
Recommended
A Wave of AV Safety Standards to Hit in 2020
Some myths about UL 4600
A few in the industry have noted confusion about the relationship between Underwriters Laboratories and Edge Case Research.
As noted, UL 4600 does not require third-party certification bodies to certify products. Contrary to some suspicions on the market, the new standard was not designed to lock in Underwriters Laboratories as the standard’s preferred certification laboratories.
Nor does Underwriters Laboratories holds any stake in Edge Case Research.
The original idea of UL 4600 emerged in December 2018, when the CEO of Edge Case Research Mike Wagner asked CTO Koopman to “write down” his #DidYouThinkofThat notes, and his safety concerns about the development of autonomous systems that completely remove human supervision.
Koopman said, “While I did a lot of typing, many examples (depicted in UL 4600) didn’t just come from me, but from Uma Ferrell and Frank Fratrik.” Ferrell, a system engineer at Mitre Corp., is also a DER (Designated Engineering Representative) for the Federal Aviation Administration (FAA), with extensive experience in aircraft certification. Fratrik, a lead engineer at Edge Case Research, “has expertise in military system safety assessment,” Koopman explained.
Collaboration among these safety experts became the genesis of the emerging standard. Wagner and Koopman decided to involve Underwriters Laboratories, because of its reputation for managing a speedy but rigorous standardization process. Koopman noted that UL 4600 couldn’t have been completed without active support by Underwriters Laboratories. Deborah Prince, UL’s standards program manager, chaired UL 4600 STP, while Jason Smith, principal engineer at UL LLC, a veteran of myriad certifications, has also played a key role. Heather Sakellariou, a standards engineer at Underwriters Laboratories, served as a project manager to sort out all the incoming comments made to the draft standard.
As the development of UL 4600 standard closely followed the ANSI (American National Standards Institute) process, UL 4600 is now automatically becoming the ANSI standard. Whether this will become an ISO standard is, however, another matter. Koopman said that the group may wait for another year before getting around to it.
UL 4600 expects it to make it beyond the United States. The STP members included participants from China, Singapore, Canada, UK, Germany and most of other European countries.
This industry shutdown will put the automotive manufacturers and suppliers in serious financial shape. I hope they realize that developing autonomous driving is a luxury they can't afford and hope they can salvage their company by getting back to basics of making affordable cars people can buy.
If you keep up with vehicle recalls, you'll realize most manufacturers can't seem to properly make things they have been doing for the last 100 years; what makes you think they will be able to produce a 5000 pound vehicle that can reliably drive in the vicinity of other vehicles or pedestrians?