SAN JOSE, Calif. — The Embedded Microprocessor Benchmark Consortium (EEMBC) launched two new benchmarks for the Internet of Things. They aim to help engineers measure the effectiveness of end-node security and performance of gateways at the network’s edge.
EEMBC invites interested companies to join the efforts that hope to deliver preliminary metrics early next year. The two new efforts join one already in progress, a benchmark for IoT connectivity that is shedding light on trends in IoT networks.
The IoT Connect benchmark will measure performance and energy consumption across a range of communications tasks and system profiles. The first version focuses on Bluetooth Low Energy (BLE), measuring various aspects of microcontroller and RF performance and energy use on a simulated IoT end node.
EEMBC members, who represent most major microcontroller vendors, currently are most interested in a benchmark for BLE. The work group formed a year ago with a focus on Zigbee, but “Zigbee seems to be fading…[so the group] decided to focus on Bluetooth because it was moving towards mesh networking…people see Zigbee won’t go away, but it’s less interesting,” said Markus Levy, president of EEMBC.
The Bluetooth benchmark, now in an alpha version, could be finished by March. The group is then expected to move on to versions for other networks, probably Wi-Fi or Thread, perhaps followed by LoRa or IoT variants of cellular.
In the fragmented space of IoT networks, companies express a diversity of preferences. So far members such as Renesas and Microchip are most interested in LoRa, while Texas Instruments and Redpine Signals are more interested in ultra low power variants of Wi-Fi. Quite a few are involved with cellular IoT variants, Levy noted.
IoT Connect requires four test boards and is “the most complex benchmark built we have built in 20 years at EEMBC,” Levy added.
The test set up for the IoT Connect benchmark uses Arduino I/O and Raspberry Pi RF boards as well as a test board for an existing energy-consumption benchmark. (Image: EEMBC)
The working group for the gateway benchmark aims to deliver system-level benchmarks measuring overall throughput, latency and energy consumption for node-to-cloud communications. It will probably start with an industrial profile but has not yet specified what parameters it will measure.
The group currently includes members from ARM, Dell, Flex and Intel and hopes to deliver a complete spec by next fall. It will use workloads generated across multiple physical ports to test multiple system components including the processor, physical and wireless interfaces and the operating system.
“Today, without a standardized methodology, IoT gateway benchmarking is not realistic,” said Paul Teich, a principal analyst at Tirias Research and technical advisor to EEMBC.
The EEMBC IoT Security benchmark will measure performance, energy consumption and memory footprint across a variety of profiles yet to be defined. The benchmark will be rolled out in phases with the first phase set for a beta release in the first quarter. It will test standalone security functions such as SHA256, AES, and ECC running on MCUs or accelerators.
“Subsequent phases [will] combine the standalone functions into specific IoT profiles that will allow users to better see, control, and optimize the impact of security at the system level,” said Mike Borza, a member of technical staff for security IP at Synopsys who co-chairs the effort.
The group is expected to define three or four profiles, probably starting with wearable devices. It hopes to have the first profile ready to demo at the Embedded World event in March.
Members currently include Analog Devices, ARM, C-Sky Microsystems, Flex, Intel, Microchip, Nordic Semiconductor, NXP, Redpine Signals, Renesas, Silicon Labs, STMicroelectronics, Synopsys, and Texas Instruments.
— Rick Merritt, Silicon Valley Bureau Chief, EE Times