SAN JOSE, Calif. – A new benchmark aims to give engineers and end users a way to measure the performance of Android-based systems. The Embedded Microprocessor Benchmark Consortium (EEMBC) released its AndEBench metric as an app on Google Play and the Amazon Appstore for Android. The source code for the benchmark is available only to EEMBC members and licensees.
AndEBench scores integer performance of a basket of tasks both on the native Android environment and on its Dalvik Java virtual machine. The jobs include a mix of state machine routines, cyclic redundancy checks and matrix multiply operations, but no floating point tasks.
The benchmark can be set to test a system with a single or a multiple core processor. The app provides binary code for testing ARM, MIPS or Intel Atom x86 cores.
A working group including engineers from ARM, Dell, Freescsale, Nvidia, Qualcomm and Texas Instruments defined the benchmark and have already done internal tests with it.
In its only public test to date, AndEBench was run on an Amazon Kindle Fire and a low cost Android tablet, the Velocity Micro T301 Cruz. The Kindle Fore scored 1,370 and 2,720 iterations per second respectively for single and dual core operation and 94 and 145 interactions/second in Java. The Cruz tablet which uses a single MIPS core scored 470 native and 17 Java interations/s.
“We hope to get a community going where people can post their scores and we will start a nice little war with it,” said Markus Levy, president of EEMBC.
The group envisions future versions of the benchmark that could test a range of functions including OS layer calls, graphics, audio, networking, floating point and SIMD functions.
A variety of other Android benchmarks are already in use, but none use open source code so programmers can see exactly what they are doing, said Levy.
“There’s a variety of them, and there are probably some that are decent, but you can’t really tell,” said Levy. “They look cool on the screen and may do some graphics functions but there’s no way to know for sure how they optimize the code or what architecture they may favor,” he said.
AndeEBench start screen and settings menu pictured below.
I forgot to mention, that MIPS is also active in this working group. This particular benchmark, although available and useful for consumer evaluations, is probably better in the hands of the platform vendor, JVM developer, and processor vendor - as it's more of a low-level, under-the-hood test. We are currently defining the features of the next gen test, and there are certainly differences of opinion.
An earlier version of this story said the benchmark was open source. That is NOT correct. The app is available to anyone. The source code for the benchmark is only available to EEMBC members and licensees.
"A working group including engineers from ARM, Dell, Freescsale, Nvidia, Qualcomm and Texas Instruments defined the benchmark and have already done internal tests with it. " Make sense to create this from the company's perspective. At the end of the day, they want something to convince the consumer to upgrade to the hot new device :)
Current MIPS-based tablet from the same SoC vendor scores close to 3x higher compared to the numbers in the story and has a retail price of $99 or less. It was also the world's first Ice Scream Sandwich tablet.
Join our online Radio Show on Friday 11th July starting at 2:00pm Eastern, when EETimes editor of all things fun and interesting, Max Maxfield, and embedded systems expert, Jack Ganssle, will debate as to just what is, and is not, and embedded system.