News & Analysis
Comment
EETIMESMARKUS
I forgot to mention, that MIPS is also active in this working group. This ...
rick.merritt
Group rolls new Android benchmark
Rick Merritt
3/14/2012 8:00 AM EDT
SAN JOSE, Calif. – A new benchmark aims to give engineers and end users a way to measure the performance of Android-based systems. The Embedded Microprocessor Benchmark Consortium (EEMBC) released its AndEBench metric as an app on Google Play and the Amazon Appstore for Android. The source code for the benchmark is available only to EEMBC members and licensees.
AndEBench scores integer performance of a basket of tasks both on the native Android environment and on its Dalvik Java virtual machine. The jobs include a mix of state machine routines, cyclic redundancy checks and matrix multiply operations, but no floating point tasks.
The benchmark can be set to test a system with a single or a multiple core processor. The app provides binary code for testing ARM, MIPS or Intel Atom x86 cores.
A working group including engineers from ARM, Dell, Freescsale, Nvidia, Qualcomm and Texas Instruments defined the benchmark and have already done internal tests with it.
In its only public test to date, AndEBench was run on an Amazon Kindle Fire and a low cost Android tablet, the Velocity Micro T301 Cruz. The Kindle Fore scored 1,370 and 2,720 iterations per second respectively for single and dual core operation and 94 and 145 interactions/second in Java. The Cruz tablet which uses a single MIPS core scored 470 native and 17 Java interations/s.
“We hope to get a community going where people can post their scores and we will start a nice little war with it,” said Markus Levy, president of EEMBC.
The group envisions future versions of the benchmark that could test a range of functions including OS layer calls, graphics, audio, networking, floating
point and SIMD functions.
A variety of other Android benchmarks are already in use, but none use open source code so programmers can see exactly what they are doing, said Levy.
“There’s a variety of them, and there are probably some that are decent, but you can’t really tell,” said Levy. “They look cool on the screen and may do some graphics functions but there’s no way to know for sure how they optimize the code or what architecture they may favor,” he said.
Related links and articles:
www.eembc.org
News articles:
European startup attempts many-core revolution
Embedded browser benchmark released
AndEBench scores integer performance of a basket of tasks both on the native Android environment and on its Dalvik Java virtual machine. The jobs include a mix of state machine routines, cyclic redundancy checks and matrix multiply operations, but no floating point tasks.
The benchmark can be set to test a system with a single or a multiple core processor. The app provides binary code for testing ARM, MIPS or Intel Atom x86 cores.
A working group including engineers from ARM, Dell, Freescsale, Nvidia, Qualcomm and Texas Instruments defined the benchmark and have already done internal tests with it.
In its only public test to date, AndEBench was run on an Amazon Kindle Fire and a low cost Android tablet, the Velocity Micro T301 Cruz. The Kindle Fore scored 1,370 and 2,720 iterations per second respectively for single and dual core operation and 94 and 145 interactions/second in Java. The Cruz tablet which uses a single MIPS core scored 470 native and 17 Java interations/s.
“We hope to get a community going where people can post their scores and we will start a nice little war with it,” said Markus Levy, president of EEMBC.
The group envisions future versions of the benchmark that could test a range of functions including OS layer calls, graphics, audio, networking, floating
point and SIMD functions.
A variety of other Android benchmarks are already in use, but none use open source code so programmers can see exactly what they are doing, said Levy.
“There’s a variety of them, and there are probably some that are decent, but you can’t really tell,” said Levy. “They look cool on the screen and may do some graphics functions but there’s no way to know for sure how they optimize the code or what architecture they may favor,” he said.
AndeEBench start screen and settings menu pictured below.
Related links and articles:
www.eembc.org
News articles:
European startup attempts many-core revolution
Embedded browser benchmark released
|
|
|
|
|
Navigate to related information
goafrit
3/14/2012 7:11 PM EDT
This is a necessary one. The key challenge will be if they can keep up with the innovation in these products.
Sign in to Reply
EyalB
3/14/2012 8:03 PM EDT
Current MIPS-based tablet from the same SoC vendor scores close to 3x higher compared to the numbers in the story and has a retail price of $99 or less. It was also the world's first Ice Scream Sandwich tablet.
Sign in to Reply
EyalB
3/14/2012 8:12 PM EDT
Make that Ice Cream Sandwich :)
Sign in to Reply
rick.merritt
3/14/2012 9:49 PM EDT
What did they do to goose the score?
Sign in to Reply
selinz
3/15/2012 11:26 AM EDT
People always love benchmarks. It gives justification for paying for an upgrade.
Sign in to Reply
eewiz
3/15/2012 2:09 PM EDT
"A working group including engineers from ARM, Dell, Freescsale, Nvidia, Qualcomm and Texas Instruments defined the benchmark and have already done internal tests with it. " Make sense to create this from the company's perspective. At the end of the day, they want something to convince the consumer to upgrade to the hot new device :)
Sign in to Reply
EETIMESMARKUS
3/21/2012 1:32 PM EDT
I forgot to mention, that MIPS is also active in this working group. This particular benchmark, although available and useful for consumer evaluations, is probably better in the hands of the platform vendor, JVM developer, and processor vendor - as it's more of a low-level, under-the-hood test. We are currently defining the features of the next gen test, and there are certainly differences of opinion.
Sign in to Reply
rick.merritt
3/20/2012 12:55 AM EDT
An earlier version of this story said the benchmark was open source. That is NOT correct. The app is available to anyone. The source code for the benchmark is only available to EEMBC members and licensees.
Sign in to Reply
rick.merritt
3/20/2012 1:03 AM EDT
BTW, Markus Levy has collected some additional benchmark scores:
Samsung Galaxy I, Native: 1535, Java: 56 (Android version: 2.3.5);
Samsung Galaxy Prevail, Native: 633, Java: 25, (Android version: 2.2.2);
HTC glacier, Native: 1613, Java: 59, (Android version: 2.3.5);
Motorola Droid X, Native: 1573, Java: 70, (Android version: 1.4.0)
ends
Sign in to Reply