Design Con 2015
Breaking News
Comments
Newest First | Oldest First | Threaded View
<<   <   Page 2 / 2
TarraTarra!
User Rank
CEO
Evil ICC engineers
TarraTarra!   7/12/2013 7:37:03 PM
Sounds like the intel compiler folks found a way to fool the benchmark by bypassing a whole lot of instructions.

 

This is nothing new with intel compilers. They seem to have an army of engineers making specific release for specific benchmarks.

 

All is fair in love and benchmarking?

mcgrathdylan
User Rank
Blogger
Re: Do we really care what the benchmarks say?
mcgrathdylan   7/12/2013 6:18:50 PM
NO RATINGS
I agree with your point about mobile phone users not giving a hoot what kind of processor their phone is running. Obviously there are some that do, many of which read EE Times, but the vast majority of people buying smartphones could care less.

 

DMcCunney
User Rank
CEO
Re: Do we really care what the benchmarks say?
DMcCunney   7/12/2013 5:59:51 PM
NO RATINGS
Oh, I can see end-users caring about benchmarks.  I just don't think most will use  them to make a purchase decision.  They will use them to validate the decision they already made.  The AnTuTu bechmarks are the sort of thing I can see a smartphone owner point to and say "Look how much better the benchmarks are for the phone I bought than they are for your choice." (with the implicit sub-text "I made a better choice than you did, so I'm smarter and cooler than you.")

And I can see cases in the industry where benchmarks will be used, and there will even be agreement they should be used.  I just hope for better understanding of what the benchmarks measure, how they measure it, and what the results actually mean.  (This may be wishful thinking on my part.)

I also wonder how many mobile device users are really aware of what processor is under the hood, or cares if they know?  As you say, they are buying an eco system, and what is available in that eco system will be far more important than what that eco system runs on.  There are probably device owners who are aware that Intel and ARM are battling for share in the mobile device market, but I really doubt anyone will say "I'm running ARM, and you're running (yuck!) Intel!  You're a real dweeb!"

 

 

JimMcGregor
User Rank
Blogger
Re: Do we really care what the benchmarks say?
JimMcGregor   7/12/2013 5:08:08 PM
NO RATINGS
Your point is well taken and I completely agree. End-users don't care about benchmarks. PC generation and pre-PC generation users usually look for common brands and features that they prefer. The mobile generation looks for something new. Benchmarks really come into play with the OEMs and carriers that are making decisions about what technology and products select. So, they still play a role within the industry, even if we all agree that they should not.

And, when we really look at an ARM or Intel processor, the decision is very similar to selecting the OS - what ecosystem do you want? Because, a processor alone does not make a successul product.

mcgrathdylan
User Rank
Blogger
Re: Wow.
mcgrathdylan   7/12/2013 2:58:29 PM
NO RATINGS
Jim, it seems as though you kicked up a hornets nest and forced AnTuTu to take another look at the benchmark. This is an example of great analysis that apparently caused action. Kudos to you Jim.

Questioner
User Rank
Rookie
Wow.
Questioner   7/12/2013 2:09:54 PM
NO RATINGS
Great Post. Fantastic to see that EE Times article was influential enough for Antutu to update their benchmark, though it does cast doubt on the value of traditional benchmarking. Also a great lesson not to 'follow one number' to judge the capability of a smartphone.  I'm sure Intel isn't too happy about such a post, so kudos to EE Times for sticking with the facts/data.

DMcCunney
User Rank
CEO
Do we really care what the benchmarks say?
DMcCunney   7/12/2013 2:00:21 PM
NO RATINGS
There are two target audiences for benchmarks: engineers and end-users.

Engineers have price/performance targets they want to meet when designing systems, and benchmarks are metrics that tell them where they are on meeting the targets.

End-users will see benchmarks as ways of comparing systems.

But in the smartphone market, where the AnTuTu bechmarks seem to be popular, the question is what decisions are made on the basis of them.

Smartphones are better seen as fashion accessories than as tech.  Most buyers are interested in cool.  Smartphones are status markers, and the incentive will be "My phone is cooler than yours!" 

What makes a phone cool?  Brand will be critical.  iPhone buyers aren't buying iOS, they're buying Apple.  There are scads of Android phones, so while Android is cool, it's a common denominator, and just running Android is not a deciding feature.  Windows Phone is not cool, and that may be the biggest challenge Microsoft and Nokia have in getting a share of the market.

The smartphone market reminds me of the movies.  In the movie business, you are as good as your last hit picture, and if your studio hits a dry patch and doesn't have hit pictures for a while, you may go out of business.  The smartphone market is similar.  Motorola had a hit with the Razr, didn't have a followup hit, and was rumored for a bit to be looking at getting out of the smartphone business.

I'd bet that most folks who run the AnTuTu benchmarks are measuring performance on the phone they already bought for other reasons, and aren't making a decision on which phone to buy based on them.

Given that, how much should anyone care what the AnTuTu benchmarks say?

 

<<   <   Page 2 / 2


Top Comments of the Week
Flash Poll
Like Us on Facebook

Datasheets.com Parts Search

185 million searchable parts
(please enter a part number or hit search to begin)
EE Life
Frankenstein's Fix, Teardowns, Sideshows, Design Contests, Reader Content & More
<b><a href=Betajet">

The Circle – The Future's Imperfect in the Present Tense
Betajet
5 comments
The Circle, a satirical, dystopian novel published in 2013 by San Francisco-based writer Dave Eggers, is about a large, very powerful technology company that combines aspects of Google, ...

Max Maxfield

Recommended Reads From the Engineer's Bookshelf
Max Maxfield
27 comments
I'm not sure if I read more than most folks or not, but I do I know that I spend quite a lot of time reading. I hate to be idle, so I always have a book or two somewhere about my person -- ...

Martin Rowe

Make This Engineering Museum a Reality
Martin Rowe
Post a comment
Vincent Valentine is a man on a mission. He wants to make the first house to ever have a telephone into a telephone museum. Without help, it may not happen.

Rich Quinnell

Making the Grade in Industrial Design
Rich Quinnell
16 comments
As every developer knows, there are the paper specifications for a product design, and then there are the real requirements. The paper specs are dry, bland, and rigidly numeric, making ...

Special Video Section
The LT8640 is a 42V, 5A synchronous step-down regulator ...
The LTC2000 high-speed DAC has low noise and excellent ...
How do you protect the load and ensure output continues to ...
General-purpose DACs have applications in instrumentation, ...
Linear Technology demonstrates its latest measurement ...
10:29
Demos from Maxim Integrated at Electronica 2014 show ...
Bosch CEO Stefan Finkbeiner shows off latest combo and ...
STMicroelectronics demoed this simple gesture control ...
Keysight shows you what signals lurk in real-time at 510MHz ...
TE Connectivity's clear-plastic, full-size model car shows ...
Why culture makes Linear Tech a winner.
Recently formed Architects of Modern Power consortium ...
Specially modified Corvette C7 Stingray responds to ex Indy ...
Avago’s ACPL-K30T is the first solid-state driver qualified ...
NXP launches its line of multi-gate, multifunction, ...
Doug Bailey, VP of marketing at Power Integrations, gives a ...
See how to ease software bring-up with DesignWare IP ...
DesignWare IP Prototyping Kits enable fast software ...
This video explores the LT3086, a new member of our LDO+ ...
In today’s modern electronic systems, the need for power ...