SAN FRANCISCO—Advanced Micro Devices Inc. (AMD) and a computing performance benchmarking consortium traded shots in the form of public statements Tuesday (June 21) after AMD withdrew from the group.
AMD resigned from the consortium— Business Applications Performance Corp. (BAPCo)—and said it would not endorse one of the benchmarks it publishes, the SYSmark 2012 (SM2012).
"Technology is evolving at an incredible pace, and customers need clear and reliable measurements to understand the expected performance and value of their systems," said Nigel Dessau, senior vice president and chief marketing officer at AMD, in a statement. "AMD does not believe SM2012 achieves this objective."
BAPCo fired back with a statement of its own later in the day which claimed that AMD supported more than 80 percent of the SM2012 development milestones and refuted an accusation that BAPCo threatened to expel AMD, which was made by Dessau in a blog posting on AMD's website.
SM2012 is the latest version of a benchmark that compares PC performance based on real-world applications, including Microsoft Office, Adobe Creative Suite, Adobe Acrobat, WinZip, Autodesk AutoCAD and 3ds Max, and others. Previous versions include SYSmark 2007 and SYSmark 2004.
Dean McCarron, principal analyst at market research and consulting firm Mercury Research, said disagreements between hardware makers and firms that create benchmarks happen from time to time. "Occasionally there have been some disputes—and it looks like this is one of them—when the testing disagrees with what one of the hardware manufacturers have done," McCarron said.
According to McCarron, BAPCo originally had a tight relationship with Intel and was actually housed within Intel at the beginning. Over time, he said, more companies subscribed to BAPCo and it became a true consortium, he said. "I'm not saying that there is any specific bias, but that is the genesis [of BAPCo]," McCarron said.
In his blog posting, dated Tuesday, Dessau wrote that AMD has argued for about a year that the SYSmark benchmark is misleading with respect to today’s commonplace applications. The heart of AMD's argument is that the SYSmark benchmark is comprised of unrepresentative workloads that ignore the importance of heterogeneous computing—benefiting Intel—and generates misleading results, he wrote.
Dessau wrote that AMD's resignation from BALCo was not about long-running antagonism between AMD and Intel, but about fairness, relevance and openness. He called for other firms to join with AMD to create a new consortium to rival BAPCo.
"BAPCo is disappointed that a former member of the consortium has chosen once more to violate the confidentiality agreement they signed, in an attempt to dissuade customers from using SYSmark to assess the performance of their systems," BAPCo said in its statement.
BAPCo refuted Dessau's acquisition that the consortium threatened to expel AMD despite what it called "previous violations of its obligations to BAPCo under the consortium member agreement."
BAPCo said it believes the performance measured in each of the six scenarios in SM2012 "fairly reflects the performance that users will see when fully utilizing the included applications."
BAPCo's membership roster includes high-tech heavyweights such as Dell, Hewlett-Packard, Hitachi, Intel, Lenovo, Microsoft, Samsung, Seagate, Sony and Toshiba, among others.
One of the more common "hardware intensive" applications for consumers is video and audio en/decoding. I have found AMD to be superior (like 2x) in these applications. This activity is typically done by me at home under horribly prejudicial circumstances. Two computers sitting side by side are often employed to do the same task at the same time. At work I do mostly CAD, programming, Excel, and other engineering type stuff and the computers that I've used have been 100% Intel. This is typically because Intel generally seems to win the clock speed award and our IT department likes things like "3GHz is faster than 1.8 Ghz." When my oldest graduated from high school several years ago I took a old copy of Performance Test "Passmark" that was bundled with some Norton stuff into fry's and measured what was on the floor (and in my price range). A Toshiba laptop with a 3.2GHz (as I recall) Pentium 4 won the day.
I guess we need a review of the performance tools...
From the BAPCo Spokesman:
"BAPCo fired back with a statement of its own later in the day which claimed that AMD supported more than 80 percent of the SM2012 development milestones..."
If AMD and Intel do not agree to 100% then the benchmarks are simply USELESS since far less than 20% of a companies CPU or GPU extensions could mean the difference between looking extremely fast or slow.
This has ALWAYS been biased toward Intel and now that AMD has resigned from the SysMark Panel it pretty well makes SysMark a tool for only distinguishing one Intel system from another.
This is LONG overdue since Intel has always held the cards (including the ones up their sleeves) in the SysMark Benchmark.
SysMark has now made itself meaningless and I sure wouldn't put out money for it, nor will I rely on their results.
Word is that AMD engineering considers this false politicking on the part of its own management - driven more as attempt to make poor performing CPU's look less poor than from driving some real-world performance value from its fushion graphics APU.
I wonder why AMD would have joined in the first place? I buy PCs for home and work and have never looked at the benchmarks. I wonder if anybody really cares (except marketers?). I have often thought of AMD as the most cost effective tradeoff between price and performance, Intel has performance but at a higher overall price (in my limited experience - but perception drives future buying decisions).
benchmarks were all the time selective by their specific design. if you know your targets then you are more or less able to favour one target and let another drop in values up to some degree.
running homogenous tests by principle is correct as it reflects some of the use cases.
running in-homogenous tests by principle is correct as well as it reflects some of the use cases.
as todays use cases include both scenarios its up to the user to decide which particular use case will match better to which of the available test cases. if one of the major test cases is not made available then this is definitely a lack that conflicts with the interests of the user for doing a solid choice on his own.
i suppose AMD is assuming it would have a some advantage for their models in the currently left out test cases. it must not be that AMD is the fastest even in that cases but it might be the case that for users that care about price/performance ratio (or efficiency or total cost of ownership) this could contribute to a significant design win for AMD.
what would you do if you have joint a group that does stick to what it has and wont move ahead into providing things that you requested for simple and sane reasons (amongst others things that those group generally does). a group that is "marketing intel products" and nothing else whilst revoking giving the end user the tools for doing the best choice is probably something AMD does not see any sense in.
In days of Landmark & PM evolutions of benchmarks thrived on higher performance with every revision. Doing so sold revisions.
Through Wintel, benchmarks thrive on ability to sell Intel intra platform because that cluster commands majority of its own value streams.
So if youíre a member of any group deriving Intel values, your job security is vested in that environment.
As UG Evangelist for Cyrix, NexGen, AMD, Centaur it was common for new platforms to outperform Intel with every introduction.
Cyrix FSB & Nx586 PII replacement ran circles around Pentium & Pro ahead of PII. Nx586 delivering 30% faster run time. An advantage PC media on Intelís Inside take was quick to reconfigure back too Intelís favor.
In period BAPCO was suspect in Intelís pocket. Competitors sought greener pastures.
Intel solution was having PC media place competition in a Public Relationís Parity Rating System. One of the most sophisticated competitive manipulations to undue competition ever witnessed.
Always weighting applicationís code to favor Intel volumes, how often has benchmark stake been moved back following competitive platform evolution? Every time in a game rigged to favor Intel system sales.
AMD concerns today address that age old phenomena of Intel intra platform replacement.
Against Intel Network, to compete means defining a differentiated space; including for benchmarks. And constituentís required to define them based on their use scenarios.
Finally, for majority, benchmarks simply do not apply over the utility benefits of replacing a five year old system. In this purchase environment decisions are made on utility values not benchmarks.
Ultimately there is only one benchmark that counts; real time applications test in the actual work load environment. All other scenarios are questionably fictitious.
Looking towards this kind of concerns and fights it seems that the future will come up with a complete solution provider for both the hardware and software, I see the merger of ARM and Google for this scenario.
As you have mentioned, AMD might have opted out of BAPco because they are afraid of Intel...possible. But again, in the recent past AMD was seen to be successful in grabbing some (small for Intel) market share from Intel and they should be rather confident than being afraid.
Another thought occurs to me, can we just eliminate the possibility of AMD being suspicious about Intel's influence on BAPco, which could be obviously greater, given the fact that Intel still holds 80% of the market share and they could see an increasing threat from AMD in the PC market?
Join our online Radio Show on Friday 11th July starting at 2:00pm Eastern, when EETimes editor of all things fun and interesting, Max Maxfield, and embedded systems expert, Jack Ganssle, will debate as to just what is, and is not, and embedded system.