A recent AnTuTu benchmark that put Intel's latest Atom ahead of ARM-based processors from Nvidia, Qualcomm, and Samsung made headlines. But something seems odd.
A further evaluation of the AnTuTu benchmark reveals significant changes in scores just from revisions in the benchmark, which is not uncommon. However, the following figure shows a disproportionately higher increase in performance for the previous-generation Intel Atom processor, the Z2460, used in the Motorola RAZR i, when compared to the Samsung Galaxy S4. Note that the Motorola RAZR i data was used because measurements using the earlier revision of the benchmark data were not available for the Lenovo K900, but the processors in both products are based on the same CPU core and semiconductor manufacturing process node.
Going from the 2.9.3 version of the AnTuTu benchmark to the 3.3 version, the overall AnTuTu scores increased 122 percent, and the RAM score increased 292 percent for the Intel processor, while the scores for the Samsung processor increased only 59 percent and 53 percent, respectively. This was just from a change in the benchmark test, not the processors. Why did the figures change so drastically for one processor and not the other with the revisions in the benchmark? This raises further concerns about the validity of the AnTuTu benchmark.
The second question is: Why are so many industry observers drawing such general conclusions from one outlying benchmark rather than citing the results from a suite of benchmarks? It is natural for a company to desire to demonstrate its products in the best possible light, but industry observers should be more objective.
Unfortunately, we live in a world where sensationalism rules and headlines seldom tell the entire story or even the truth. In any case, the results of one benchmark, especially one that seems to contradict other benchmarks, seem more suspicious than conclusive. It also seems a bit ironic that performance benchmarks for Intel's next-gen Atom processor called Bay Trail were just leaked, and the only benchmark referenced was AnTuTu once again.
The only clear conclusions that should be drawn are that the leading ARM-based processors still have a performance lead over the latest Intel processor (a recent review of the Samsung Galaxy 3 10.1 by GSMArena came to the same conclusion); all benchmarks should be questioned and none used exclusively; and recent headlines were more sensational than truthful.
To be sure, the battle for dominance in mobile processors is a critical one, especially as many smartphone and tablet OEMs turn to designing their own processors. In the end, however, the battle is much broader than just processor performance or even one type of processor performance. The real battle is on the platform level where performance efficiency, connectivity, and user experience are the clear differentiators.