     |
| |
Oh yes, things are really starting to kick off in the arcane world of smartphone benchmarking. First, there came clear evidence of phone makers manipulating scores in apps like AnTuTu and GFXBench, and now a more mainstream benchmarking company, Futuremark, has publicly delisted specific Samsung and HTC phones that it suspects of cheating. Futuremark says that the devices in question -- including the Galaxy Note 3, HTC One and HTC One Mini -- fail to adhere to the fairness policy, which requires that a device treats its 3DMark app just as it would treat any other app, with no tailor-made bursts of performance designed to achieve artificially high scores. Clearly, this bad behavior is just as endemic as we originally feared, so benchmark apps either need to toughen up, as Futuremark appears to be doing, or they need to find entirely new ways of measuring performance.
Filed under: Cellphones, Mobile, Samsung, HTC
Via: HotHardware, The Register
Source: Futuremark
Remember the allegation that Samsung cheats at benchmarks? Despite the manufacturer's semi-denials, there's growing evidence to suggest that not only Samsung, but also a number of other Android phone makers engage in some level of chicanery. Usually, it involves programming a device to temporarily ramp up its performance if it detects the launch of a benchmark test, regardless of the consequences for battery life or processor temperature. This results in a higher score on the artificial test, but one that is unrepresentative of what the device could actually achieve if it had to pace itself for a real-world task that lasted for a longer period of time -- such as a 20-minute bout of gaming.
So far, so bad. However, a startup called GameBench reckons there's another way. Its founders, who previously worked at chip companies like ARM and MediaTek, claim to have developed an "uncheatable" performance test that can be used to corroborate (or refute) the scores from traditional benchmarking apps, and which can help to rank Android phones and devices according to their true gaming capabilities. Although GameBench's app is still in beta and likely won't be released until the first quarter of next year, it has already collected scores for two devices, the HTC One and the Samsung Galaxy S 4. Ironically, as you're about to see, Samsung may actually have less to fear from this cheat-free test than some of its rivals.
Filed under: Cellphones, Mobile, Samsung, HTC
Samsung has come under fire this week for allegedly inflating the benchmark scores for the Galaxy Note 3 among other Android devices, but -- shocker -- the company is not alone in enhancing scores. AnandTech just published a report that shows ASUS, HTC and other companies using the same "benchmark detect" function to artificially bump up the numbers. According to the site, virtually all OEMs run a CPU optimization on at least one of their devices, save for Apple, Google's Nexus 4 and Nexus 7 and Motorola's latest crop of phones. Dishonesty aside, though, what's the big to-do about these inflated scores? AnandTech found that companies notched less than a 10-percent performance boost in AnTuTu and Vellamo. And when's the last time you based your smartphone purchase on benchmark scores alone, anyway?
Filed under: Cellphones, Software, Mobile, Samsung, HTC, ASUS, Google
Via: 9 to 5 Google
Source: AnandTech
Samsung drew criticism for inflating the benchmark scores of Exynos devices earlier in the year, but the company appears undaunted; it's reportedly boosting test numbers for other hardware as well. Ars Technica has discovered that the Snapdragon 800-based Galaxy Note 3 (and possibly the new Note 10.1) includes code that runs all CPU cores at full speed during certain benchmarks. The tweak gives the smartphone a minimum 20 percent higher score in any affected app, or enough to claim an artificially large advantage over an LG G2 using a similar chip. There may also be a graphics boost, Ars says. We've asked Samsung for its take on the findings. Whether or not the company responds, we don't envy its position -- it's hard to form an alliance devoted to accurate mobile benchmarking when you're accused of doctoring results.
Filed under: Cellphones, Mobile, Samsung
Source: Ars Technica