Archive for Benchmarks

Samsung To World: We Don’t Cheat On Benchmarks

Posted in Commentary with tags , on October 3, 2013 by itnerd

Let the games begin.

After reports surfaced that Samsung among others have cheated on benchmark tests, the smartphone giant has decided to fire back at those accusations. Here’s what they said on CNet UK:

 “The Galaxy Note 3 maximises its CPU/GPU frequencies when running features that demand substantial performance,” Samsung said in a statement to CNET UK.

“This was not an attempt to exaggerate particular benchmarking results. We remain committed to providing our customers with the best possible user experience.”

Okay. This is pretty black and white with no wiggle room for interpretation. So the question is who’s telling the truth? Samsung or AnandTech? We’ll likely never know for sure. But it is an interesting discussion point as I wouldn’t want to buy a product based on performance and never actually get it because someone cheated.

Leave a comment »

AnandTech Says Almost Everyone Cheats On Benchmarks

Posted in Commentary with tags , on October 2, 2013 by itnerd

Yesterday, I posted a story on the fact that Samsung was being accused of cheating on benchmark tests by AnandTech. Today, AnandTech is saying almost everyone in the Android universe cheats when it comes to benchmark tests. They’ve posted a very lengthy report on this. While I encourage you to read it, I’ll give you the executive summary. It would be a shorter list to tell you who isn’t cheating when it comes to benchmarks. The companies that are running clean are:

  • Motorola
  • Apple

That’s a pretty short list. I Included Apple because according to the report, they don’t cheat either. That implies that you may not be getting the performance that you’re paying for. And it’s likely to become difficult to detect going forward:

Going forward I expect all of this to become more heavily occluded from end user inspection. App detects alone are pretty simple, but what I expect to happen next are code/behavior detects and switching behavior based on that.

That’s not good at all for anyone. I would be interested to hear these companies who do cheat explain their rationale.

Leave a comment »