By disassembling the benchmarking app, changing only the package name, and reassembling it, we could run the app without the CPU knowing we were running a benchmark app. So we slapped together "Stealthbench," a renamed version of Geekbench 3. A bit of testing showed that the device's boosted benchmark mode is triggered by the package names of the most popular benchmarking apps-loading Geekbench, for example, starts this mode. If we could defeat this behavior, we could have before and after benchmark numbers and thus see just how deep the rabbit hole goes. While it's difficult to determine every bit of special programming that affects the CPU while a benchmark is running, one sure-fire way to see what's going on is to trick the phone into not entering a special "benchmark mode" during a benchmark. Benchmarks exist to measure the performance of a phone during normal usage, and a device should never treat a benchmark app differently than a normal app. Stopping the CPU from idling shouldn't in and of itself affect the benchmark scores a whole lot, so this was our first sign that something was wrong. However, if you load up just about any popular CPU benchmarking app, the Note 3 CPU locks into 2.3GHz mode, the fastest speed possible, and none of the cores ever shut off. The above picture shows how differently the CPU treats a benchmarking app from a normal app. Normally, while the Note 3 is idling, three of the four cores shut off to conserve power the remaining core drops down to a low-power 300MHz mode. The smoking gun here is CPU idle speeds, which can be viewed with a system monitor app while using the phone. Right: The Note 3 in a benchmarking app, unable to idle. We also found a way to disable this special CPU mode, so for the first time we can see just how much Samsung's benchmark optimizations affect benchmark scores.Įnlarge / Left: The Note 3 idling normally, with 3 cores off, and one in a low-power mode.
What makes one Snapdragon so different from the other?Īfter a good bit of sleuthing, we can confidently say that Samsung appears to be artificially boosting the US Note 3's benchmark scores with a special, high-power CPU mode that kicks in when the device runs a large number of popular benchmarking apps. Samsung did something similar with the international Galaxy S 4's GPU, but this is the first time we've seen the boost on a US device. A quick comparison of its scores to the similarly specced LG G2 makes it clear that something fishy is going on, because Samsung's 2.3GHz Snapdragon 800 blows the doors off LG's 2.3GHz Snapdragon 800.
#Benchmarks deleted geekbench over cheating pro
Getting back to the original issue, I was really excited when a Windows PC snagged the top spot on the Geekbench Browser after months of Mac Pro domination.We noticed an odd thing while testing the Samsung Galaxy Note 3: it scores really, really well in benchmark tests-puzzlingly well, in fact.
#Benchmarks deleted geekbench over cheating how to
In the meantime if you’re interested in running benchmarks on virtual machines, or in understanding the issues behind timers on virtual machines, VMWare has a great white paper that explains the issue and how to work around it under VMWare. That said, it’s still an issue, and I’m investigating ways of detecting and correcting inaccurate timers on virtual machines. People who use Geekbench competitively (like the folks involved in the Forum Wars) know that Geekbench scores from virtual machines aren’t always accurate. I do a bit of maintenance and keep virtual machine scores out of the top Geekbench scores list. For example, if a benchmark takes a second to run, it might look like it only took half a second to run on a virtual machine, which makes the virtual machine look twice as fast. Why I’m removing cheaters is pretty obvious, but why am I also removing virtual machine results? Well, virtual machines have problems emulating certain bits of hardware that make it hard, if not impossible, to implement accurate high-resolution timers. The only results I’ve removed have been results forged by people trying to cheat or results from virtual machines. While it’s true I’ve removed results from the Geekbench Browser, I’ve never removed results to favor one platform over another. When a video claims Geekbench is biased, though, the fun goes out the window: It’s great to see people getting excited about benchmarks in general and Geekbench in particular.
Most of the videos are fun and I enjoy watching them. I’ve talked about Geekbench on YouTube before.
0 kommentar(er)
