Two REDMAGIC smartphones removed from 3DMark: benchmark results were unreliable

3DMark is, for a long time, one of the most popular benchmarks for testing smartphones with a series of tests designed to evaluate the real performance of the device.

However, to be tested with 3DMark, smartphones must adhere to specific guidelines set by UL Solutions, the company responsible for developing the test.

Following some anomalies detected after certain tests, UL Solutions decided to remove two REDMAGIC smartphones from the 3DMark list. The smartphones in question would exhibit behavior that does not comply with the guidelines.

The models involved are REDMAGIC 11 Pro and 11 Pro+, which are no longer part of the ranking of the best smartphones tested by 3DMark.

It’s important to note that these smartphones feature the Qualcomm Snapdragon 8 Elite Gen 5 SoC and an advanced cooling system that should allow for optimal chip performance.

In a post on the official blog, it was clarified:

"We tested each model with the public version of 3DMark, available on Google Play, and with a private, renamed version of 3DMark, which is not accessible to the public or manufacturers. In our tests, the scores obtained with the public 3DMark app were up to 24% higher than those of the internal version, even though the tests were identical."

Essentially, the REDMAGIC devices recognize the 3DMark app and alter the behavior of some components to achieve better results, which, however, do not reflect the actual performance.

The benchmark tests would be conducted in a particular high-performance mode, capable of providing a substantial boost in the final results.

UL Solutions added:

"The optional performance modes that users can enable are allowed by our rules, as long as they are disabled by default. A device must run the benchmark in the same way it would run other applications, unless the user has explicitly selected an optional mode."

According to UL Solutions' findings, REDMAGIC smartphones would reach an external temperature exceeding 50° C in some areas of the chassis, in order to achieve the results highlighted in the 3DMark test. For this reason, the company has advised its users not to perform stress tests with these devices. Alternatively, after a test, it will be necessary to cool the device before conducting a new test.