Under the microscope: Analyzing the Xiaomi Mi 8 performance

We recently published our in-depth Xiaomi Mi 8 review, complete with a full set of tests: display, audio quality, performance, you know the drill. For the most part, it was business as usual, except for certain performance aspects.

Even though the phone comes with the latest Snapdragon 845 chipset, in most synthetic benchmarks it only seemed to perform at the level of a Snapdragon 835 instead. A lot of you were confused by the performance numbers we published. Some even thought our review device or its software was to blame.

So we went ahead and re-tested the handset under more favorable conditions - by applying extra cooling to its back. We reported our findings in the Performance section of the review driving the point that the software is not to blame, it's the device's thermal management.

Even so, we felt it would be beneficial to go into a bit more detail about what happened with the Mi 8 and how we went about investigating it. Also, a few words about what can be done and what this means for the end-user. Long story short:

Thermal throttling

After seeing the Snapdragon 845-equipped Xiaomi Mi 8 get benchmark scores in the Snapdragon 835 ballpark, it was pretty clear that the handset was capping its performance, to adhere to some thermal constraints. At least, that was the first theory, since there were no obvious signs of any software issues, bugs or benchmark incompatibilities. Plus, MIUI has a pretty solid track record, when it comes to proper hardware optimization.

So, we broke out our home-made active cooling solution. It is comprised of a Peltier element, sandwiched between two beefy CPU coolers. This allows us to rest a phone on a flat actively cooled surface while using it. You could always run these tests inside a fridge to the same effect but that's nowhere as convenient, isn't it?

AnTuTu 7

Higher is better

• vivo NEX S
  287081
• Xiaomi Mi Mix 2S
  270814
• Xiaomi Mi 8 (w/ extra cooling)
  269276
• OnePlus 6
  264200
• Samsung Galaxy S9+ (Snapdragon)
  264044
• HTC U12+
  263696
• LG G7 ThinQ
  259393
• Sony Xperia XZ2
  259244
• Samsung Galaxy S9+
  246660
• Xiaomi Mi 8
  217298
• Huawei Mate 10
  214037
• Huawei P20 Pro
  209884
• Nokia 8 Sirocco
  209577
• Google Pixel 2 XL
  203119
• LG V30
  182374

Just to be clear, we sometimes use the cooler to chill phones in between benchmark runs, not during, so that we don't have to wait as long between runs. If you leave the phone on the cooler, the results are understandably a bit skewed and could only apply if your natural every-day environment is a meat freezer. Still, it helped us illustrate a point, namely, with a cooler applied, the Mi 8 performed exactly as expected of a Snapdragon 845.

AnTuTu seems to show the variance between the cooled and un-cooled runs best, but GeekBench was also affected. Especially the multi-threaded workload segment.

GeekBench 4.1 (multi-core)

Higher is better

• vivo NEX S
  9160
• Xiaomi Mi Mix 2S
  9158
• Xiaomi Mi 8 ((w/ extra cooling))
  9073
• OnePlus 6
  9011
• HTC U12+
  9001
• Samsung Galaxy S9+
  8883
• LG G7 ThinQ
  8865
• Xiaomi Mi 8
  8494
• Sony Xperia XZ2
  8466
• Samsung Galaxy S9+ (Snapdragon)
  8349
• Nokia 8 Sirocco
  6725
• Xiaomi Mi 6
  6719
• Huawei P20 Pro
  6679
• Huawei Mate 10
  6625
• Google Pixel 2 XL
  6428
• LG V30
  6365

GPU performance was naturally also affected by the thermal-throttling issues, but not quite so consistently. When rendering graphics in the off-screen test, the Adreno 630 gets pushed to the brim of its performance. Naturally, since the GPU is part of the same chipset as the CPU cores, thermal limitations kick in right away, and you get throttling.

GFX 3.1 Manhattan (1080p offscreen)

Higher is better

• Xiaomi Mi Mix 2S
  61
• Samsung Galaxy S9+ (Snapdragon)
  61
• Xiaomi Mi 8 (w/ extra cooling)
  60
• HTC U12+
  60
• vivo NEX S
  60
• OnePlus 6
  58
• LG G7 ThinQ
  57
• Sony Xperia XZ2
  55
• Xiaomi Mi 8
  53
• Samsung Galaxy S9+
  47
• Google Pixel 2 XL
  42
• Xiaomi Mi 6
  41
• LG V30
  41
• Huawei P20 Pro
  40
• Huawei Mate 10
  38
• Nokia 8 Sirocco
  34

On-screen, however, the situation is a bit different, since some other performance limitations come into play. The Mi 8 is obviously hitting other limitations, way before the thermal threshold of the SoC, which cap the displayed frame-rate at 50 fps, in the particular test, regardless of cooling.

GFX 3.1 Manhattan (onscreen)

Higher is better

• Xiaomi Mi Mix 2S
  55
• vivo NEX S
  55
• OnePlus 6
  55
• Sony Xperia XZ2
  51
• Xiaomi Mi 8
  50
• Xiaomi Mi 8 (w/ extra cooling)
  50
• Xiaomi Mi 6
  41
• Huawei P20 Pro
  37
• Samsung Galaxy S9+ (Snapdragon)
  34
• HTC U12+
  33
• LG G7 ThinQ
  30
• Samsung Galaxy S9+
  24
• Huawei Mate 10
  23
• Google Pixel 2 XL
  21
• LG V30
  19
• Nokia 8 Sirocco
  18

This is mostly good news for potential owners since this is the test more closely representative of real-world usage. It means that you probably won't be losing any frames in your games on the Mi 8. At least not until you play a pretty extended session. And practically every passively cooled mobile device out there is subject to this kind of performance loss.

The CPU performance drop could potentially be felt easier in real life, but only with a very specific subset of tasks. For instance, transcoding video, archiving or de-archiving large pieces of data - long-lasting, CPU intensive loads. Since most users don't really do those a lot, it's fair to assume that the Mi 8 will perform comparably to another Snapdragon 845 flagship in a real-world scenario.

Some of you might be happy with this conclusion. But if you really want to know what's happening behind the scenes (we certainly did), read on, since we did a few other tests after we published our review.

Going deeper

For this next portion of the testing, we grabbed a couple of really handy apps off the Google Play Store - CPU Throttling Test and CPU Float.

One conducts a prolonged stress test on the CPU while monitoring core frequencies and temperature. Then spits out an average, min and max estimated performance metric and even calculates how much performance the device lost over time, due to throttling. The second app is a simple overlay to see the real-time CPU core frequencies and a second thermal reading. We just wanted to double up the readings, just to make sure we are not missing anything.

So, first, here is the Xiaomi Mi 8, running the stress test for five minutes at room temperature, with no extra cooling. The screenshots are taken at about 30 second intervals, so you can see how things progress in time.

Xiaomi Mi 8: first, un-cooled run

We can clearly see that as early as two minutes into the test, things start to taper off quickly. Performance drops, and it doesn't really recover until the very end of the run. We can note, from the breakdown at the end, that the Mi 8 ended up with about 76% of its maximum performance in the short 5 minute period.

That's all fine and dandy, but these numbers mean little without a reference point. So, we grabbed the OnePlus 6 and ran the same software. OnePlus is known for squeezing the maximum performance possible out of their chips, and the OnePlus 6 is a frequent chart-topper among Snapdragon 845 devices. We also have pretty solid empirical evidence that it doesn't really have thermal throttling issues.

Here is its extended 6-minute run.

OnePlus 6: un-cooled run

OnePlus 6: un-cooled run

We can clearly see the effects on longer workloads on a chipset. Or rather, pure heat. But both the average values and the percentage of maintained performance, in the end, are better. That is roughly the performance graph we can expect from a Snapdragon 845 chipset.

So what's happening? Well, you might notice that during the tests, the power-efficient quartet, as well as the more powerful four CPU cores of the OnePlus are maintaining a significantly higher temperature than those of the Mi 8. What that essentially means is that OnePlus raised the thermal limit of the Snapdragon 845 a bit higher. Or if you look at it from another angle, Xiaomi lowered the one on the Mi 8.

Some explanation might be necessary here. A typical CPU basically adheres to at least a couple of thermal constraints automatically. One constitutes a maximum safe temperature for operation. If exceeded, the chip might suffer some damage - physical and often irreversible. This is generally set by the chip vendor, and when reached, the hardware usually takes drastic measure to ensure its well-being, like a forced shutdown or a restart.

Then there are one or more thermal limits that set a temperature, that when reached, trigger an automatic measure to lower the performance of the chip in an effort to prevent it from getting warmer or at least reduce the pace at which it is accumulating heat. This is the threshold we are interested in and the ones phone manufacturers typically have control over.

Now, there might be numerous reasons for a manufacturer to decide to go easier on the chip and set a lower thermal threshold. Longevity of components comes to mind. Also, with a device as compact as a smartphone, you have to think about the overall package heat and other close-by components that might raise the overall internal heat level, while active alongside the chipset. For instance, the Mi 8 has a fancy new dual frequency GPS chip. It could potentially be the source of some extra heat as well.

Before wrapping things up, we will show you the effects of active cooling on the Mi 8. The eagle-eyed among you might have noticed a small detail, we ourselves were initially a bit perplexed about. Despite the obvious manifestation of thermal throttling, the tests seemed to always show all eight cores maintaining their maximum frequency.

This doesn't really add up since the way throttling typically works is by lowering the current operating frequency of the core. Without getting overly technical and opening up a can of worms, known as the Android CPU governor, let's just say that the reported values within the app are effectively maximums. Hence, these are not really representative of the current CPU frequencies, which are constantly changing. We just didn't have the means to monitor them.

This is the reason we used the CPU Float app. It is the small overlay window, covering part of the graph in the following screenshots. Unfortunately, this didn’t help much, as it appears that the precise CPU frequency values aren't really available without root access.

Xiaomi Mi 8: second, un-cooled run

Xiaomi Mi 8: second, un-cooled run

This time we let the Mi 8 run for a full 10 minutes. We had initially thought that 5 minutes would be enough to showcase the full extent of the performance loss in time. It turns out we were wrong. Letting the torture go for 10 minutes left us with as little as 67% of the maximum performance.

Since we felt pretty good about the temperatures the chipset was maintaining through, we let the test run for another 10 minutes and are happy to report that this is about the minimum performance level we observed.

Last, but not least, just to verify the extent the thermal management policy affects the Mi 8 performance, we also did a 10 minute, actively-cooled run. Same parameters, otherwise. See for yourselves:

Xiaomi Mi 8: actively cooled run

Xiaomi Mi 8: actively cooled run

All that said, we feel like there should be an interesting conclusion to make here. Well, this was an exciting, even if time-consuming way to illustrate a different approach to thermal and power management.

If there is one real takeaway from this, it has to be the same one we typically mention in the Performance sections of our reviews - benchmark numbers are a cool tool for comparison, but don't always translate well to real-life scenarios. In reality, the Mi 8 performs as smooth as the next 2018 flagship.

You might not be getting the full raw performance potential of the Snapdragon 845, but there are other benefits to having the CPU run cooler. Plus, manufacturers often switch-up frequencies, thresholds and performance parameters on their devices with software updates so this pre-programmed behavior may change over time. Most of the time, it's for the benefit of the average user, one way or another. It's just how the industry works.

Special thanks to HonorBuy for providing the review unit.