Android 9 Pie review

Android P improvements under the hood

A lot of the improvements Android 9 has to offer live under the hood. Silently, doing its work, hidden-away, behind the scenes. Developers have a lot to look forward to, in particular, with many new exciting native Android API's to leverage in their apps. The specific list hasn't changed much for Android 9 since the release of the first public beta build we already reviewed. Still, progress has been made pretty much across the board.

In no particular order, here are the most essential baked-in developer changes we found in Android Pie:

• New API for display cut-out support, complete with simulations in developer options
• A number of new APIs are added to the autofill framework, as well as some bugfixes. Specifically, there will be improvements to dataset filtering, input sanitization, and compatibility mode.
• Expanding the 8.1 Oreo neural network API with a bunch of new tools ("Pad, BatchToSpaceND, SpaceToBatchND, Transpose, Strided Slice, Mean, Div, Sub, and Squeeze."). Pixel 2 devices will also have support for Qualcomm's Hexagon HVX driver for hardware ML acceleration.
• HEIF and HDR VP9 Profile 2 native support, for both content playback and capture.
• The native camera API can now access two or more stream simultaneously. There are also new parameters for speedier access to the camera stream, native display flash, and OIS implementations.
• Background applications can no longer access the camera, microphone or device sensors. This might explain the absence of a Battery saver location mode since most apps should be restricted from over-using the GPS. The documentation is still a bit fuzzy on the restrictions.
• Support for IEEE 802.11mc, which allows measuring the distance to Wi-FI access points and exact indoor navigation
• General ART virtual machine improvements for faster app start-ups and less system memory usage (up-to-11% reduction on how often DEX files get rewritten across "popular apps")
• Kotlin apps also get a performance boost
• The Developer options menu now has a hidden list of 'feature flags,' just like Google Chrome
• Bug reports can be accessed in the files app
• Apps built for Android 4.1 and lower may not work on Android P. More details should be shared soon.

Benchmarking the Google Pixel 2 XL performance on Android 9

Before we get to the actual numbers, we feel like a little context needs to be laid out. Just to be on the same page here.

First off, like we've said a million times, synthetic benchmark loads and tests are not representative of real-life performance. So, please don't go basing any purchase decision on these numbers alone.

Secondly, while already on its first, official public release, Android 9 Pie is still in its infancy. It will take some more time before it is properly optimized and fully de-bugged. That's just the nature of things, especially with an OS as complex as Android.

That being said, as it currently stands, Android 9 on the Pixel 2 XL (the same unit we used for the official review and our first batch of tests) seems to bring overall CPU scores down a fair bit.

GeekBench 4.1 (multi-core)

Higher is better

• Google Pixel 2 XL (Android 8)
  6428
• Google Pixel 2 XL (Android 9)
  6055

This is true for both single and multi-core loads and the variance is almost identical. Almost suspiciously so.

GeekBench 4.1 (single-core)

Higher is better

• Google Pixel 2 XL (Android 8)
  1915
• Google Pixel 2 XL (Android 9)
  1807

Furthermore, going back to an older GeekBench 4 APK yields pretty much identical results.

GeekBench 4 (multi-core)

Higher is better

• Google Pixel 2 XL (Android 8)
  6084
• Google Pixel 2 XL (Android 9)
  5885

GeekBench 4 (single-core)

Higher is better

• Google Pixel 2 XL (Android 8)
  1999
• Google Pixel 2 XL (Android 9)
  1872

This may look pretty bad at first, but it comes to prove a few important points, mainly having to do with OS and app optimization. We already mentioned that it's going to take some time for Android Pie to be as optimized and streamlined as its predecessor.

The same goes for the apps that you run on top of Google's mobile OS. It's no secret that fragmentation has always been a major problem for Android. It's just an inherent part of the open nature of the OS and the hardware and software ecosystem it fosters. On the software side of things, it's up to developers to work their magic and optimize their creations for new Android versions.

Failing to do so often leads to poor performance. And since benchmarks are apps as well, this applies to them as well. But performance degradation is not necessarily the consequence of poor app optimization and compatibility as a rule.

AnTuTu 7

Higher is better

• Google Pixel 2 XL (Android 9)
  206711
• Google Pixel 2 XL (Android 8)
  203119

Take, for instance, AnTuTu. Unlike its pure-CPU GeekBench relative, it seems to favor Android 9 on the Pixel 2 XL. Just to prove our point, we ran the even older AnTuTu 6, as well. Which, by our previous logic, should be even less optimized and ready to cope with the modern OS.

AnTuTu 6

Higher is better

• Google Pixel 2 XL (Android 9)
  171348
• Google Pixel 2 XL (Android 8)
  170407

At this point in time, we can't really fault pretty much any developer out there for being tardy with their adaptation to Android 9. It's just too early. The OS is barely out into the general public.

Basemark OS 2.0

Higher is better

• Google Pixel 2 XL
  3379
• Google Pixel 2 XL (Android 9)
  3293

This particular set of circumstances has always put Pixel and Nexus users, before them, in a rather unique early-bird position. Sure, you get to play with the latest and greatest Google's mobile OS division has to offer, but you do have to be prepared to live with impaired performance and even broken compatibility with some apps and experiences, on occasion. This is exactly one such occasion.

GPU results are no different. Again, we see a noticeable gap in the on-screen frame rate after the jump from Android 8 to Android 9.

GFX 3.0 Manhattan (onscreen)

Higher is better

• Google Pixel 2 XL (Android 8)
  36
• Google Pixel 2 XL (Android 9)
  27

Just to reiterate our earlier point - you might not feel a difference in real-world scenarios. Then again, your might. Some games might not run, at all, in their current state. Before all this is properly addressed by the app developers, anything is possible, and that has always been a fact of life for early adopters.

GFX 3.1 Manhattan (onscreen)

Higher is better

• Google Pixel 2 XL (Android 8)
  21
• Google Pixel 2 XL (Android 9)
  15

GFX 3.1 Car scene (onscreen)

Higher is better

• Google Pixel 2 XL (Android 8)
  13
• Google Pixel 2 XL (Android 9)
  13

Just in case you were wondering why things appear to level out with the last OpenGL ES 3.1 test, the truth is, we can't exactly be sure, Our best guess is that the increased difficulty of the test itself, caused the Pixel 2 XL to start bumping into some bottleneck, unrelated to the OS version. Or, at least, consistently severe across Android 8 and Android 9.

Basemark ES 3.1 / Metal

Higher is better

• Google Pixel 2 XL (Android 8)
  874
• Google Pixel 2 XL (Android 9)
  766

And just to further prove our point about the uncertain nature of early software, take a look at the BaseMark X results. Just like AnTuTu, it seems to favor Android 9.

Basemark X

Higher is better

• Google Pixel 2 XL (Android 9)
  39372
• Google Pixel 2 XL (Android 8)
  39143

Retesting Google Pixel 2 XL battery endurance on Android 9

Going into the battery test, frankly, we expected to see quite a bit of difference, between our original run and the one with Android Pie. Just for reference, here are the numbers we got in the original Google Pixel 2 review. Back then, the unit was running Android 8.0 Oreo.

After crunching the numbers and then checking them again, we pretty much ended up with the same overall battery endurance for the Pixel 2 XL.

Our endurance rating indicates how long a single battery charge will last you if you use the Google Pixel 2 XL for an hour each of telephony, web browsing and video playback daily. We've established this usage pattern so our battery results are comparable across devices in the most common day-to-day tasks. The battery testing procedure is described in detail in case you're interested in the nitty-gritties. You can also check out our complete battery test table, where you can see how all of the smartphones we've tested will compare under your own typical use.

But, didn't Android Pie introduce a much stricter policy towards background activity? What happened to that bit? Well, the battery management tweaks are definitely a fact. So is the AI-powered Adaptive Battery tech. The pair might, very well, save you from some headaches, dealing with pesky, misbehaving apps.

However, we run our battery tests in the unit's factory default state. Since we are dealing with AOSP here, that means practically no bloat, to speak of and no background activity that isn't already sanctioned by Google. And since the tests themselves run the same app over and over again, the new battery optimizations don't get any chance to shine through as they would once they start to pick up your usage routine. But even without those, we admit we squeezed a few extra hours of standby time after the Android 9 update so there's that.

Verdict

So overall, Android 9 Pie doesn't seem to provide any tangible performance improvements. Instead, it focuses on visual tweaks and quality-of-life improvements more than anything else.

It feels more modern, more in tune with the times, even if not exactly novel. We'll just have to see how much of a trendsetter this iteration of AOSP turns out to be in the future.