Apple releases statement over Consumer Reports MacBook Pro battery life tests

Last month, Consumer Reports released their report for the new 2016 MacBook Pro testing, which featured rather disappointing battery life results. But perhaps more perplexing was the incredible inconsistency in the results, with battery life varying from 3.75 hours to 19.5 hours.

Since then, Apple has been working with Consumer Reports to figure out what could be the source of this discrepancy and inconsistency with its own results. According to Apple's statement, the reason is two-fold, with the first being the way the notebooks were tested and the other being a bug in Safari.

The first one is not much of an issue but rather the way Consumer Reports runs battery life tests on its computers. They open a set of locally stored websites on the default browser, set the display to 100 nits brightness, and let the battery run dry as the computer cycles through the websites. The goal here is to mimic the real world usage and opening and closing websites is a pretty reliable way to do it. But to make sure all the machines are comparable, Consumer Reports also disables some of the features on them, which includes caching of website data.

Apple identified this as one of the problems, since the figures that the company provides are taken into account caching of webpages, which is the default behavior in Safari and Consumer Reports did deviate from the default behavior, not just for Safari but for users in general, in order to run their tests.

The other issue was caused due to the first, as Consumer Reports' disabling of cache in Safari triggered a bug in reloading icons. Apple has since patched the bug and the fix will be released to customers soon.

After enabling the cache for the browser and using the patched software, Consumer Report' battery life tests for the MacBook Pro models were on par with Apple's claims. They will be publishing the full results soon.

The following is Apple's statement on the situation.

“We appreciate the opportunity to work with Consumer Reports over the holidays to understand their battery test results. We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life. We have also fixed the bug uncovered in this test. This is the best pro notebook we’ve ever made, we respect Consumer Reports and we’re glad they decided to revisit their findings on the MacBook Pro.”

Source