GSMArena feature labs: The tests

Camera compare tools

Our Photo/Video Compare tools are designed to give you an idea of how well a given camera performs in real-life conditions, and how it stacks up against its peers. Being able to look at the images produced by up to three devices side-by-side is immensely helpful in figuring out how good a given cameraphone actually is.

While we still take "real-life" outdoor photos with all of the devices we review (which you can check out in the reviews themselves), those are frequently at the mercy of the weather, and not as easy to reproduce all year round as we would like. That's why we designed this test to be as objective as possible - all samples are taken in controlled conditions in our studio, at the same time of day under the same lighting.

The tools allow you to judge a cameraphone's performance not only in terms of resolved detail, but also image processing (noise reduction, sharpening) and camera lens distortion.

With the Photo Compare Tool, you'll be able to choose from one of three different charts, and up to three devices to compare. Each device is listed with its native resolution, but you can scale all the sample photos to see how cameras of different resolutions compare against each other. A good 8MP camera, for instance, will typically be sharper when scaled down to 5MP, but sometimes the lower-res camera will win out when you equalize the resolution.

Apple iPhone 4S in the Photo Compare Tool

Our Video Compare Tool allows you to compare still frames taken from videos captured with the phone. We've set up two test scenarios: one under normal lighting conditions and one under low-light to give you an idea of how the handsets fare in different environments. The motion in the Ferris wheels will allow you to see how much fine detail survives the post-processing, as well as the compression levels, noise levels and so on. Of course, a still frame doesn't tell the whole story - there's also framerate, rolling shutter effects and other considerations which the tool doesn't cover, but it's still a good starting point.

We also have our resolution chart shot on video. It shows how much of the available resolution is retained when shooting video. Just like in the Photo Compare Tool, the three charts are accessed by clicking on their names below the main sample image. Other tested devices are listed alongside their natively supported video recording resolution.

Nokia N9 in the Video Compare Tool

For both tools, at the bottom of each device selection field you'll find a list of reference cameras available, which allow you to see how a particular cameraphone compares to a professional DSLR unit, such as our in-house Canon EOS 5D Mark II. You'd be surprised at how well certain cameraphones stack up against these heavy hitters.

Keep in mind that only devices reviewed after November 2010 will have compare tests available, and only cameras with 720p video recording and higher will have video compare.

Test scene update, Nov 25, 2015

We have updated our Photo and Video quality comparison tools with a new setup and a bit of extra functionality. Those two were launched five years ago, but given the improvement smartphone cameras have achieved since, we felt it was about time for an update.

For one, over the years our previous color charts had started to fade (they were losing color so we had to retire them). Also, smartphone cameras became so good they surpassed the limits of the print in those two (they were printed at a certain DPI, it turns out it was too low for a modern high-end camera).

So we added a new setup to the testing procedure, you'll find it in the comparison tools as "Studio setup". It is located in a photo studio, where the sun won't damage it for years, and it's made up of physical objects with very fine texture. As cameras continue to get better, they will resolve more detail in the objects, but they will never hit an upper limit (there's no DPI here, just real detail). We've retained the ISO 12233 poster.

The first two camera samples (the ISO chart and the Studio setup) are taken with the studio lights at full blast, hence they're at the lowest ISO for the cameraphone. We hand-pick the best image from many samples, to make sure it shows the maximum resolving power of the camera and how it handles real-world textures.

The third sample is of the same Studio setup but we reduce the light in the room to a controlled value of 28 lux, measured at the poster. We've added the ISO and shutter speed data from the photo's EXIF as an additional reading of how well the camera performs.

Our legacy Gray and Color posters remain available next to the new images and they let you compare older phones, which weren't around when we created the Studio setup.

Samsung Galaxy S20 Ultra 5G against the iPhone 11 Pro Max and the Note10+ in our Photo compare tool

We have also retired the Ferris wheel setup for our video quality comparison tool. Instead, we've switched to shooting the same scenes as in stills. As before, we're picking a still frame from the video to represent the phone's ability to capture detail and to give an idea of the colors and processing.

Samsung Galaxy S20 Ultra 5G against the iPhone 11 Pro Max and the Note10+ in our Video compare tool