Xiaomi builds DeepExposure AI to restore detail in over/under exposed photos

Peter, 06 December 2018

Modern smartphone photography is as much a product of what the image sensor captures as it is of what an image processor does with the information afterwards. Xiaomi has presented a paper that aims to solve a common issue of small pixels – limited dynamic range – by employing an AI to fix the exposure of the photo.

But not the whole photo, instead the image is segmented into sub-images and their exposure is adjusted separately. Then the different parts are merged to form the final image.

Take this image for example. The buildings are white, the clouds are white and the overexposed sky is white. The AI, dubbed DeepExposure, does a great job at restoring detail.

The schematic illustration of the algorithm. Firstly, it harnesses image segmentation to obtain sub-images. For different sub-images, it uses different exposures according to the policy network and they are fused together to form the final high-quality image

To teach the AI (based on a Generative adversarial network), the Xiaomi team used the images from the MIT-Adobe FiveK dataset. It contains unedited RAW photos as well as the same photos retouched by five experts (the team used 3,000 images, picking the ones retouched by Expert C).

The network works on low-resolution images – its goal is to come up with the best parameters for classic image filters. Think of it as the AI fiddling with the dials in Lightroom. This simplifies the learning process, but should also speed up image processing.

Retouched images of different algorithms. From left to right, top to bottom: Original input image, DeepExposure I, DeepExposure II, Exposure, FI, Expert C, DPED, CycleGAN, Deep Photo Enhancer and Deep Guided Filter

Source (PDF) | Via

Reader comments

njsilva84
18 Dec 2018
Lcc

Yes, and no. Before explaining why, let me tell you some things that no one cares but that also matter: A "camera" (in smartphones) is a set of a sensor combined with a lens (that can have more or less elements). The glass in front of the sensor ...

Lyndino
17 Dec 2018
QHA

So you opinion is that the camera in my Galaxy S8 is just as good as the 3 camera set-up on the Mate 20 or the Single super camera on the Pixel 3?

njsilva84
16 Dec 2018
Lcc

Clearly, you have no idea of how not good the smartphone sensors are. I said "not good" not to say bad. There's no way that such small sensors can produce nice images without the need of heavy post-processing. CMOS sensors didn't evolve much in the l...