Snapdragon Sight: Qualcomm's new 18-bit ISP is made for the 108MP, 8K future

01 December 2021

The Snapdragon 8 Gen 1 brings notable improvements in performance and power efficiency for the CPU and GPU. But the camera hardware inside the chip also received major upgrades, so much so that Qualcomm is launching a separate brand for it.

Snapdragon Sight is the branding for the ISP, which now comes with a variety of features oriented towards professionals. Those are thanks to much-improved throughput and even casual users will benefit from that.

The ISP can now process 3.2 gigapixels per second (up from 2.7 gigapixels per second on the SD888). Coupled with other optimizations, this means that it can capture 240 12MP photos per second – double what the Snapdragon 888 ISP could manage.

But 12MP is old hat, if paired with a new 108MP sensor the ISP can capture full resolution photos 30 times per second. Or if it’s connected to three cameras simultaneously, it can do 3x36MP at 30 shots per second. This obviously benefits burst mode, but will also be a boon for frame stacking.

Qualcomm redesigned the ISP to work at 18 bits per channel, up from 14 bits. This leads to an impressive improvement in dynamic range – 4 stops – and better HDR handling. The chipset can export uncompressed 18-bit RAW for editing so that professionals can use every ounce of data captured by the image sensor.

The hardware handling video recording has been enhanced as well. The maximum resolution remains 8K at 30fps, but it now supports HDR capture (both HDR10 and HDR10+). And it can do Electronic Image Stabilization at the same time. Also, the user can snap 64 MP photos while recording an 8K/30 video. Qualcomm also added a hardware engine to render bokeh in videos at up to 4K.

The Mega Multi-Frame Engine will improve night modes in phones – it can merge up to 30 frames into a single image, up from 6 frames, so that is a 5x improvement (helped along by the new burst capability). Video Super Resolution will improve the quality of digital zoom thanks to a dedicated bit of hardware.

Qualcomm also partnered with Leica to develop software, which emulates the behavior of classic Leica lenses in the form of filters that accurately recreate the look of images captured by those lenses. The engine dedicated to ultra wide cameras (which handled things like dewarping the image) now also runs a filter to remove chromatic aberration.

There’s more. AI was already used for auto exposure and autofocus, now there is AI face detection with improved speed and accuracy. It can track 300 facial landmarks (double what the previous chip managed), these are things like lips, eyebrows and so on. One benefit of that is that the animation of the 3D animoji avatars will become much more accurate and nuanced.

There is actually a fourth ISP in the Snapdragon 8 Gen 1. It is part of the third-generation Sensor Hub and enables always-on camera features. This will enable new face unlock modes, but also some privacy features. This ISP can, for example, detect if you are sharing your screen with someone else so that the phone can silence notifications. It can also detect when someone is looking over your shoulder, both help ensure that only you see private information.

Of course, there are privacy concerns with an always-on camera and Qualcomm is taking them seriously. For starters, no data from this ISP will leave the device and it is protected by the new security features of the Snapdragon chip. Also, this feature will be up to OEMs to enable or disable and provide appropriate controls to the user.

The AI engine in the new 8 Gen 1 chip will be able to do 4x the work with 70% higher power efficiency. Future AI models will benefit from support for mixed precision mode, which allows a neural network to mix INT8 and INT16 weights.

For more details on the Snapdragon 8 Gen 1, check out our main article, which focuses on the CPU and GPU side of things.