New algorithm could automatically remove obstructions from photos

We've all had to deal with this issue at one point or another - you managed to capture that one perfect shot and time it just right, but sadly a nasty fence or your own reflection are in the way and horribly distracting. Well, researchers at Google and MIT have been working on a software solution for the problem and it looks like they are onto something.

The algorithm promises to almost magically remove obstructions from photos. An ideal usage would be for fences, like in the demonstration, as anything bigger simply blocks objects and filling in lost content is going to be very tricky.

Michael Rubinstein, a research scientist at Google, familiar with the project explains that the algorithm relies on the phenomenon of motion parallax. It is a perceptive illusion that leads us to believe objects that are closer to us move faster than the ones further away when we change our own vantage point. The software leverages this effect by analyzing a few consecutive shots taken by panning the camera slightly, much like a panorama shot. It then distinguished between the different layers and attempts to remove the one in the foreground, which should be the unwanted obstruction.

Since they’re moving differently, we can use that information to figure out there are two layers we’re actually looking at, and we remove one of them,

This approach is quite clever and according to Tianfan Xue, lead author to the MIT paper, it can currently be applied to reflections, dirt and water on windows, as well as the aforementioned chain fences. The technology has already been tested in practice at a local zoo, where it produced a lot of clean and impressive, "fence-free" shots with the use of nothing more than Android phones.

Rubinstein says that Google has expressed interest for the algorithm, so it is entirely plausible that after the bugs get worked out, it makes its way as a new camera mode.

Source | Via