Depth Refocusing and Aperture Adjustment with Light Field Data

Overview

The goal of the project is to apply light field to mimic camera’s Depth Refocusing and Aperture Adjustment. As Prof. Red Ng’s paper on light field camera demonstrated, the light field is created by capturing multiple images over a plane orthogonal to the optical axis. In the Stanford Light Field Archive, I get the grid of 17 x 17 images of various object. Each image is labeled with (x,y) position where the picture was taken.

Part 1: Depth Refocusing

To mimic the depth focusing effect, I first read in all images with their xy positions. First, I took a naive average over all of the images. We can see that since the objects which are far away from the camera do not vary their position significantly when the camera moves around while keeping the optical axis direction unchanged. The nearby objects, on the other hand, vary their position significantly across images. Thus, by taking a naive average over all images in the grid without any shifting, we get an image which is sharp around the far-away objects but blurry around the nearby ones.

Then, to produce the effect of depth refocusing, I shift the images appropriately and then taking the average, which allow me to focus on the object at different depth. To do this, I picked the center image 08-08 of the light field of 16*16 directions. Then, I shifted the images by their position difference from the center images multiplied by a constant factor c. We can see that a low value of c will let the further part of image in focus and as we increase the constant factor c, we shift the area closer to the observer so that the closer part of the image becomes clearer.

Part 2: Aperture Adjustment

In the second part of the project, I implemented the camera effect of changing aperture size by averaging different number of images sampled over a large grid perpendicular to the optical axis. As aperture size increase, depth of field decreases and the image will remain sharp for the focusing area while getting blurrier on other places. To achieve this, I set a fix focal point by fixing the constant factor c = 0.4 in the previous part. Then, instead of averaging over all images together, I averaged over a subset of the original grid of images. In particular, a larger aperture size will allow us to average over more images and having the image get blurrier on unfocused area with smaller depth of field. In the example below, the aperture size a ranges from 1 to 5, allowing us to taking the subset of 1*1 to 11*11 images around the center image at 08-08 and averaging them.