Project 5: Depth Focusing and Aperture Adjustment with LightField Data

This project takes 289 images taken from the Stanford Light Field Archive, which is essentially light field data, and perform refocusing and aperture adjustment. This is possible through light field because the 289 images, made from a 17x17 grid of cameras, capture a span of the plenoptic function, a function that can measure light at any view point. To digitally refocus and adjust the aperture we simply choose which viewpoints used and averaged.

To change the focus on an object, we need the object to be further or closer away. So, with the lightfield data, we need to compute how the 289 images will be such that the camera looks further back or further forward. First off, we need a center viewpoint, which is (8, 8) in a 17x17 grid. In order to get the default focus just from these 289 images, we must first align them to the center and then average them. To figure out how much to align each images, I essentially treat the images to be in the same 17x17 grid. This is where the top leftmost image is (0, 0) and the bottom rightmost image to be (16, 16). To align all an image at (img_u, img_v), I compute the displacement vector (x, y) = (img_u, img_v) - (8, 8). I shift that image using scipy.ndimage.shift. If this is done for every image in the grid, the resulting image is:

To then simulate the camera being moved further or closer to the scene, we add a factor, C, to the above computation: (x, y) = C * (img_u, img_v) - (8, 8). C was picked to be a factor in between -1 and 1, and spanning C from -1 to 1 produces the follow animation.

Here are also some individual images.

To imitate the changes of aperture size, we essentially change the subset of the 289 images we average. A bigger aperture means more light, therefore more rays coming from different directions from the center. The default image from above, averaging all the 289 images, is essentially a fully opened aperture. A smaller aperture means less light from different directions from the center, so it would only require averaging a subset of images around the center, depending on how open the aperture is.

Say our aperture has a radius r. r = 8 is a fully opened aperture from this 17x17 grid. r = 0 would be the tighest aperture and would only correspond to the center image. For any aperture r, the resulting image is produced by averaging every image within the the radius r from the center. For example, if the radius is 1, then we would average (7, 7), (7, 8), (7, 9), (8, 7), (8, 8), etc.

Below are the results of the radius changing from 0 to 8 and back.

Here are some individual images.