Project 5: Lightfield Camera

James Lin, CS194-26-abr

Light fields are sets of images taken of the same subject. Their angle remains the same, but their position varies subtly on a grid perpendicular to the optical axis. We can use the information from light fields to apply post processing techniques onto an existing image.

Depth Refocusing

The first exercise was depth refocusing: simulating a change in focus to a different part of the image. To see how we did this, first examine what happens when we average all images from the light field together.


You'll notice the top half of the image is more in focus. This is because the objects closer to the camera move more (between light field images) while objects farther away move less, causing them to stay aligned and in focus during the averaging. What, then, happens if we shift the light field images before averaging them?


This is the result of shifting all images to the center. For example, if an photo had been taken 2 units up and 4 units left from the center, it would get shifted 2 units down and 4 units right before being added to the average. This process lines up the centers of every image, which is why the center of the photo looks in focus.

We could extend this process to move the focus to any part of the photo. For our purposes, we simply shifted each light field image by how far it was offset from the center, multiplied by a constant. We then repeated this process for varying values of this constant (from -3.75 to 1), and combined the results together into a gif.


Aperture Adjustment

The second excerise was aperture adjustment: manipulating the size of the aperture used to take the photo. We do this by, selectively taking averages of the light field images. For instance, consider the individual image taken by the center light field camera.


The picture has a large depth of field, which indicates that the aperture used to take it was small. But we can "expand" the size of the aperture by including in the photos taken by light field cameras near the center one.


More cameras means more rays of light come from the object and hit different pixels on the final image (since the cameras are offset from one another). This simulates the enlargement of the aperture. Since the cameras for this light field were set out in a 17x17 grid, we can vary the aperture size from a distance of 0 (just the center camera) to a distance of 16 (all cameras on the grid)