CS 194-26 Project 5: Light Field Camera

Jeffrey Wang (cs194-26-acl)

Overview

In this project, I used data from the Stanford Light Field Archive in order to shift and average a set of images taken by one camera with multiple lenses such that the resulting image would be manipulated in depth and aperture.

Depth Refocusing

By averaging the whole set of images from the light field camera, we are able to produce a resulting image that is "focused" further away towards the back of the chessboard. This is due to the fact that the pixels farther in depth from the image tend to be closer to each other and move less as we take a look at the different shots from the different camera positions. The pixels closer in depth in the image appear blurry as they move around more, thus making the average more noisy.

We are able to manipulate this by shifting each light field image based on its offset. In this case, I shifted each image based off of the camera in the center of the array and an arbitrary one towards the bottom. Each image was further scaled by a ratio, and the resulting effect would be that the final image would have focus on separate parts of the image itself, as shown in my examples below:

Center Bottom

Aperture Adjustment

Another manipulation we could implement on the final results is to adjust the aperture of the combined image itself.

By taking the average of more and more images, we can mimic increasing the size of a camera's aperture, thereby reducing the overall focus of the image itself towards the center. To do this, I only took the averages of the images whos position fit within a certain radius from the center, slowly increasing the radius size to build up on the final image.

Summary

Overall, I found this project really straightforward and enjoyable (albeit I began late and needed a slip day). One thing that I realized later on was that I was incorrectly taking the averages of the images as a whole, where for each input image I would add it to the result and divide by two, leading to inaccurate results. This was a bit frustrating to reimplement, but the results ended up looking much nicer.