Overview

Light field cameras are a specific type of camera which takes multiple images each from a slightly different perspective. The gif below is made up of each of the images taken from a light field camera. Using image processing, we can combine these images to get a new image where we can digitally change the aperture and focus plane.

Part 1: Depth Refocusing

In this part, I had to shift each image from the light field the appropriate amount and then average the images. To do this, I had to shift each image linearly with respect to how far it is away from the center of the microlens array. Here is a gif of the results, when varying the image plane:

Part 2: Aperture Adjustment

In this part, I had to digitally change the aperture of the image. To do this, I only averaged the images which were within a certain radius of the center. When the image is sharpest around the edges, the aperture is as small as possible (like a pinhole camera), and when the image is blurriest, the aperture is the largest. Like the Depth Refocusing section, I combined all the different images into one gif, to better show the effect of changing the aperture.

Part 3: What I Learned

When I took CS 184 last semester, I learned about light field cameras, but I didn't ever implement something related to light field cameras. This project helped me further my understanding of light field cameras. Furthermore, I learned how difficult it is to actually implement this physically. The slightest change in angle can mess up the results a lot.

Bells and Whistles: Interactive Refocusing

To implement Interactive Refocusing, I borrowed some code from project 1. The basic idea is to find the shift for each image that would align all images at the selected point, and then average the shifted images. Since finding the optimal shift is time intensive, we leverage the fact that we know the shift for all images, if we know the shift for one of them. I find the shift from the center image to the image 1 unit away above and to the right, and shift each image scaled to the distance away from the center.

Bells and Whistles: Real Data

For this part, I had to take multiple pictures with my phone camera from different positions to simulate a light field camera. I attempted many times to get good pictures. It is significantly more difficult than I thought to get images that are static distances away from each other. The following one of my living room is the best one I got: