CS 194 Final Project

Final Project 1: Light-field Camera

In this project, I investigated how we can use a dataset of images taken on an explicit grid to do depth refocusing and aperture adjustment in a post-processing step. Usually these steps are performed when the image is taken. However using the process that Ren et al. (2005)describes in their paper, we can first collect a series of images and then refocus and adjust aperture using some pretty simple concepts.

Depth Refocusing

The most difficult part of this was how to interpret the file names in the datasets provided. I used the image positioned at 8-8 on the grid as the reference one. Then for each image I found the sub-pixel shift in relation to the reference and shifted each image accordingly. These shift amounts are multiplied by a constant c which changes where the focus of the image is (towards the back or towards the front). Averaging all of these images gives you the results as shown below.

Chessboard

c = 0

c = 0.25

c = 0.5

Bulldozer

c = 0

c = 0.25

c = 0.6

Flowers

c = 0

c = 0.25

c = 0.5

We can see as the factor c increases, the focus moves from the back of the images to the front. Seems like it's working as expected. We can now selectively choose where we want to focus the image in the post-processing step! Here are some gifs of the refocusing for ease.

Truck Refocus

Aperture Adjustment

We can take a similar approach for adjusting the aperture in the post-processing step. We take the 8-8 image again as the reference. Now we can pass in a constant that chooses the aperture radius r. Now instead of averaging all of the images together in the dataset, I only choose images that fall within the radius r from the reference 8-8 image. This is done by calculating the euclidean distance using the numpy.linalg.norm method. After the images are chosen, I shift the images by the sub-pixel differences again and average. Some results are shown below.

Chessboard

r = 1

r = 3

r = 6

Truck

r = 1

r = 3

r = 5

The results seem pretty good and make sense when thinking about how cameras usually work. When apertures are smaller, the range of focus will be larger and more things will be in focus. When the aperture is larger, things closer to the lens will be in focus while the background will be blurry, creating a "bokeh" effect. This is exactly what we see in the examples above. It's more obvious for the chessboard then the truck because there is more going on and more distinction between the foreground and background. Below are some gifs for ease.

Chess Aperture

Flowers Aperture

Overall this project was pretty fun. It showed me we could do super cool things with post-processing using really simple methods!