Project 5: Light Field Camera

by cs194-26-abp

Overview

We explored a LightField and the plenoptic function to see how we can refocus and change the aperature of pre-existing images. We can refocus an image to have different parts of the image in focus and change the aperature of our `camera` after the image has been taken. I used images from the Stanford Light Field Archive.

Depth Refocusing

Since the cameras were arranged in a 17x17 grid, the camera at 8x8 was declared to be the center camera. Each camera's image was shifted to align with the center camera's image. Specifically, if the center camera was at (x,y) and the other camera was at (u,v), then the image was shifted by c*(u-x,v-y) where c is some fixed constant. By varying c, we varied the magnitude of the shift and which section of the image was in focus. When we change the position of the camera, objects far away hardly move while objects close move significantly. So, when c is small, objects that are far away are in focus since they are in the similar locations across all camera positions. When c is large, we are focusing on closer objects since the object moved significantly depending on camera position. I let c vary from 0 to .65 and picked 35 evenly spaced points. The aligned images were averaged to get an image with different depths in focus.

Results


Shift constant c = 0.0

Shift constant c = 0.153

Shift constant c = 0.363

Shift constant c = 0.65

Aperature Adjustment

By changing which cameras I included in my averaging, I can adjust the aperature. Recall that a narrow aperature has a wide depth of field aka a wide range of depths where the objects are in focus. A large aperature has a narrow depth of field and only objects in the foreground are in focus with a blurry background. To mimic a different aperatures, I restricted the cameras I averaged to only ones within a certain distance from the center camera. To increase the aperature, I increased the acceptable radius from the center camera. I picked 10 radius values between 0 and 12. I defined the distance from a camera at the grid position (x,y) to be the radius r such that
r =[(x-8)^2 + (y-8)^2]^(.5)
Using my chosen subset of images, I refocused them as I did in part 1 with c = 0.25 since c = 0.25 has the center in focus.

Results


Radius r = 0

Radius r = 4

Radius r = 8

Radius r = 12

Summary of What I learned

I learned that we can use multiple images taken from different spots to create a lightfield. The lightfield can be used to refocus an image and change the aperature. I thought it was really cool to see the power of a light field. We are creating synthetic images with data created years ago!

Bells and Whistles: Interactive Refocusing

The center image is displayed and the user selects the point she wants to focus on. The code randomly picks another camera that we will use to find the shift c constant. To find the c, the code finds a square centered on the user selected point. The square is 400x400 (or the smallest square that does not go over the image's boundaries). We then try 35 different c values evenly spaced in the range 0 to 0.6 . I saw in part 1 that the c values that resulted in non-blurry images tended to be in that range. The best c value is chosen to be the one that maximizes the normal cross correlation between the center section of the two squares. This works since we want the user selected point to be in focus which implies that all images must be aligned on that point. We assume that the camera has not moved enough to cause the user-selected point to be outside the square of the other image. We are finding the shift that makes the user selected point in the center image the closest to the center of the square in the shifted random image. As c is a fixed constant across all images, we only need one other image to find it. We then run the code from part one to focus the images.
The selected point is shown as a blue dot. The refocused image is shown to the right of the selected point.

Results


Selected Point

Refocused on Selected Point

Selected Point

Refocused on Selected Point

Selected Point

Refocused on Selected Point

Selected Point

Refocused on Selected Point