CS 194-26 Project 5

Lightfield Camera

Overview

In this project, I got to use lightfield data to achieve depth refocusing and aperture adjustment effects. The datasets used contained multiple images of a scene taken over a regularly spaced grid. Using this data observed from different sensors, I applied elementary operations to achieve my results.

Depth Refocusing

We have images of our scene captured from many different positions laid out over a plane orthogonal to the optical axis. Objects closer to the plane vary their positions significantly depending on the position of the camera in the grid, whereas objects further away do not vary their positions as greatly. For this part, I shifted images appropriately and averaged them to achieve a depth refocusing effect, in which the images appear to focus at different depths.

For image shifting, I utilized the position values stated in the file metadata. Once I had all of these shifts, I applied different alpha (scaling) values that focused the image on nearby objects, faraway objects, and everything in between. This effect is illustrated in the following images, showing the varying depths at which I was able to refocus.

Close focus

Close focus

Aperture Adjustment

In this part, I generated images that correspond to different lens aperture sizes while keeping the focus constant. For this, I used the scene of the marble and cards, since it has a central subject with other objects of interest surrounding it. Changing the aperture affects the depth of field of the image, which changes how much of the image is in focus. With a large aperture, we would expect to see a shallow depth of field with a blurry fore- and background, which is accomplished by averaging together many different images. With a small aperture, we would expect to see a lot more of the image in focus, which is accomplished by averaging together relatively few images. I performed this process by setting varying radii and averaging together only the images that fell within each radius. The effects can be seen below.

Small aperture

We see that the marble remains in focus, but its surroundings go in and out of focus, mimicking a changing lens aperture size.

What I Learned

From this project, I learned that taking photos of a scene from many different angles gives us a wealth of useful lightfield information. We can use this information to go back and make changes to the image composition that would otherwise be impossible. After attempting Bells and Whistles with using my own data, I also learned that it's hard to capture all of this data. The slightest error in camera angle or positioning can make it very hard to use the data from our grid of images.