CS 194-26 Project #5: Lightfield Camera:

Depth Refocusing and Aperture Adjustment with Light Field Data

Yue Zheng

Overview

In this project, we explore some effects we can achieve using data obtained by a lightfield camera. A lightfield camera enables capturing multiple images over a plane orthogonal to the optical axis. Using lightfield data, we can achieve complex effects using very simple operations like shifting and averaging.

For this project, I used images from the datasets of the Stanford Light Field Archive.

Part 1: Depth Refocusing

In this part, we generate multiple images which focus at different depths by shifting the images taken by a lightfield camera appropriately and then averaging them.

When we do not shift the images and just average them, we get an image where the objects far away from the camera is sharp while the nearby objects are blurry. This is because the objects which are far away from the camera do not vary their position significantly when the camera moves around while keeping the optical axis direction unchanged. The nearby objects, on the other hand, vary their position significantly across images.

Average image without Shifting

To obtain an image which focus at a certain depth, I shift each image by a scalar factor (alpha) of its distance from the center camera and average them.

Results

alpha = -0.1

alpha = 0

alpha = 0.1

alpha = 0.2

alpha = 0.3

alpha = 0.4

alpha = 0.5

alpha = 0.6

GIF of refocused images: alpha = -0.1 to 0.65, interval = 0.025

Another Example

Part 2: Aperture Adjustment

In this part of the project, we generate images of varying aperture sizes by averaging corresponding images from a lightfield dataset. Averaging a large number of images sampled over the grid perpendicular to the optical axis mimics a camera with a much larger aperture and using fewer images results in an image that mimics a smaller aperture. This is because the number of images models the amount of light coming from different directions.

To generate images which correspond to different apertures while focusing on the same point, I choose the focus point to be the position of the center camera, and average images within a specified radius.

Results

radius = 0

radius = 1

radius = 2

radius = 3

radius = 4

radius = 5

radius = 6

radius = 7

GIF of images with adjusted aperture sizes

Summary

This project was very interesting and defintely reinforced our knowledge of how light travels and how it can be modeled. I learned that just by understanding how light is captured by a camera, we can use simple techniques to achieve various complex effects on lightfield data.