Lightfield Camera: Depth Refocusing and Aperture Adjustment with Light Field Data
cs194-26-aea
Lightfield images are a variation of traditional images that attempt to capture more features
from the underlying
Plenoptic Function.
A lightfield basically captures information not only about the landing point of an array (aka the value at a pixel)
but also the angle of the ray. This allows one to use the lightfield to dynamically create photos that can be
refocused, move the viewpoint, and change the size of the aperture all after the images have been captured.
Lightfield images can be created by taking mutliple images at slight spatial shifts.
Our particular images were taken by
The Stanford Multi-Camera Array
pictured below. The camera array took 289 images simulataneously in a 17x17 array.
The Stanford Multi-Camera Array
Because of the spatial orientation, each camera captures a slightly different perspective of the object being captured.
This difference in orientation provides enough information to then perform the cool post-processing stuff that come
with lightfield captures.
Below we can see the different perspectives when we look through the pixels that originate just from a single point in the lightfield
with respect to each of the cameras.
Sampling the same points seen by different cameras
Sampling the images produced by the different cameras
Depth Refocusing
This technique is fairly simple. Basically, the objective is to change the focal point of the resulting image. All
one needs to do is shift the images taken at the edges of the array closer to the center of the array to increase focus on
closer objects.
Refocusing on an image of an amethyst taken by a lightfield camera.
Aperture Adjustment
Aperture adjustment is also relatively simple to do with a lightfield image. Instead of taking the average of all
the images captured by the lightfield, instead we select a subset within a certain range. This simulates the
closing of an aperture. By decreasing the area of images chosen from the camera array, we can simulate the change
of a full aperture slowly shrinking to become a pinhole by gradually using less and less images for the averaging
operation until we are left with only one.
Changing the simulated aperture from fully open
to pinhole size.
Perspective Adjustment
The simplest rendering we can make with a lightfield is a perspective adjustment. By just keeping the aperture size at pinhole level (ie only use
one image from the array), we can simulate movement around the field of view to create a spatial effect in an image. With proper
software alignment, one could use this technique to follow the perspective of a viewer and simulate spatial depth.
Changing the perspective alignment of the image.