Light Field Camera Simulation

George Geng

The light field is a function that describes the amount of light flowing in every direction through every point in space; the part of the lightfield that enters our eyes or the camera provides us with visual information about the world around us. We can take discrete samples of the lightfield with plenoptic cameras. Here, I worked with collections of images from the Stanford Light Field Archive to simulate depth refocusing and aperture adjustment.

Depth Refocusing

Far Away Focus

 
For images with different focal points, certain features closer or further from the camera appear clear. When we take multiple pictures of a scene from slightly different positions on a plane orthogonal to the scene, moving the camera only slightly between images, objects that are closer appear to move more relative to each other than ones that are further away. So, when we average the images, objects that are closer appear more blurred than ones that are further away, simulating a far focus point as shown above.
By aligning different regions of the image, we can therefore change the focal point. Here, I simulate different focal points by shifting images in our lightfield (in the form of a grid of images). In my collection of 17 x 17 images, I chose the middle one at (8,8) to be my frame of reference. Each image in the grid was shifted by a distance proportional to its distance from our refernece image, multiplied by a scalar. Adjusting the scalar allows us to adjust the focal depth. The chessboard at different focal depths is shown below.
Adjusting the Focus

Aperture Adjustment

Averaging a large number of images sampled over the grid perpendicular to the optical axis mimics a camera with a much larger aperture. Using fewer images results in an image that mimics a smaller aperture. For an n x n grid, we mimic apertures from 0 to n/2. For an aperture of size m, we average the middle m x m images. Here are some simulated aperture sizes from 0 to 8 for an image of a bunny.
Adjusting the Aperture

 
Overall, this project was an interesting introduction to the power and a few of the many fascinating properties of lightfield cameras.