CS194-26: Image Manipulation and Computational Photography, Fall 2017

Project 5: Lightfield Cameras

Sudeep Dasari (cs194-26-aae)



Overview

Lightfields capture the direction which light rays travel through a scene, as well as the point the rays originate from. This is more data than captured by a traditional camera: normal cameras only capture where on the sensor light rays terminate, which means direction information is lost. The extra data allows for computationally rendering images as if they were taken by a camera focused at different depths or with different aperture sizes. This project renders such images from lightfield data recorded by a 17 by 17 camera grid, and available for use at the Stanford Lightfield Archive.

As mentioned before, the light field data was recorded by a 17 by 17 calibrated camera grid. The cameras fire at the same time resulting in 289 1400x800 pixel images. Each pixel is a color triplet associated with a 4 dimensional location: the (u, v) coordinates of the source camera in the imaging array, and the (x, y) coordinates of the pixel in the image. Since there are no occluding objects in between the scene and the imagers, each pixel in the new image stack corresponds to a light ray drawn from a location in space to the target camera.



Depth Refocusing

Chess Scene Rendered Focused at Steadily Increasing Depth

In order to generate a focused image, images from each camera are shifted by C * (u, v) (where C is a constant and (u,v) are coordinates of camera), and then the shifted images from each camera are averaged into one image. The constant C determines the depth at which objects are focused. In my implementation, negative constants focus far from camera, and positive constants focus closer (although this is an arbitrary design decision). Intuitively, this refocus algorithm works by shifting images from each camera so objects at the target depth line up correctly during averaging. Objects placed at different depths are blurred, because during averaging light rays from the same point on the object map to different points in the generated image (aka object is not in focus). Above is a generated depth stack from the chess scene’s lightfield data.

Changing Aperture

Exploring different apertures is relatively simple! Smaller aperture renders are generated by using the subset of cameras that are closest in distance to the center camera. Decreasing the number of cameras decreases rendered aperture and vice versa. This is analogous to decreasing aperture size in a DSLR, which lets light in from fewer directions (aka fewer cameras in lightfield array). Below is a gif of the chess scene focused at the same depth, with increasing aperture size. Note how larger aperture corresponds to more blur, as expected in a normal camera!

Renderings with Steadily Increasing Aperture Size