Fall 2018

CS194-26 - Project 5

Lightfield Camera

Depth Refocusing and Aperture Adjustment with Light Field Data

Regina Ongowarsito - cs194-26-aeq

10/30/2018

0: Summary

Using light field data, i.e. a data set of images taken with a camera grid, we can synthesize images of various camera settings (e.g. aperture/f-stop values, focal length, etc) via different operations on the lightfield data.

1: Depth Refocusing

We simulate an adjustment in focal length of the camera by shifting images corresponding to non-center cameras to the ratio of the corresponding camera's distance to the center.

For images left of the center camera, we shift it to the left. For images right of the center camera, we shift it to the right. For images above of the center camera, we shift it up. For images below of the center camera, we shift down. These directions are determined via observing how edgemost pictures in the same axis as the center camera (e.g. for a center camera of a 17x17 grid that lies on the (8,8) spot, the relevant pictures are (0,8) (17,8), (8,0) and (8,17))

chess dataset, rectified - 0x shift, full aperture

chess dataset, rectified - 3x shift, full aperture

chess dataset - focal length change sequence, 10 steps from 0x to to 3x shift

Interestingly, in datasets where the aggregated 0x shift creates an image that is not focused at the farthest point of the object, it is possible to synthesize of such image by using a negative shift.

bulldozer dataset, rectified - 0x shift, full aperture

bulldozer dataset, rectified - -2x shift, full aperture

2: Aperture Adjustment

We simulate an adjustment in aperture of the camera's lens by adjusting the radius of images used in the grid.

For example, to use the "full aperture" of a 17x17 dataset, we use all 17 * 17 pictures. To use "half of the aperture", we use the inner (17 * 0.5) * (17 * 0.5) pictures in the inner (17 * 0.5) x (17 * 0.5) grid.

chess dataset, rectified - 3x shift, 0x of full aperture

chess dataset, rectified - 3x shift, ~0.55x of full aperture

chess dataset, rectified - 3x shift, ~1x of full aperture

chess dataset - aperture change sequence, 3x shift, from 0x to 1x aperture

3: Summary

I've learned that lightfield can accurately synthesize camera results due to its analagous behavior with camera and their sensors! The information that lightfield datasets capture are similar to those captured in normal cameras, except in normal cameras, these light information are averaged out in the singular sensor a normal camera has.