Project 5: Depth Refocusing and Aperture Adjustment with Light Field Data
Brandon Huang, Fall 2017
Overview
In this project I aimed to exploit the special properties of 4D light field data to achieve image effects beyond those possible with 2D imaging. In particular, it is possible to both refocus and change the apparent aperture of an image using synthetic techniques.
A lightfield camera aims to capture a more thorough sample of the "plenoptic function". The plenoptic function can be written as f(azimuth, elevation, wavelength, time, X_pos, Y_pos, Z_pos). While a typical camera can only capture the 2 dimensions of (azimuth, elevation), a lightfield camera can sample 4 dimensions: (azimuth, elevation, 2D position). This allows us to approximate changes in aperture and focus by manipulating the extra dimensions of the data.
Depth Refocusing
In a traditional camera, for an image to be in focus, the rays coming from a given point on the world plane must converge to a point on the imaging plane. There is no way to keep objects at different distances simultaneously focused due to lens geometry. However in a lightfield camera, we can capture rays incident on the imaging plane at different angles. Thus by selecting the correct rays from each point on the imaging plane, we can focus images at arbitrary depths.
GIF showing a progression of focus from the back to the front. Each refocus corresponds to a different shift. The animation starts with no shift, which focuses on the objects at the back of the image that have less parallax. With greater shifts, we can correct for the parallax of objects at the front and bring them into focus, at the cost of defocusing objects in the back.
Aperture Size
Each image in the lightfield data is essentially an image from a pinhole camera. If we start from the center image and start averaging in images around it, we simulate an expansion of the camera aperture.
GIF showing a synthetic increase in aperture size, with constant focus on the center of the chessboard. Depth of field markedly decreases throughout the animation.
