Lightfield Camera: Depth Refocusing and Aperture Adjustment with Light Field Data
Overview
The goal of this project is to reproduce the effects of image refocusing at various image depths, and readjusting the image
aperture using the rectified and grid aligned light field images taken from the Stanford LightField database. The light field is a 5-D data structure storing images at every sub-aperture at different location in an 2D array of sub-apertures.
Part 1: Depth Refocusing
The objects which are far away from the camera do not vary their position significantly when the camera moves around while keeping the optical axis direction unchanged. The nearby objects, on the other hand, vary their position significantly across images. Averaging all the images in the grid without any shifting will produce an image which is sharp around the far-away objects but blurry around the nearby ones. Similarly, shifting the images 'appropriately' and then averaging allows one to focus on object at different depths.
In this part of the project, I implemented this idea to generate multiple images which focus at different depths. To get the best effects, I used all the grid images for averaging. Using the sub-aperture (8, 8) as the center since we have 17 by 17 sub-apertures, for a general sub-aperture located at (x, y), we can denote its shift as:
, for the constant alpha. After applying this transformation for all the images, the refocused image is found by computing the average image. In the case that alpha is greatly positive, there is a depth refocusing
further from physical focus, while in the case that alpha is greatly negative, there is a depth refocusing closer towards the
physical focus. Below are the static images produced by various stepsize values of alpha as indicated:
Depth Refocusing Image 1: Chessboard, alpha: [0, 3]
Part 2: Aperture Adjustment
In this part of the project, I generated images which correspond to different apertures while focusing on the same point.
More specifically, to change the image aperture, only some apertures around the image center is used instead the aperture
of all the images. The parameter radius that we specify determines this boundary of which sub-sample of apertures we use. An image aperture at (x, y) is used if it satisfies:
x such that x: [8 - radius, 8 + radius] and y such that y: [8 - radius, 8 + radius] Aperture Adjustment Image 1: Chessboard, radius: [0, 8]
Summary
I now have a better understanding of the effects that aperture adjustment and image refocusing produces on the image,
and how cinematic photography & professional photography use these features to produce blurring background effects
(aperture adjustment), and different image perspectives (refocusing different parts of the image). Furthermore, this
is a powerful technique that can be used to edit photos after the fact.