Project 5: Lightfield Camera

In this project, I explored the images captured by lightfield cameras. Thanks to Stanford Light Field Archive for the awesome collection. I chose the rectified necklace and lego knight, each of which contains 289 images captured by cameras on a 17x17 grid plane. The main tasks for this project is to refocus depth and adjust aperture using the image sets!

Part 1: Depth Refocusing

In this first, we utilized all images captured from 17x17 grid plane and implemented to generate image with effects of different depths. The backbone idea comes from that far-away objects do not vary much when cameras move around at unchanged optical axis, while the nearby objects vary more significantly.

Therefore, we want to shift the images and then average them to have the effects of different focus depths. I first calculated the average aperture (u', v') and the offsets of all 289 images to (u', v'). Then, with pre-defined weighting factor alpha, I shifted them accordingly by alpha * offset using scipy.ndimage function shift. Lastly, I averaged the shifted 289 images to get the final result.

Here are three samples with alpha = -0.01, 0.05, and 0.13 on resized(1/4 of original) rectified necklace image:

Part 2: Aperture Adjustment

Besides refocusing, we can also set a center image and range ahead, and then select the images subaperture within the range from the center subaperture. Then using these images to average and shift properly to mimic different sizes of aperture. The larger the range we choose, the larger the aperture.

My algorithm uses the image (8, 9) to be the center image and set alpha to be 0.25 by default. Then I calculated the offset at grid level (offsets between camera position in the grid). I chose images within the pre-defined range, shift all images to the centered image subapeture and average to get the final result.

Here are three samples using R = 1, 5, 7 for lego knight: