Image Manipulation and Computational Photography

Project 5: Lightfield Camera

Jianglai Zhang



Card image cap
Card image cap
Card image cap


Overview

This time we explore the light field of a camera where the light field literally describes the amount of light flowing through. We will now use sets of pictures taken by a special light field camera to discover methods to refocus at different depths as well as to adjust the apertures of the image after the fact. The algorithm we use and the theory of plenoptic functions are based on the idea in the paper by Ng et al. For reference, the images used here are from the Stanford Light Field Archive.


Depth Refocusing

By playing with the results in sets of images in the Archive, we can achieve depth refocusing of the image. In general, each of the 17 x 17 grid photos in each set is taken shifted along a plane as a different perspective; we could simply shift each of the grid images by an appropriate amount and finally average them to achieve the effect of depth refocusing. An interesting idea behind this, by the way, is that when observer moves up close or far away from a scene, the object further away is less affected than is the closer object in the scene. Specifically, to get a specific depth, we first calculate the center coordinates, and then for each image we compute an offset from the center cooridnates; then we shift each image by this offset multiplied with some scaling constant; finally we average all shifted images. By changing the value of the scaling constant, we can achieve different depths:


Card image cap

constant = -0.5

Card image cap

constant = 2.5

Card image cap

constant = 5.5


Then we can create a smooth sequence by varying the constant (here I use constant ranging from -0.2 to 0.6) and apply it to the shifts:


Card image cap


Aperture Adjustment

We can also change the aperture of the image after the fact given the sets of images provided in the Archive using a slightly modified method from the one we used in depth refocusing. In general, the only difference is that we would like to only sample images from within a certain radius from the center coordinates. The greater the radius, the greater the aperture size, thus the more blurred the outskirt areas:


Card image cap

radius = 1

Card image cap

radius = 3

Card image cap

radius = 6



Then we can create a sequence of changing aperture sized images (here I use radius ranging from 0 to 7 in grid unit):


Card image cap


Reflections

The light field of a camera is an extremely important set of parameters for photography. With careful manipulation of such and abundant resoures, we could achieve changing apertures or depth refocusing after the fact. The theory used above also makes me reflect on how we can think of the physical world in a 'camera's perspective'.


Credits

Inspiration taken from friends, past projects, python libraries, papers and other online resources.