Background
in this paper: http://graphics.stanford.edu/papers/lfcamera/lfcamera-150dpi.pdf , Ren Ng. and Al outlines a number of things you can do with a lightfield camera. While there are many complex details about how the apparatus works regarding lenses and what not, the main idea is that using a handheld device, we can take many photos at once on the same, plane, but all slightly shifted. Using this, we can do things like shift our focal point, refocus the image, and change the aperture after we have actually taken the picture.
The dataset used for this project comes from the Stanford light field archives: http://lightfield.stanford.edu/lfs.html
There, they have an apparatus that contains an array of cameras on the same plane that captures the images at the same time. We will use this dataset to refocus our image, and change the equivalent aperture after the fact. Below is a picture of what the apparatus looks like. The cameras are placed on a 17x17 evenly spaced grid.
The dataset used for this project comes from the Stanford light field archives: http://lightfield.stanford.edu/lfs.html
There, they have an apparatus that contains an array of cameras on the same plane that captures the images at the same time. We will use this dataset to refocus our image, and change the equivalent aperture after the fact. Below is a picture of what the apparatus looks like. The cameras are placed on a 17x17 evenly spaced grid.
Depth Refocusing
The first thing we can do with these lightfield images is depth refocusing.
If we take all of the images as is and average them out. We notice that we get an image where the background is sharp, but the foreground is not. This is because of parrallax causing objects further away to be less variant in position across the cameras, whereas closer objects vary a lot.
If we take all of the images as is and average them out. We notice that we get an image where the background is sharp, but the foreground is not. This is because of parrallax causing objects further away to be less variant in position across the cameras, whereas closer objects vary a lot.
However, because we know the x,y location of each camera relative to a central camera (which is provided with the dataset), we can shift all the images towards the center. This has the effect of lining up the closer objects, while further objects then become unaligned. This essentially gives us the effect of changing the plane of focus to different parts of the image. By varying the amount of shift from 0 to 0.8 (with 1 being all images are aligned to central camera), we can create the following gif that illustrates the depth refocusing.
Aperture Adjustment
We can also effectively change the aperture of the image after the fact, allowing us to dial the amount of depth of field that we want in the image. Notice that with one image, the picture is relatively clear, whereas when we shift and avg multiple images, only items in focus remain clear while the rest of the image becomes blurry.
To do this, we set a constant focus (ie same alpha from above) by shifting all of the images. Next, we can avg a variable amount of images. The more images we average together, the larger the effective aperture is. Items not in the plane of focus will become more blurred.
To do this, we set a constant focus (ie same alpha from above) by shifting all of the images. Next, we can avg a variable amount of images. The more images we average together, the larger the effective aperture is. Items not in the plane of focus will become more blurred.
Conclusion
The idea of lightfield cameras proposed by Ng. et Al. is very interesting. It allows for relatively simple operations to create some really cool effects. With simple shifts and averaging, we can dial in various aspects of the photo after the fact. This could be really helpful in not worrying about specific settings while taking photos, and adjusting for creative effect after the fact.
I learned how to adjust the focus and aperture from this project, and really enjoyed the simplicity and elegance of it.
I learned how to adjust the focus and aperture from this project, and really enjoyed the simplicity and elegance of it.