Project 5: Lightfield Camera
Student Name: Divyansh Agarwal (cs194-26-aba)
Course: CS 194-26 (Computational Photography)
Background
In this project, we try to simulate the refocusing of the depth of images and adjustment of the aperture size using light fields. These experiments are done using the light field data from the Lego Gantry in the Stanford Light Field Archive.
Depth Refocusing
To simulate the refocusing of the depth of images, I shifted each image in the dataset based on its distance from the center image, scaled by a constant factor. Then, I averaged all the shifted images together.
When the scale factor was 0, i.e., when I did not shift any of the images, averaging the images gave a result focused on far-away objects, and increasing the scale factor brought the focus to closer objects.
The following GIF shows the result of gradually increasing the scale factor from 0 to 0.5, with increments of 0.05:
via GIPHY
Aperture Adjustment
I first focused all images on the same point, by fixing a scale factor of 0.2 while shifting each of them, in order to focus the images on the center. Then, to simulate an aperture size of r, I only averaged shifted images within a square grid with a side length of 2r + 1, with the center of the grid being at (8, 8). Since I focused the images on the center, increasing the aperture size led to the regions farther away from the center getting blurrier. I incremented the aperture size from 1 to 7, in increments of 1.
via GIPHY
Summary
This was a super cool project, where I learned through practical experience about the magic you can do with an understanding of lightfields! It was really insightful to see how increasing the aperture causes blurring of the area that is not in focus as well.
Bells and Whistles: Interactive Depth Refocusing
To implement interactive depth refocusing, I first let the user click on a single point they want the image to be focused on. I then tried to refocus at the depth corresponding to the clicked point. I did this by taking a single image that was not the center image, and while shifting it based on its distance from the center image scaled by a constant scale factor, and experimented with scale factors in the range [0, 0.5], incrementing values by 0.05. For the shifted image produced by each scale factor, I computed the cross correlation of the shifted image with the image at the center. I then picked the scale factor that gave me the highest cross correlation. I only needed to do this for one image, as the optimal scale factor would be the same for all images, since the same scale factor would be used for all images. I then used this optimal scale factor to compute the shift for all other images, and averaged them together.
The results can be seen below:
Selecting a Far-Away Object to Focus On
Selected Coordinates
Please observe the red dot that indicates the point selected to focus on.
Re-Focused Image
Selecting an Object in the Middle to Focus On
Selected Coordinates
Please observe the red dot that indicates the point selected to focus on.
Re-Focused Image
Selecting a Close Object to Focus On
Selected Coordinates
Please observe the red dot that indicates the point selected to focus on.
Re-Focused Image
