Introduction

In this project, I worked with data from The Stanford Light Field Archive which has sample datasets comprising of multiple images taken over a regularly spaced grid. The Light Archive and Lytro camera project were lead by professor Ren Ng. By moving a camera over a plain orthogonal to the optical axis and capturing multiple images, a grid of images is generated. With this grid, we are now able to digitally change the focus of an image or simulate the adjustment of aperture. The results of my algorithm are demonstrated below.

Depth Refocusing

We can create digital depth refocusing using the fact that different positions of the camera result in slightly different pictures, like these:



As we vary the position of the camera relative to the object an interesting phenomena occurs. The objects which are far away from the camera do not vary their position significantly when the camera moves around while keeping the optical axis direction unchanged. The nearby objects, on the other hand, vary their position significantly across images. We can use this observation to focus different parts of the image. If we average all the images without any shifting, we will achieve an image that is sharp for far away objects and blurry for nearby ones. In the similar manner, if we shift the images and then average, we can direct to focus to different areas.
I first implemented this idea on the images of the chess board by varying the values of c from -1.0 to 3.5.

c = -1
c = -0.5



c = 0
c = 0.5



c = 1
c = 1.5



c = 2
c = 2.5



c = 3
c = 3.5



Putting all these images together results in the following gif:


Next, I used the images of the jelly beans with values of c ranging from -4 to 0.5
c = -4
c = -3.5



c = -3.0
c = -2.5



c = -2.0
c = -1.5



c = -1
c = -0.5



c = 0
c = 0.5



Putting all these images together results in the following gif:


Aperture Adjustment

Averaging a large number of images sampled over the grid perpendicular to the optical axis mimics a camera with a larger aperture. On the flip side, using fewer images results in an image that mimics a smaller aperture. In this part of the project, I generated images which correspond to different apertures while focusing on the same point.
I set an aperture variable around the center of the grid and tried values of 1, 2, 4 and 8 images from the center. Below are the results from this experiment:

Aperture = 1
Aperture = 2



Aperture = 4
Aperture = 8



Bells and Whistles

I wanted to try setting up my own grid of images. I borrowed a camera and a tripod with a friend and decided to give it a shot.
For my image, I set up two objects, a globe and a diamond. I set up the diamond in the front and the globe in the back.




For my first attempt, I tried to take a grid of 4 by 4 images, shifting the camera from left to right by about an inch and shifting the camera up by about half an inch for each row. The results were pretty blurry:


I realized I will need to move the camera much less to achieve anything reasonable. I also decided to make my grid a 2 by 4 instead of a 4 by 4 to reduce the amount of possible error between images. Also instead of physically moving the tripod, I only adjusted the camera position on the tripod. I tried to move the camera by less than half a centimeter each time. The grid looked like this:





I then ran my algorithm with c values ranging from 1 to 40. Higher c values were required because my images are more shifted from each other than the images in the Stanford project. I picked the 6 least blurry results to highlight here and the 16 lest blurry results to turn into a gif.
c = 24.5
c = 25




c = 25.5
c = 26




c = 26.5
c = 27





These images basically almost work. The front diamond is getting more in focus with higher values of c, while the back globe gets slightly sharper with lower values of c. In order to achieve better results more precision is needed. Because I only have one camera and I have to physically move it, it's almost impossible for me to align all my images correctly especially as I shift the camera up and down to create new rows of my grid. This results in misalignment that cannot be fixed with the shift function, meaning that the blurriness is unavoidable.

Final Thoughts

This project was fun and I enjoyed experimenting with different images and focusing them in different ways.
I wish I had the opportunity to work with a more accurate equipment to generate my own grid images.