CS194-26 Project 5: Light Fields

Hersh Sanghvi | cs194-26-add

Summary

In this project, we learn about light fields! Light fields are basically approximations to the plenoptic function, which is a theoretical function that captures all the light rays that are reflected off an object, allowing one to perfectly reconstruct the scene. We use the Stanford Light Field Archive, in which pictures are taken of various objects in a gridded fashion. This allows us to simulate the effects of changing the focus of an image after it’s taken, and reducing the size of the aperture, since both of these operations can be framed in terms of just operating on rays of light. This project taught me more about how these operations can be framed in terms of manipulations on the individual images themselves.

Part 1: Refocusing an image

In this part, we try to adjust the focus of the image. This can be achieved by shifting the images towards the “center” image of the grid by varied amounts. The idea is that if you just average together all of the images, the region of the image close to the camera will be blurry, since the nearby region moves the most with camera movements. However, if you shift each image to the center, then the nearby parts will be aligned, while the far parts will become blurry. This simulates the effect of focusing on the near field vs. focusing on the far field. Here is an example on the “Lego knights” picture:

And here is a full gif showing a smooth transition:

Here is another example, with the treasure chest image:

And the full gif:

Part 2: Adjusting the Aperture

Similarly to refocusing, we can also simulate the effect of having a smaller or larger aperture. We can do this by taking a smaller and smaller radius of images around the center image, which will simulate the effect of light rays coming in at more acute angles. We can then average together the images satisfying these radial bounds, resulting in a smaller aperture. Since smaller apertures have a smaller depth of field, we expect to see objects that are out of focus become blurrier. Here are the Lego knights, once again:

And here's the full gif:

Here is the example of the treasure chest:

And the full gif: