|
In the image of a normal camera, we only see the rays of light from the world that converge at the camera's focal point. An image is a projective transformation of the light in the surrounding scene. If we consider all of the light that can be seen from any point in a scene, there are rays of light that converge at every single possible viewpoint. Formally, we can characterize this by the plenoptic function. In real life, it is impractical to densely sample the complete plenoptic function, but instead we can sample 4 dimensions of it. This is a light field. A light field can be imagined as a set of images taken of the same scene with the camera in a slightly different viewpoint in each image - the camera rasters over a small region. (A light field can also be generated from one camera, if one adds a microlens array in between the aperture and the imaging plane). However it is generated, the extra dimension in a light field allows us to use the concept of ray reuse to do some cool image post-processing, like refocusing, aperture adjustment, and novel view synthesis. We show these techniques applied to a light field of a chess board from the Stanford Light Field Archive.
To refocus an image, we shift each image in the image array by an amount proportional to its location in the image array. (The shift is C*(u,v), where the image array is parametrized by a u,v plane and C is a constant that affects the depth of refocusing. When we increase C, we are effectively moving the focal point closer to the aperture.
|
|
To adjust the aperture size of our image, we can simply average images in a radius around a point we want to focus on. This works because our light field can be thought of a camera with a huge aperture taking a bunch of sub-aperture pictures. Increasing the aperture of a camera for an image allows more light in at a single point, which makes the image more blurry. We can see this effect in the gif below. We can also notice that the front of the image blurs more quicker than the back. This is because the distance to the back is longer, so the different camera positions in our raster are not as significant.
|
In this project, I learned a lot about how to interpret a light field and generate different types of novel views from a light field. It was very cool to be able to refocus the image after it was taken.
As a chess player, I must say that I am a little bit disappointed in the folks at Stanford who captured this image. They don't know how to set up the pieces! White's king and queen are swapped, and White has played an extra move...
GO BEARS!