CS 194-26 Project 5: Face Morphing

This is the submission of Myron Liu (cs194-26-afp) for CS 194-26 Project 5.

Background Depth Refocusing Aperture Adjustment Summary

Background

In this project, I took light field data from the Stanford Light Field Array Archive which took multiple images over a regularly spaced grid. Then following section for of this paper with some elementary shifting and averaging operation to perform some image manipulation techniques such as depth refocusing and aperture adjustment.

Depth Refocusing

When objects are far away from the camera array, their positions in the image do not vary significant from different camera array indices as the images are taken with the camera moved around the optical axis direction. Nearby objects will vary their position significantly across the images. Thus by shifting the images relatively towards a central image focused at point (u, v) (in this case the camera located at indices (8, 8)), we can produce images that are shifted and averaged to appear to focus on different depths of the image objects. I produced shifts using a constant C value multipled by the relative position away from the central point (u, v) and then averaged the image to get the following depth refocused images.

Tarot Cards (No Depth Refocusing)

Tarot Cards (Far Away Focus)

Tarot Cards (Close Focus)

Tarot Cards (Depth Refocus Gif)

Necklace (No Depth Refocusing)

Necklace (Far Away Focus)

Necklace (Close Focus)

Necklace (Depth Refocus Gif)

Jellybeans (No Depth Refocusing)

Jellybeans (Far Away Focus)

Jellybeans (Close Focus)

Jellybeans (Depth Refocus Gif)

Aperture Adjustment

With a light field camera array, objects in an image for a particular index of the camera array can be viewed as a camera image of fixed aperture size. Larger aperture sizes would result in a blurrier image - objects perpendicular to the camera stay in focus as rays from them are near perpendicular and do not veer too far off from each other. Objects near the edge of the aperture end up blurrier because of multiple different angled rays ending up further from each other as they hit the projection plane. I simulate this effect using a lightfield array and averaging a grid of images of different sizes - going from 1 single image, 3x3 grid, 5x5 grid, ..., to the full 17x17 grid.

Tarot Cards Small Aperture (3x3 Grid)

Tarot Cards Medium Aperture (11x11 Grid)

Tarot Cards Large Aperture (17x17 Grid)

Tarot Cards Aperture Animation (Gif)

Necklace Small Aperture (3x3 Grid)

Necklace Medium Aperture (11x11 Grid)

Necklace Large Aperture (17x17 Grid)

Necklace Aperture Animation (Gif)

Jellybean Small Aperture (3x3 Grid)

Jellybean Medium Aperture (11x11 Grid)

Jellybeans Large Aperture (17x17 Grid)

Jellybeans Aperture Animation (Gif)

Summary

Light fields are really awesome because they provide ways to manipulate images without having to change details in the capture like exposure, aperture size etc. It all happens using computational photography which is super cool.