Image Manipulation and Computational Photography

Final Projects: Fake Miniatures, Vertigo Shot, Seam Carving

Zheng Shi, Jianglai Zhang



Card image cap
Card image cap


Fake Miniature: Overview

Our first project is creating fake miniature photos. The general idea is to blur some parts of the image to simulate shallow depths of field that usually only seen in close-up photography. The objects in the image thus seem a lot smaller than they actually are. The approach we take is to simulate a tilt-shift effect, which is explained more thoroughly below.


Tilt-Shift

The method is actually quite straight-forward. The idea is that different horizontally straight levels represent different depth of field. So we first ask the user to select a desired point in the image that will be used to produce a horizontal focus line around which the image will stay sharp. Within a certain boundary size both above and below the focus line, we will keep a range of pixels that are in our simulated depth of field area. Then we blur the areas increasingly as pixels get increasingly further away from the imaginary focus line. Here are some high-vantage point images before and after our fake miniature effect:


Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap


Bells & Whistles

Applying the simulated tilt-shift method described above to a consecutive frames of images, we can get fake stop motions animations:


Card image cap


Vertigo Shot: Overview

This project is to duplicate the effect of dolly zoom. We used Sony RX100M4 for this project. We first picked an object and placed it in the center of camera view. Then we changed FOV by adjusting focal length. Meanwhile, we moved the camera front (or, back) to keep the chosen object in the same size for every pictures taken. Here are the result sequences with a stuffed animal and some legos as the subject, respectively (taken with 70mm - 24mm):


Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap


Bells & Whistles

Assemble sequences into animated gif

Different combinations of moving the subject and camera - Move camera & Change focal length (70mm-24mm)

Compute FOV:
With simple geometry, we can derive distance = width / (2 * tan(1/2 * FOV)) Rearrange terms and we get: FOV = 2 * arctan(width / (2 * distance))

FOV at maximum zoom f=70mm distance=91cm, width=51cm 31 degree

14.5'' 21'' FOV at minimum zoom f=24mm distance=37cm, width=53cm 71 degree

Here are the sequences created (stuffed animal and lego -- with both moving the camera and moving the image subject):

Card image cap

Card image cap
Card image cap


Seam Carving: Overview

Seam carving is an algorithm that resizes the image while being aware of the content and only touching important pixels as little as possible. In seam carving, we use an energy function to reflect the importance of every pixels on the image. Since human eyes are sensitive to large gradient (color changes, or edges), it is natural to define energy as the sum of gradient on x-axis and gradient on y-axis. In this way, large patch of same color will be shrunk first, and the gist of the image will be preserved.


Seam Carving: Method

We implemented a horizontal seam carving routine. A vertical seam is a connected path from the top border to the bottom. Each time, a vertical seam with smallest weight is selected and removed from the image and from the energy matrix (a height x weight array that records the energy of each pixel). Now the image is horizontally shrunk by 1 pixel. Then we can use the updated image and energy matrix for the next iteration. Doing it for hundreds of times effectively resizes the image. To do vertical seam carving, all we need to do is to transpose the original image and feed it to horizontal seam carving routine. Transposing the output back will give us the final result. Here is a set of results with the original image(left), its horizontal(middle) and vertical(right) seam carving:


Card image cap
Card image cap
Card image cap

Here are more results of horizontal seam carving (left--original, right--horizontally carved):


Card image cap
Card image cap

Card image cap
Card image cap

Here are more results of vertical seam carving (left--original, right--vertically carved):


Card image cap
Card image cap

Card image cap
Card image cap

Card image cap
Card image cap

In the example above, even though the geometry in original image is not preserved, all intersections are almost evenly distorted.


Failed Cases

In the below example, the roof is wrecked, even though snow in the bottom is less important. Such problem can be resolved if we add a heuristic part to the energy function, and make the algorithm more willing to discard that part.


Card image cap
Card image cap

Not quite a failure but if looked upon closely, in the below example, the algorithm fails to preserve the border of Cloud Gate in Chicago, because there is only one major color change, and most light weight seams cut through that place.


Card image cap
Card image cap
Card image cap


Reflections

For seam carving, we learned that such a simple algorithm can do great in resizing the image while tricking human eyes. The failed cases expose some problems of the algorithm, which are discussed in the paper (Seam Carving for Content-Aware Image Resizing). We can purposefully modify the energy function in order to keep some parts of the image untouched, or to trigger the algorithm to remove them.
For vertigo shot, we got to explore a lot about how moving the camera and/or the objects as well as playing with FOV can help us create a "horror-movie-like" effect.
For fake miniatures, we learned that by making use of blurring and depth of view / focus line, we can create very artistic results such as fake miniatures and fake stop motion animations.


Credits

Inspiration taken from friends, past projects, python libraries, papers and other online resources.