In this project, we create fake miniatures by simulating tilt-shift photography that is normally achieved using selective focus cameras. We are effectively narrowing the perceived depth of field in a scene and creates the illusion that it was taken with a lens was really close to the subject. To achieve this effect, we first create a mask around our desired subject. We then use the mask to blur the background such that regions that are farther from the subject are increasingly blurred.
Bells & Whistles
I implemented complex DOF regions using object masks. By using scikit-image's binary dilation function, we can grow the masked region so that we end up with multiple masks of different sizes. Then we can blur the defined background in regions, with regions that are farther from the subject being blurred more than regions that are closer to the subject.
Santorini, original
Santorini, mask
Santorini, miniaturized
Santorini, miniaturized & saturation-boosted
More Results
THe following images are more results of creating fake miniatures. The left images are the source images, and the right are the fake miniatures, where they are the blurred using dilated masks and then saturation-adjusted to increase the illusion of miniatures.
Berkeley, original
Berkeley, miniaturized & saturation-boosted
Safeco Field, original
Berkeley Field, miniaturized & saturation-boosted
Sproul Hall, original
Sproul Hall, miniaturized & saturation-boosted
Summary
This has been a fun project. I think this is very relevant with shallow DOF being the current trend and computional photography being integrated into every smartphone camera. It is amazing how DOF and percieved quality of a photo can be changed by a Python/Matlab script, altough the quality can never be as good as DOF from a lens due to imperfections of drawing masks. I think with edge detection and/or image segmentation algorithms, we can improve drawing masks for subjects we want to focus on.
In this project, we implement seam carving, a content-aware image resizing technique. A seam is a connected path from one side of the image to the other that chooses exactly one pixel from each column (or row). We remove seams until we end up with an image of our desired size. The resulting image should be a cropped image where we remove "unimportant" details.
Algorithm
First, we compute an energy map, which ranks pixels by "importance". For this project we use x and y gradient to compute an energy map.
Gradient energy function
Next we find a path of minimal total energy. We compute the minimal total energy using dynamic programming - we first traverse the image from the second-to-last row to the first row computing the minimal total energy up to each pixel while also keeping track of its neighbor of minimal total energy below:
Cumulative energy equation
Then we backtrack starting from the pixel with the smallest total energy in the first row all the way to the last row to obtain our seam to remove.
Results
Te following images are more results of seam carving. The left images are the source images, and the right are the resized images.
Vertical Seam Removal
Sky Tree, original
Sky Tree, with vertical seams removed
Hawaii, original
Hawaii, with vertical seams removed
Supertrees, original
Supertrees, with vertical seams removed
Horizontal Seam Removal
We can rotate the image to remove horizontal seams
Piling, original
Piling, with vertical seams removed
Hiroshima, original
Hiroshima, with horizontal seams removed
Failures
The choice of energy map and image to resize influence the quality of seam carving. Here, we used the gradient energy map on this image of a bullet train. Since the surface of the bullet train is smooth and curvy, some seams carve into it.
Shinkansen, original
Shinkansen, with vertical seams removed
Here it's easy to see where the seams were removed by the obscured braces.
Golden Gate Bridge, original
Golden Gate Bridge, with vertical seams removed
Bells & Whistles
Seam Insertion
I implemented seam insertion (described in paper by Shai Avidan and Ariel Shamir). The algorithm runs as follow: first find k seams found by seam carving. After collecting all the seams, we then add new seams next to the seams found in reverse order to the original origin, where the new artificial seams is computed by taking the average of seams to the left and right of the old one.
Sky Tree, original
Sky Tree, with vertical seams inserted
Supertrees, original
Supertrees, with vertical seams inserted
Object Removal
I also implemented object removal using seam carving and seam insertion. I first make a mask for the object I want to delete. I set the values in the mask such that the masking region for the object is negative and the rest of the mask to be zero. Then for seam carving I multiply the mask with the energy map, keeping the rest of the algorithm the same. Then I inserted vertical seams to resize the image back to the original size.
Hawaii, original
Hawaii, with tree removed
Sky Tree, original
Sky Tree, with Sky Tree removed and vertical seams inserted
Summary
Seam carving is a simple, yet functional approach to image resizing. Using the right energy function, it can work beautifully. The use of dynamic programming is elegant, although it can be computationally expensive if you want to remove many seams on large images.