Final Project: Seam Carving

Overview

In this project, we investigate methods to resize images without stretching or squashing things that we don't want squashed or stretched. This is an imitation of a dynamic image resizer like a browser webpage. For this effect, all you need to do is remove seams of minimum energy. Seams are paths that can be either horizontal or vertical. This methodology is simple yet flexible, and it was a lot of fun learning about seam carving!

My implementation

Step 1: computing the energy

To first find what seams to remove, we need to know how the image translates to energies. We imagine the image to be a map with pixels translating to terrain and elevations. I used an energy function that involved taking the difference of the left pixel and right pixel, squaring the value, and adding that with the square with of the difference of the top pixel and bottom pixel. This energy includes both the horizontal and the vertical channels, so differences are diagonally dependent. We do this for every pixel in the image.

Step 2: Finding the seam

Finding the seam is pretty hard. I initially thought it was just finding the minimum energy from the top row or left column, and moving to the minimum of its adjacent pixels. However, the intended implementation is as follows: from each pixel that's not in the leftmost column or the topmost row, we add the minimum energy of the pixel that could've been in the route that led to that pixel. In the end, we have a cumulatively increasing 2D array steadily downward or rightward. Then, we backtrack by going to the end of the 2D array, finding the minimum cumulative sum, and backtracking the pixel that contributed to adding it.

Step 3: Removing the seam

Removing the seam is relatively easy. We know that, if we horizontally seam carve, we have one coordinate for each column, and if we vertically seam carve, we have one coordinate for each row. All we need to do is omit that one value from each column or row when we resize.

Some carves below!

Places where my seam carving broke

Below are some examples of trying to seam carve on people's faces. As you can see, they don't look so hot. Because the seam carving algorithm doesn't see images as a bunch of objects, a person's face that has a lot of the same pixels will have the same energy. As you can see, faces get deformed, strips of clothing are omitted, and full on parts of the body are completely gone. One way to get around this is to go into the energy function and explicitly make those weights so positive that there will be no seams that will intersect with the face. Below are some images of my follies

My thoughts on this project

This project was a lot of fun! I want to continue seeing what I can do with a different energy function, or manually changing energy values to see what kinds of carves I can make.