CS 194-26 Final Project - Seam Carving & Miniatures

by Max McArthur


Part 1: Seam Carving

Seam carving, as a process, creates images that are essentially "cut" to smaller sizes with a more content-aware method than simply cropping or scaling the image. The idea is that we can find "seams" to remove from the image by assigning each pixel an energy and finding the minimum "energy" seam. Essentially all the difficulty involved in this project comes from choosing this energy function, as the code itself is rather simple and only involves one dynamic programming algorithm. I tried a couple energy functions, which worked more on some images than others, as will be seen below:

Energy Function 1: abs(dX) + abs(dY)

The idea behind this energy function is that, where the image is changing a lot, there will be a lot of "important" pixels; this means that, when the seam carving algorithm attempts to find a seam through the cumulative energy map, it will try to find one that intersects the least prominent "energy lines". This seemed to work well for many landscape images, but not as well for images that have a lot of fine detail involved, as can be seen below:

Original Image Energy Map First Seam Removed 50 Seams Removed 100 Seams

As we can see from the energy map, there are clear areas that the seams will almost never pass through if given the choice (where the derivatives are high, at the edges of the mountains). The energy function works very well for this image because there is a single object focused on in the image, and there is plenty of space on either side that could be removed without making a huge visual impact. A few more examples of this function at work, some better than others:

Decent! Pretty awful.

So how can we improve on this energy function? My best idea was to, instead of doing a raw derivative, to instead take the gaussian and then derivative (or convolve derivative matrices with the gaussian and use that):

Energy Function 2: dX(gaussian) + dY(gaussian)

Original -100 px

I don't have any examples of it, but this will also work for vertical seams (i.e. seams that cut the height of the image), as I simply transpose the image before processing if we want to work with vertical seams.

Part 2: Fake Miniatures

The programming of this project was fairly simple; I made the DOF choice interactive, so that the user will define the part of the image that should be "in focus" as well as the depth of that focus. As a result, we essentially blur the image recursively, starting at the center of the region defined and spreading out, applying a gaussian with increasing sigma more and more as we move further away from that center. Specifically, the algorithm removes strips equal to half of the region defined by the user, applies a gaussian to them, and then recursively calls itself until it cannot remove a region of that size.

Original Minified
(last two images courtesy of NYCDOT and https://www.ncel.net/2017forum/boston-skyline/ )