CS 194-26: Final Project

Karuna Wadhera, CS194-26-abo

Seam Carving

Seam Carving solves what seems to be an incredibly complex problem with a simple dynamic programming algorithm.

In my code, I define the "energy function" to be the magnitude of the horizontal gradient. I calculate this energy for all of the pixels in the image.

Then, I go through every pixel, row by row and calculate the "lowest energy from top". I do this by summing the energy of the current pixel with the smallest sum of a pixel in the row above me that is adjacent or diagonal to me (there are three choices). As I do this, I also set a "backpointer" to the pixel I chose in the row above me.

Then, when we're finished, I just need to locate the pixel in the last row that has the lowest energy from top. I follow the backpointers are remove all those pixels from the image.

I repeat this as many times as the number of columns I want to remove.

In order to remove rows, I just call upon my removeColumns function, but I pass in a transpose of the image and transpose the result that is returned.

Something I struggled with was speeding up this algorithm. One thing I did was instead of calculating the gradients manually, I used a kernel and convolved the image to come up with the gradients (thanks to a friend for reminding me about fftconvolve). I noticed my program was still really slow so I timed some different parts. I noticed a "np.argmin" call was also (surprisingly) the cause of a lot of the delay! I'd say figuring out the speed up was the most important thing I learned from this project.

Examples

Original Source: https://www.lonelyplanet.com/travel-blog/tip-article/wordpress_uploads/2016/02/Santorini-sunset_CS.jpg	Removed 100 Rows
Original Source: https://s.abcnews.com/images/International/dolphin-stock-gty-jef-180827_hpMain_16x9_1600.jpg	Removed 100 Columns
Original Source: https://wallhalla.com/thumbs/preview/y/yWbQgkbI1N8.jpg	Removed 100 Columns
Original Source: me	Removed 100 Rows
Original Source: https://www.nationalparks.org/sites/default/files/yosemite-merced.jpg	Removed 100 Columns
Original Source: https://www.planetware.com/photos-large/EGY/egypt-cairo-pyramids-of-giza-and%20camels-2.jpg	Removed 100 Columns

Didn't work :(

Seam carving doesn't work that well with faces! As you can see in the following two images, the proportions of the faces are incorrect! :(

Original Source: me	Removed 100 Columns
Original Source: me	Removed 100 Columns

Bells & Whistles

I decided to use a different energy function and test it out with the one I used to generate the rest of my images. Though using a horizontal gradient seemed to work just fine, I also experimented with a vertical gradient! I expected a horizontal gradient to do better with removing columns and a vertical gradient to do better with removing rows. However, this turned out not to be the case when I tested it empirically. For example, here are the spongebob image results side-by-side.

Original

Source: me

Horizontal Gradient

Vertical Gradient

Fake Miniatures

Seeing the fake mini sculptures of real-life scenes made with Legos or other material are always fascinating to look at. It's quite amazing that we can make real, full-size pictures look mini as well!

To be frank, the hardest part of this project was finding the right images to make this work on, especially when only using horizontal or vertical depth planes.

My process was to create many different images with different amounts of blurring. I did this by repeatedly applying a gaussian blur. Then, according to the distance between the pixel I'm looking at and the line the user selects, I'd decide which gaussian blurred image to use for that pixel. The farther the distance, the more blurred image I'd use. I'd do this for every pixel. Afterwards, I'd adjust the saturation to make the images look more miniature. Here are my results:

Images from the internet

Original	Miniature
Original	Miniature

Sources: https://i.pinimg.com/originals/cf/82/b7/cf82b7814f5de9a45d3eaa87c4a66690.jpg
https://www.lonelyplanet.com/travel-blog/tip-article/wordpress_uploads/2016/02/Santorini-sunset_CS.jpg

My own images

Original	Miniature
Original	Miniature
Original	Miniature

On the 3 example images

(Because there was a error on the link I went to that course page and the results there to figure out what the given images were).

Original	Miniature
Original	Miniature
Original	Miniature