CS194-26 Final Projects

Yao Fu, cs194-26-aen

Fall 2018

Seam Carving

Overview

In this project, we will implement the amazing algorithm of seam carving described in the paper Seam Carving for Content-Aware Image Resizing by Shai Avidan and Ariel Shamir, which resize an image while preserving its main content.

Algorithm

Here is a brief overview of the algorithm to shrink an image horizontally:

Define an energy function and for each image, calculate the energy value of each pixel.

Apply dynamic algorithm to calculate paths(seams) from the first row to the last row and find a path with minimum cumulative energy and delete all the pixels in this path.

Iteratively apply the same algorithm until we get the size we want.

For vertical shrinkage, simply rotate the image and do the same as above and rotate back in the end.

Some results by using e1 as the energy function:

Original

shrink horizontally

Original

shrink horizontally

Original

shrink vertically

Original

shrink horizontally

Original

shrink vertically

Original

shrink vertically

shrink vertically

Original

Bells & Whistles

It seems that some energy functions may be better than others for certain scenes. Here, I apply 3 kinds of energy functions: using derivative, entropy, and HoG, and here are some comparison between them:

Various energy functions

Original

Derivative

Entropy

HoG

Original

Derivative

Entropy

HoG

In order to determine which energy performs better, we may first apply each of them for several times and calculate the change of energy compared to the original average energy. The one that change the average energy of the image the least should be chosen.

We can also do the seam insertion using the method mentioned in the paper. We use the same algorithm as in seam removing. First, select the first n seams that should be removed. However, we do not remove them in this case. Instead, beside each seam, we insert a new seam by averaging the pixel value of its two neighbors. Here are the results:


Seam Insertion

insert vertically

Original

insert horizontally

Original

Original

insert vertically

This project is very inspiring for that it provides a better way to resize an image simply using code.

What I have learned

Failure Cases

However, there are still failure cases:

In these cases, the main objects cover a great portion of the picture thus the cumulative energy of every seam are close. Deleting every seam is a great loss.

There are also cases where some energy function perform pretty bad. For example, HoG perform bad on this image.

Original

shrink vertically

Original

shrink vertically

Entropy perform very bad on this one.

Fake Miniatures

Overview

In this project, we try to simulate the effect of Tilt Shift to produce some fake miniatures. The basic idea is to select a focus plane manually and for the rest of the image, apply a blurring filter.

Algorithm


Here is a brief overview of the algorithm:

Set a focus line by choosing a point in the image and produce a horizontal or vertical line that it lies on.

Define the size of the fake depth of field (DOF). For a horizontal line, it should be several pixels below and above the focus line, and for the vertical one, several pixels on the left and right side of the focus line.

Calculate a Gaussian Stack.

For the pixels further from the focus line, they should be more blurry. Thus, we should calculate the distance between each pixel from the focus line and then change the pixel values to the corresponding image in the Gaussian Stack.

Change the image from rgb format to hsv and enhance saturation and value to produce better effect. Then change back.

Here are some results:

Using my own images

horizontal depth line

horizontal depth line

vertical depth line

vertical depth line

Using images from the internet

horizontal depth line

horizontal depth line

horizontal depth line

horizontal depth line

vertical depth line

original

original

original

original

original

original

original

original

original

Bells & Whistles

Here is a fake stop motion video. Source: A Glance at Beijing 坐標·北京 Beijing time lapse - 4K


What I have learned

This project is so cute that we can use simple technique to produce fake miniatures. Also, I learned how to make a stop motion music video in python now lol.

!Thank you!