Final Projects

Part 1: A Neural Algorithm of Artistic Style

This portion of the project is based of the paper A Nerual Algorithm of Artistic Style which uses Deep Neural Networks to create artistic images of high perceptual quality. "The system uses neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images." The general process is depicted below.

The paper uses a modified VGG-19 with pre-trained weights. When computing content layers we use standard MSE loss and the style layers utilize the MSE of the gram matrix and the target image. Th optimizer used is the LBFGS. Pooling layers 1-5 are used for for the style of an image to compute the correlation of the different style features of an image. Content features are extracted only from a single layer (they choose 4_2 in the paper implementation, but other layers will work). They also modified the MaxPool layers to AvgPool instead. I tried both and didnt notice much of a difference. They used weights evenly spaced across all 5 layers, although it would be possible to manually shift the weight values to achieve different effects.

Results

Content Images are shown on the left, and style images to the right. Content Weights = 1, Style Weights = 100000.

These results turned out pretty well, I speculate due to the detail of the content image.

Heavy Content Weighting: Content Weight = 1, Style Weight = 1000

I tried weighting content more heavily, but the algorithm didn't seem to like that for this picture and results were not good

Content Weight = 1, Style Weight = 10000

I choose this weight here because I wanted to keep some of the original picture intact but still get the effect of long lines and brush strokes in the painting.

My dogs when they were puppies. This one was difficult and I had to increase the weight of the style in order to get any meaningful change in the output. I hypothesize this was due to the large uniform background of concrete.

Other Examples

Part 2: Poor Man's AR

For this segment, I will project a cube onto a box that has drawn points of interest.

The points of interest are uniformly spaced 1 inch apart. The box used in this experiment is a tissue box wrapped in white paper.

The image on the left is the first video frame in which points were selected, and the video on the right is the source video.