Content Loss
This was a really cool project that involved implementing a paper that can transfer the style of one image to another
In order to use gradient descent to change the style of an image, we first have to define loss functions for both the style and content of an image. From the paper, I implemented the following loss function:
Content Loss
For image style, the loss function is as follows:
Style Loss
For this project, I stuck to the paper and used a VGG-19 network.
In order to backprop through the network and add to the loss, we insert these loss layers in between various convolutional layers. I chose to use the method (e) from the paper to decide which layers to put loss functions after.
For actually running images through the network, I chose to use the Adam optimizer over the image pixels. I then summed the content/style losses from the corresponding layers and weighted the style losses by a factor of 1e6, as recommended in the paper.
After Following the procedure, I obtained the following result:
Style Image
Content Image
Result Image
For this project, we used gradient domain fusion to seamlessly blend a source image into a target image
For this part, the goal was to use practice setting up and solving a sparse system of linear equations. Because the constraints are such that the resulting matrix will be sparse, we can solve the system much faster using scipy's optimizations.
For every pixel in the image, we have 2 equations, to minimize the diference x and y gradients between the source and target. Additionally, we have a final equation to match the top left corner pixel in order to make the system have one exact solution.
Toy Problem Equations
Since this is a trivial problem and we have the same source/target we expect the result to be identical, as pictured below
Toy Problem Results
This part is similar to the last, but we use different blending constraints, shown below:
Poisson Blending Equations
To implemnent this I again used a similar method to the previous part, iterating over all pixels in our target region and adding an equation to the system.
Because this method is more forgiving than our previous project, I used rectangular masks to blend the images.
Poisson Blending Results 1
Poisson Blending Results 2
Overall, I really enjoyed this project and learning about how to setup blending as an optimization problem in order to get better results than previous methods we've used.