Three Methods
There are three ways for texture synthesis: random, overlapping, and seam-finding. Here are their results, we can see seam-finding produces the most natural picture.
random
overlap
seam-finding
Seam-finding illustration
Since I am not using provided matlab code, I coded my own version of seam-finding. Here are some graphs that will help the illustration.
ssd driect paint
up patch
down patch
mask
More synthesis results
Here I will explain how I my synthesis method. First, I random generate the first patch. Then I loop through the output canvas from left to right, top to bottom.
When applying seam finding, I first use ssd_patch function to get a error matrix. One special design is that I just let the ssd add twice when there are 2 overlaps. See ssd driect paint above. Each index element means how bad my patch would be if I start my patch from that index in the orginal image.
Then I pick the index with the min value to generate a patch. When putting up that patch, up patch and down patch above, I go through the element-wise SSD in the overlap region from left to right.
In each column, I pick the index with min ssd, and start our new patch's according column from that index. In mask above, the black region means down patch part, while white region means up patch parts.
Texture transfer
I tried to use sketch and face to produce a transfer, results are not very good
Bells
I did the first Bells, which is defining my own quilt_cut method. In this method, I iterate through the overlap region's ssd matrix, and find a least cost path. With the index of the path, I use fill function to fill the patches with 1 and 0 to create a new patch that can directly added on the new position of the output.
Project Two: A Neural Algorithm of Artistic Style Model structure
Here is my model structure and forward function. Notice Here, I only used 5 conv layers. Also, instead of outputing an image, I interseact the results right after the conv layers. I know the paper suggests we should intersect the result
I trained for 300 epochs for all pictures. Like the paper suggested, I assigned a weight 1000 to style loss and 1 to content loss. I preprocessed the image with mean=[0.485, 0.456, 0.406] and std=[0.2, 0.2, 0.2]. For both style and content losses, I used MSELoss.
As the paper suggested, I replaced MaxPool with AvgPool Neckarfront
starry night
Composition VII
Picasso's Femme nue assise
Comparing my result with the ones in the paper, I think my results caputres the style of the painting. Starry night and Femme nue assise are less good, but Composition VII is better. However, the color is a little bit dark. I tried multiple mean and std and this is the closest I have.
Own Pics and Style
style
content
success
style
content
failed
style
content
success
I think the middle one failed because sunflower's style really does not fit the blue sky. The color is too contrast. Hence the result is hardly human readable.
