For this project, I follow the methods from the paper A Neural Algorithm of Artistic Style from Gatys, Ecker and Bethge. I create a neural net with PyTorch which can capture the artistic style of one image and apply the style to another image.
I use the VGG19 model architecture from PyTorch for constructing the neural network. The initial network has 16 convolution layers and 5 max-pooling layers. I deleted all fully-connected layers. After that, following the suggestion from the paper, I use two loss functions to analyze the network’s performance, which are the content loss and the style loss. Below is the architecture details:
| Neural Network Structure |
|---|
|
Below are some results from adopting artistic style from paper (the Starry Night, the Picasso, and the Composition VII) applied to my own selected image: food. The leftmost is the original food picture, the artistic style comes from the art works in the middle column, and the final results are shown in the right column.
| Food | Starry Night | Starry-food |
|---|---|---|
 |
 |
 |
| Food | Composition VII | VII-Food |
|---|---|---|
|
 |
 |
| Food | Picasso | Picasso-food |
|---|---|---|
|
 |
 |
Finally, I tested my algorithm with more images. Below are three examples. All pictures come from Google. As we can see, the algorithm successfully captures the artistic style (includes the color, the texture, and the blending details) from the pictures in the second column.
| Sky Wheel | Sunflowers | SunWheel |
|---|---|---|
 |
 |
 |
| Berkeley | Sea | BerkeSea |
|---|---|---|
 |
 |
 |
| Lavender | Gummy Bear | Lavender Flavored Gummy Bear |
|---|---|---|
 |
 |
 |
This is my favorite project in CS194. It is a decomposition of artistic and aesthetic elements of art work. Also, it adds different skin to different flesh and bones. This is fantastic!
The second project uses images from the Stanford Light Field Archive. It shows how can we mimic aperture adjustment and depth refocusing from shifting and averaging a set of images. Thanks to the Computer Graphics Laboratory at Stanford University for acquiring several light field images for research in computer graphics and vision.
Objects which are far from the camera will not change their postion significantly when the camera moves with a fixed optical axis. At the same time, the objects that closer to the camera change their position significantly. From the Jellybean example, we can see that by averaging the images, we can get get an image that has a clearer view for the near beans and more blurry view for the further ones. Following this idea, we can generate multiple images that focus on different depth of the image. By shifting the images corresponding the the center image and applying increasing weights, we can change the focal points. Sample results are shown below:
| Original | Depth Refocusing |
|---|---|
 |
 |
 |
 |
Through aperture adjustment, we can simulate the varing aperture sizes of light field camera. The more pictures we averaging together, the larger the aperture. The less pictures we averaging accross, the smaller the aperture. The reason behind such facts is that a larger aperture can capture more lights, which is also capturing more image positions and information. Sample results are shown below:
| Original | Aperture Adjustment |
|---|---|
|
 |
|
 |
I decided to take a set of pictures to simulate a lightfield camera. I took my pictures with a 3x3 grid. After that, I generate depth refocusing and aperture adjustment images according to my images. I tried my best to differentiate pictures from each other by focuing on different points in the image. However, the results is not as good as the previous ones. There is some technical limitation in this process, which includes the camera and the number of pictures I can take to form a set. Considering the technical limitations, the results seems reasonable and can show evidence of the Depth Refocusing and the Aperture Adjustment.
| Original | Refocusing | Adjustment |
|---|---|---|
 |
 |
 |
This is the very last project for CS194. It is amazing and brings me a deeper understanding of camera. I learned a lot from CS194 and will definitely recommand it to friends! Many thanks to the hard work of Professor Efros and the course staffs. Wish you all have a great winter break!