CS 194-26 Project 4: Image Classification & Semantic Segmentation

Reini Lin

Part 1: Image Classification

Overview

The purpose of the project was to classify different images in the FashionMNIST data set. The network was trained using the following architecture:

The loss function was cross-entropy loss, and the optimizer was Adam. The net trained on 3 epochs, eventually reaching accuracy levels of ~91%.

An example of the ground truth and predictions from the classifier are below.

Ground truth: Bag Pullover Pullover T-shirt/top

Predictions: Bag Pullover Pullover Shirt

Accuracy

The training and validation accuracies over time are visualized in the graph below. Accuracies were computed on sets of 2000 images. As expected accuracy for the training set is slightly higher than that of the validation set.

Accuracy per class for the validation set:

Accuracy per class for the test set:

The shirt class had the lowest accuracy for both the validation and test set. Here are examples of images the network classifies correctly and incorrectly - for each class, two correctly classified images are shown as well as two incorrectly classified images with their ground truth labels.

Filters

Below, the 3x3 filters have been visualized.

Part 2: Semantic Segmentation

Overview

The purpose of Part 2 is to train a network to segment an image into one of five classes: facade, pillar, window, balcony, and other. The optimizer used was Adam (learning rate 1e-3, weight decay 1e-5), and the loss function was cross-entropy loss. The architecture of the CNN is detailed in the following section.

Architecture

The layers used are below:

Evaluating Loss

The graph below illustrates the loss over time as the net trains over 20 epochs.

Average Precision

The reported values for the average precision of the network are below.

Example Image

The sample image below illustrates how the net would classify the parts in an image.

Left: original image, right: result.

The net mostly correctly classifies the facade and windows, but there are some red markings for balconies in areas where there is not a defined balcony.