CS194-26 Project 4: Facial Keypoint Detection with Neural Networks

Sairanjith Thalanki | 3032739634 | sthalanki@berkeley.edu | cs194-26-adm

Part 1: Nose Tip Detection

For this section, I trained a convoluational neural network in order to detect the nose keypoint on the IMM Face Database. I split up the images into 192 for training set and 48 images for the validation set. Since I was working with such a small dataset, the model was not very accurate. The model especially failed on cases where the face was rotated to the left or right; it performed significantly better on the faces that looks straight ahead. Additionally, the model sometimes confused the lips for right beneath the nose; I believe this is because the model was looking for regions of high contrast since the keypoint is supposed to be situated right below an edge.

Model Architecture
Optimizer SGD w/ learning rate of 0.01
MSE Loss
25 epochs
Part 1 Training Loss
Part 1 Validation Loss
Example Successes:
Example Failures:

Part 2: Full Facial Keypoints Detection

For this section, I trained a convoluational neural network in order to detect all 58 keypoints on the IMM Face Database. I split up the images into 192 for training set and 48 images for the validation set. Since I was working with such a small dataset, the model was not very accurate. However, I improved the accuracy by doing transformations like color jitter which changed the brightness and contrast, like rotating the image by -15 to 15 degrees around the origin, and shifted the image left/right & up/down by 10 pixels. The model especially failed on cases where the face was rotated to the left or right like the nose keypoint detector. This model seems to run into similar issues as the nose kepypoint detector model. Specifically, when the face is rotated, the model is less effective and either rotates or shifts some/all of the keypoints.

Model Architecture
Optimizer Adam w/ learning rate of 0.001
MSE Loss
20 epochs
Part 2 Training Loss
Part 2 Test Loss
Example Successes:
Example Failures:
Learned Filters
Part 2 Learned Filters

Part 3: Full Facial Keypoints Detection on ibug dataset

This section was generally unsuccessful for me. I was unable to create a model that accurately detected face keypoints. It was able to detect a few keypoints on the lips and the left eye, however, the other keypoints would evaluate to the edges of the image. Unfortunately, I ran out of time and was unable to train a model that I could submit to Kaggle.

Model Architecture
Optimizer Adam w/ learning rate of 0.005
MSE Loss
5 epochs
Part 3 Training Loss
Part 3 Test Loss