CS194-26 Project 4

Facial Keypoint Detection with Neural Networks

Austin Kao

Part 1: Nose Tip Detection

For the first part, we will use the IMM Face Database available on this website for training an initial toy model for nose tip detection. simply image from the data loader, train/validation accuracy, and the result are shown below

<simple image with ground-truth keypoints 1>

<simple image with ground-truth keypoints 2>

<train and validation accuracy during the training process>

<incorrect image 1>

<incorrect image 2>

<facial images which the network detects the nose correctly 1>

<facial images which the network detects the nose correctly 2>

Part 2: Full Facial Keypoints Detection

in this section we want to move forward and detect all 58 facial keypoints/landmarks

<simple image with ground-truth keypoints 1>

<simple image with ground-truth keypoints 2>

<train and validation accuracy during the training process>

<incorrect image 1>

<incorrect image 2>

<facial images which the network detects the nose correctly 1>

<facial images which the network detects the nose correctly 2>

Part 3: Train With Larger Dataset

For this part, we will use a larger dataset, specifically the ibug face in the wild dataset for training a facial keypoints detector. the result, tho not as well as expected, are reported below. the mean absolute error is 28.6 on kaggle. I'm using resnet18 for this model with a step size of 0.00001. the training error as well as the result is shown below.

<training error>

<images with the keypoints prediction>

<images with the keypoints prediction>

<the trained model on my own image>

<the trained model on my own image>

<the trained model on my own image>