Facial Keypoint Detection with Neural Networks

Nir Levin

Part 1: Nose Tip Detection

Dataloader Samples

Loss plots

Validation Examples

Here are some examples of my validation tests. Green is ground truth and blue is prediction. Most of these are fairly successful, except for the man with the spiky hair, which is probably what threw the model off.

Changing Hyperparamters

My hyperparameters for the above training included: 5 convolutional layers with 20 channels each, 5x5 filters, and a learning rate of 0.001.

Part 2: Full Facial Keypoints Detection

Dataloader Samples

Training and validation Losses

Architecture and Results

Unforunately, I could not figure out a way to not have an underfit model, as the resulting model would return very similar points for all inputs. If I had more time, I would try to use different types of layers instead of convolutional ones. Hyperparameters include: 18 convolutional layers with 32 channels each and 5x5 and 3x3 filters.

Part 3: Train With Larger Dataset

My kaggle score: 19.18537


The model I used is a stoch ResNet18 pretrained model. The only changes I made to it were the number of input channels to the first convolutional layer, and the number of output channels from the last fully connected layer.

Training and validation Losses

Some Testing Set Results

Testing my own photos

Since I only used 10 epochs, the underfitting might be the cause for the lackluster results here. However, the model is still generally able to detect certain facial features, such as the cluster of points on the mouth in in each image.