First, I found some image of meryl streep from the website of the photographer that was linked in the project description. This will be my morph. I chose this because she looked very different from me. source
Here are the pictures:
Next, I highlighted some points in both images. This will be the keypoints.
Next, I found the average of those points and found the Delaunay triangulation for them. Here is the triangulation on both faces.
Visualizing the triangles
I started with the equation from lecture.
$$\begin{bmatrix} x'\\ y'\\ 1 \end{bmatrix} = \begin{bmatrix} a &b &c \\ d &e &f \\ 0 &0 &1 \end{bmatrix} \begin{bmatrix} x\\ y\\ 1 \end{bmatrix}$$That is just for 2 points. We can modify it to solve for $a$ to $f$:
$$\begin{bmatrix} Ax' & Bx' & Cx' \\ Ay' & By' & Cy' \end{bmatrix} = \begin{bmatrix} a &b &c \\ d &e &f \end{bmatrix} \begin{bmatrix} Ax & Bx & Cx\\ Ay & By & Cy\\ 1 & 1 & 1 \end{bmatrix}$$To solve it with numpy, we need it in the format of $A \cdot x = b$ where $x$ is the matrix from $a$ to $f$
Then, the transformation matrix is just the transposition of $x$, with 0,0,1 added to the bottom.
I first found the triangulation and points for the mid way face, by taking the average of both faces
In terms of interpolation, I used nearest neighbor. This is for practical reasons - the other ones are much slower.
In this part, I used the same code from 1.1 but used a loop instead.
Data is from source
After I imported the data, I showed the points so I can accordingly highlight my own face
And here is my own face, with the same points as the danes.
Triangulate the average points
I calculated the average points of the danes, then got a triangulation.
I also added the four corners to the points. This is so that the triangles could cover most of the image. I don't know if this was the best way to do so; however, if I didn't do it, some images will have double faces.
We could morph faces to the avg points by reusing the morph function from before. We can use the same image for both image parameters, but use the avg points for one of the points parameters
Wow, what a good looking face.
First, I morphed my face to the mean shape. Then, I morphed the mean face to my shape
Oh dear!! I think my image was too different from the original.
First, I exaggerated the shape of my face
Then, I exaggerated the pixels of my face.
To do this, I first morphed the mean face to my exaggerated shape:
Then I used the difference to boost the deviation of my face.
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
<matplotlib.collections.PathCollection at 0x16a80f104f0>
Morphing myself to the shape of an average female danish computer vision scientist.
Then, I morphed the average female face to my shape, and used its pixels to shift my pixels towards the female side.
Then I combined the results of the previous two steps
We do pca on the faces that are already warped to the average.
These are the components. I think the standout feature here... is whether or not the image is of my own face.
I tried not including my face in the PCA at first - however, the result of that made it nearly impossible to recreate my face from the principle components
My own coefficients, expectedly, was very heavy on the first component.
my_coefs
array([775.98168662, 19.39916899, 66.29862941, 43.73038386,
70.97378988, -41.56330815])
The coefficients of the average face:
avg_coefs
array([159.69758576, 32.55009573, 78.05092065, 73.18884917,
68.92953861, -46.70988823])
Reconstructing my exaggerated face from pca
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
