CS 194-26:Image Manipulation, Computer Vision and Computational Photography

Project 3:Face Morphing

Xuanbai Chen,cs194-26-ags

Overview

This project let us produce a morphing face,which use the affine transformation and interpolation and some other knowledge which can make a fun project.The process is just as what the instruction said.First,make correspodence;Second,make a mid-way face;Third affine(both color and shape). We should also get the mean face of an annotated faces and make comparision and having fun.he correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.We should get clear of the theory and then the result can be generated easily.

Part 1: Defining Correspondences

With the help of ginput in pyplot,we can make correspondence points easily.I use 43 points to correspondence as recommended.Besides,we should save the points into .txt file so that we can just read the file next time.(Don't forget 4 points in the corner)

Label:Order

After downloading the IMM_face dataset,I find the size of my face's image is different from the image in the dataset,so I just resize the image to (640*480)

Origin Images

Resize Images

Part 2: Computing the "Mid-way Face"

In this part,we should change the shape into the middle shape as well as the color one.MidShape = weight * ShapeA + (1 - weight) * ShapeB and in order to generate the mid-way face, the weight is 0.5.There are two ways to affine images,I choose the Algebraic one.By using three points of the images to ensure the parameter of the matrix.The math formulation is as below:

And in order to let the generated image have integer pixel after affining,I use inverse warp and compute the origin pixel values which are not integer by using interpolation:

Results:

Part 3: The Morph Sequence"

I write a function that produces a warp between im1 and im2 using point correspondences defined in im1_pts and im2_pts (which are both n-by-2 matrices of (x,y) locations) and the triangulation structure tri . The parameters warp_frac and dissolve_frac control shape warping and cross-dissolve, respectively. In particular, images im1 and im2 are first warped into an intermediate shape configuration controlled by warp_frac, and then cross-dissolved according to dissolve_frac. For interpolation, both parameters lie in the range [0,1]. They are the only parameters that will vary from frame to frame in the animation. For my starting frame, they will both equal 0, and for your ending frame, they will both equal 1.

Part 4: The "Mean face" of a population"

I use IMM_face dataset and use the .asf file which are annotated.The float number is the ratio of the height and width.The number of points are 58.Compute the average face shape of the whole population or some subset of the population - say, all the girls or all the old/young/white/asian/blond etc. .Morph each of the faces in the dataset into the average shape. Show some examples of mid-way faces.Compute the average face of the population and display it.

Some examples of mid-way faces:

Mean faces of the dataset:

The project also let us morph our face to the mean face and their face to our geometry.But one thing that I want to mention is that we have to ginput the 43 points again since the dataset is 58 points:

My face to mean face:

Mean face to my face:

Part 5: caricatures: Extrapolating from the mean

Produce a caricature of your face by extrapolating from the population mean you calculated in the last step. Face = weight * ShapeA + (1 - weight) * ShapeB. The point is that the weight should larger than 1.I just increase the weight from 0-1.75 and get a gif:

Weights from 0 to 1.75: