Problem   1:  A   simple   face   detector
You are given a corpus of facial images [here] from the LFWCrop database. Each image in this corpus is 64 x 64 and grayscale. You must learn a typical (i.e. Eigen) face from them
You are also given four group photographs with multiple faces [here]. You must use the Eigen face you have learnt to detect the faces in these photos
The faces in the group photographs may have different sizes. You must account for these variations
Matlab is strongly recommended but you are free to use other programs if you want.
Some hints on how to read image files into matlab can be found here
You must compute the first Eigen face from this data. To do so, you will have to read all images into a matrix. Here are instructions for building a matrix of images in matlab. You must then compute the first Eigen vector for this matrix. Information on computing Eigen faces from an image matrix can be found here
To detect faces in the image, you must scan the group photo and identify all regions in it that “match” the patterns in Eigen face most. To “Scan” the image to find matches against an N x M Eigen face, you must match every N x M region of the photo against the Eigen face.
The “match” between any N x M region of an image and an Eigen face is given by the normalized dot product between the Eigen face and the region of the image being evaluated. The normalized dot product between an N x M Eigen face and a corresponding N x M segment of the image is given by E.P / norm(P), where E is the vector (unrolled) representation of the Eigen face, and P is the unrolled vector form of the N x M patch.
A simple matlab loop that scans an image for an Eigen vector is given here
The locations of faces are likely to be where the match score peaks.
Some tricks may be useful to get better results.
- Some of your test images (the group photograph) are in color; your Eigen faces are greyscale. You will have to convert the color photograph to greyscale by taking the mean of the red, green and blue values. The matlab method for doing this is given here.
- You will obtain better Eigen faces if all of the faces in the training data are histogram equalized. The faces in the training data all have somewhat different lighting and contrast. These variations can affect your estimate of the Eigen face. Histogram equalization can be performed in matlab as explained here.
- You will also be able to detect faces better if you histogram-equalize each patch of the group photo before you evaluate its match to the Eigen face.
Scaling and Rotation
The Eigen face is fixed in size and can only be used to detect faces of approximately the same size as the Eigen face itself. On the other hand faces in the group photos are of different sizes -- they get smaller as the subject gets farther away from the camera.
The solution to this is to make many copies of the eigen face and match them all.
In order to make your detection system robust, resize the Eigen faces from 64 pixels to 32x32, 48x48, 96x96, and 128x128 pixels in size. You can use the scaling techniques we discussed in the linear algebra lecture. Matlab also provides some easy tools for scaling images. You can find information on scaling images in matlab here. Once you've scaled your eigen face, you will have a total of five “typical” faces, one at each level of scaling. You must scan the group pictures with all of the five eigen faces. Each of them will give you a “match” score for each position on the image. If you simply locate the peaks in each of them, you may find all the faces. Sometimes multiple peaks will occur at the same position, or within a few pixels of one another. In these cases, you can merge all of these, they probably all represent the same face.
Additional heuristics may also be required (appropriate setting of thresholds, comparison of peak values from different scaling factors, addiitonal scaling etc.). These are for you to investigate.
Problem  2:   A   boosting   based   face   detector
You are given a training corpus of facial images. You must learn the first K Eigen faces from the corpus. Set K = 10 initially but vary it appropriately such that you get the best results. Mean and variance normalize the images before computing Eigenfaces.
You are given a second training set of facial images. Express each image as a linear combination of the Eigen faces. i.e., express each face as
where is the ith Eigen face and is the weight of the ith Eigen face, when composing face . can, of course, be computed as the dot product of and
Represent each face by the set of weights for the Eigen faces, i.e.
You are also given a collection of non-face images in the dataset. Represent each of these images too as linear combinations of the Eigen faces, i.e. express each non-face image as
As before, the weights can be computed as dot products. Represent each of the non-face images by the set of weights i.e..
The set of weights for the Eigen faces are the features representing all the face and non-face images.
From the set of face and non-face images represented by the Eigenface weights, learn an Adaboost classifier to classify faces vs. non-faces.
You are given a fourth set which is a collection of face and non-face images. Use the adaboost classifier to classify these images.
The classifier you have learned will be for the same size of images that were used in the training data (64 x 64). Scale the classifier by scaling the Eigenfaces to other sizes (32 x 32, 48 x 48, 96 x 96, 128x 128).
Train and test data for this problem is here. It is a collection of face and non-face data.Use the data in the "train" subdirectory to train your classifier and classify the data in the "test" subdirectory.
It will generally not be possible to represent a face exactly using a limited number of typical faces; as a result there will be an error between the face F and the approximation in terms of the K Eigenfaces. You can also compute the normalized total error in representation as:
errF = ||F - Σi wF,iEi||2 / N
where, ||.||2 represents the sum of the squares of the error of each pixel, and N represents the number of pixels in the image.
Represent each face by the set of weights for the Eigen faces and the error, i.e.
As in the case of faces, the approximation of the non-face images in terms of Eigenfaces will not be exact and will result in error. You can compute the normalized total error as you did for the face images to obtain errNF.
Represent each of the non-face images by the set of weights i.e..
Learn and build a classifier in the same way you did for problem 2 but including normalized error as a feature. Use this classifier for Problem 4.
Scan the group photographs to detect faces using your adaboost classifier
You can adjust the tradeoff between missing faces and false alarms by comparing the margin () of the Adaboost classifier to a threshold other than 0.
The homework is due at the beginning of class on October 23,2012.
What to submit:
- A brief writeup of what you did
- The segments that your detector found to be faces. You may either copy those segments into individual files into a folder, or mark them on the group photograph. Make sure we can understand which part(s) of the image was detected as a face.
- The code for all of this
Put the above in zipfile (tar is fine as well) called "yourandrewid_mlsp_hw2.zip" and email it to the two instructors with MLSP hw2 in the subject line
Face Recognition Assignment 1
April 13, 2004
DUE: April 21, 2004, 5PM.
In this assignment you willcreate 6 images of your face to fit all the specifications listed below. Theseimages will be used in a class database for your face recognition project.
General Image Specifications:
- All images will be frontal view, and in JPEG format. Use a plain white wall as your background.
- Convert images to 8-bit gray-scale.
- All images must be dimension 100x128 (width x height), and normalized such that the center of the eyes is fixed in all pictures. The pixel coordinates for the eyes will be (35,58) and (65,58). See the following figure for examples of image sizing and normalization. Note that this normalization is a manual procedure. You do not need to automate this.
You will create 6 images,following the specifications outlined above, with the filename convention‘abcde_X.jpg’, where abcde is the first 5 letters of your last name (alllowercase). If your last name is less than 5 letters, append an appropriatenumbers of zeros. The specifics of the 6 images are as follows:
‘abcde_0.jpg’—normalexpression (similar to the images in the figure above)
‘abcde_1.jpg’—another normalexpression (don’t just make a copy of ‘abcde_0.jpg’)
Reading:Start reading through the following papers. This will help you get an idea of existing algorithms for face recognition. M. Turk and A. Pentland, "Eigenfaces for recognition," Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 71-86, Winter 1991.download from: http://www.cs.ucsb.edu/~mturk/research.htm M. Turk and A. Pentland, "Face recognition using eigenfaces," Proc. IEEE Conference on Computer Vision and Pattern Recognition, Maui, Hawaii, 1991.download from: http://www.cs.ucsb.edu/~mturk/research.htm Peter Belhumeur, João Hespanha, David Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. Special Theme Issue on Face and Gesture Recognition of the IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(7):711-720, July 1997.download from: http://www.ece.ucsb.edu/~hespanha/published.html Peter Belhumeur, João Hespanha, David Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. In Proc. 9th Yale Workshop on Adaptive and Learning Systems, pages 225-234, June 1996.download from: http://www.ece.ucsb.edu/~hespanha/published.html Peter Belhumeur, João Hespanha, David Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. In Proc. of the 4th European Conf. on Comput. Vision, volume 1, pages 45-58, Apr. 1996.download from: http://www.ece.ucsb.edu/~hespanha/published.html Ming-Hsuan Yang, Narendra Ahuja and David Kriegman, "Face Recognition Using Kernel Eigenfaces", in Proceedings of the 2000 IEEE International Conference on Image Processing (ICIP 2000), pp. 37-40, vol. 1, Vancouver, Canada, September, 2000.download from: IEEExplore Bartlett, M.S., Movellan, J.R., & Sejnowski, T.J. (2002). Face recognition by independent component analysis. IEEE Transactions on Neural Networks 13(6) p. 1450-64.download from: http://mplab.ucsd.edu/~movellan/mypapers/BartlettMovellanSejnowski2002.pdf