Recognize face orientation using a new approach based on eye position and size by c#

Author: Md. Salahuddin Yousuf & Kishor datta gupta

We are trying to develop a method to recognize face orientation using OPENCV harr detection and neural network  . The process can be done without neural network but we use neural network for better performance and trying implement a new technique in pattern recognition.

Our technique is depend on 4 features of a face – They are

• Right Eye
• Left Eye
• Mouth
• Nose

We use respective distance and size of these feature’s for finding the orientation of the faces. We draw some pattern from these distance in a image file, and then send that image file to neural network.

The algorithm is

To see the patterns it can explain better how it works

Our pattern just draw some lines by using positions and length of feature items. if input image is like below

Then we get pattern like below

For coding we use C# languages. Its true c# is not good as matlab but we find it more applicable.
we also use EMGUCV which is a c# wrapper of OPENCV and For neural network we use BACKPROPAGTION Neural network which is entirely folowed the code of Murat Firat which u can get easily from his codeproject article.

Now starting the explanation how the code done. when a person show himself in a web camera or capture a image file send, it detect the human face from these image and extract only the faces . I explain the part of code before in here.

Now, we save detected face image’s in a folder and start to extract features. Process was simple , The code is

DirectoryInfo Folder;
Folder = new DirectoryInfo(filepath);
Images = Folder.GetFiles(".", SearchOption.TopDirectoryOnly);
string newPath = System.IO.Path.Combine(filepath, "profilelines");
System.IO.Directory.CreateDirectory(newPath);//creating folder for pattern
for (int i = 0; i < Images.Length; i++)
{
{
if (Images[i].FullName.Contains(".jpg") || Images[i].FullName.Contains(".jpeg"))
{//resizing images
Bitmap bmpResized = null;
Console.Write(Images[i].Name);
Bitmap bmp = new Bitmap(Images[i].FullName);
int resizedW = 320;
int resizedH = 320;
bmpResized = new Bitmap(bmp, resizedW, resizedH);
bmp.Dispose();
Images[i].Delete();
bmpResized.Save(filepath + "\\" + Images[i].Name, ImageFormat.Jpeg);
bmpResized.Dispose();
//create image files for pattern
Bitmap graph = new Bitmap(400, 400);
Graphics g = Graphics.FromImage(graph);
g.Clear(Color.White);
graph.Save(filepath + "\\profilelines\\" + Images[i].Name, ImageFormat.Jpeg);
}
}
}

Now code for extracting lefteye and draw pattern from left eye data
int inf = 0;
for (int i = 0; i < Images.Length; i++)
{
{
if (Images[i].FullName.Contains(".jpg") || Images[i].FullName.Contains(".jpeg"))
{
haar = new HaarCascade("haarcascade_mcs_lefteye.xml");
Bitmap bmpResized = null;
Bitmap bmp = new Bitmap(Images[i].FullName);
Image nextFrame = new Image(bmp);
Image grayframe = nextFrame.Convert();
var lefteye =
grayframe.DetectHaarCascade(haar, 1.4, 4,
HAAR_DETECTION_TYPE.FIND_BIGGEST_OBJECT,
new Size(nextFrame.Width / 8, nextFrame.Height / 8)
)[0];
if (lefteye.Length > 0)
{
{
if (lefteye[0].rect.Height * lefteye[0].rect.Width < 120 * 120 && lefteye[0].rect.Location.X < 130)
{
Bitmap face = new Bitmap(lefteye[0].rect.Height, lefteye[0].rect.Width);
Image ia = nextFrame.Clone().ToBitmap();
Graphics g = Graphics.FromImage(face);
g.DrawImage(ia, 0, 0, lefteye[0].rect, GraphicsUnit.Pixel);
//draw pattern
Bitmap b = new Bitmap(filepath + "\\profilelines\\" + Images[i].Name);
Graphics g2 = Graphics.FromImage(b);
Pen pen1 = new Pen(Color.Black, 3);
SolidBrush b5 = new SolidBrush(Color.Black);
int x = lefteye[0].rect.Location.X;
int y = lefteye[0].rect.Location.Y;
int h = lefteye[0].rect.Height;
int w = lefteye[0].rect.Width;
Point pp = new Point();
pp.X = Convert.ToInt32(x + w / 2);
pp.Y = Convert.ToInt32(y + h / 2);
Font f = new Font(Font, FontStyle.Bold);
Point p3 = new Point();
p3.X = Convert.ToInt32(x + w / 2);
p3.Y = Convert.ToInt32(y);
Point p4 = new Point();
p4.X = Convert.ToInt32(x + w / 2);
p4.Y = Convert.ToInt32(y + h);
Point p1 = new Point();
p1.X = Convert.ToInt32(x);
p1.Y = Convert.ToInt32(y + h / 2);
Point p2 = new Point();
p2.X = Convert.ToInt32(x + w);
p2.Y = Convert.ToInt32(y + h / 2);
Point[] pon = new Point[5];
pon[0] = pp; ;
pon[1] = p2;
pon[2] = p3;
pon[3] = p4;
pon[4] = p1;
g2.DrawPolygon(pen1, pon);
lefteyepositionx[inf] = len(pp.X, pp.Y, p2.X, p2.Y) + len(p2.X, p2.Y, p3.X, p3.Y) + len(p4.X, p4.Y, p3.X, p3.Y);
lefteyepositiony[inf] = lefteye[0].rect.Width;
b.Save(filepath + "\\profilelines\\a" + Images[i].Name, ImageFormat.Jpeg);
g2.Dispose();
b.Dispose();
File.Delete(filepath + "\\profilelines\\" + Images[i].Name);
File.Move((filepath + "\\profilelines\\a" + Images[i].Name), (filepath + "\\profilelines\\" + Images[i].Name));
}
}
}
bmp.Dispose();

write same kind of code for righteye, nose and mouth just changing the xml template. u will get all the template from emgucv site.

it is the code for draw the pattern, in next part i will write the code of neural network and explain how neural network trained by these pattern and recognizing it