Replace ur face with another one face in realtime video in c# (just for fun)

One of my past blog i write about how to detect and extract faces using live webcam, now using that , i m going to make a funny software which detect the face and replace it by ur desired face or image whatever u like it, its not to perfect but fun to see.

first using emgucv harrdetection technique i detect the the face and extract it, here is the summery of how i did it.

So first step is download the emgucv dll files because we are going to use it in our code. After download just add the dll in reference of ur project solution.

Now we need a very important file which is used as template to detect face we called it face_template, It is a xml file, u can get it from here . secondary link is here.

After download the xml file saved in “bin” folder and named it “face_template.xml” .

Now prepare a simple gui, A windows from, two picture box and a timer which is enabled at interval not less than 100.

first step is declare the library just add following

using Emgu.CV;
using Emgu.CV.Structure;
using Emgu.CV.CvEnum;

second step is initialize the web cam and load my detection template. U can just write this simple code for that

private Capture webcamvideo;
private HaarCascade facedetection;

 public Form1()
 {InitializeComponent();
 webcamvideo = new Capture(0);
 facedetection = new HaarCascade("face_template.xml");
 }

Now timerclick event which will take the webcam captured image one after one. first see the full function code then analyze it.

private void timer1_Tick(object sender, EventArgs e)
{
 using (Image<Bgr, byte> Frame = webcamvideo.QueryFrame())
 {
  if (nextFrame != null)
{
 Image<Gray, byte> grayframe = Frame.Convert<Gray, byte>();
var faces =grayframe.DetectHaarCascade(facedetection , 1.4, 4,HAAR_DETECTION_TYPE.DO_CANNY_PRUNING,new Size(Frame.Width/8, Frame.Height/8))[0];
foreach (var face in faces)
{
 Frame.Draw(face.rect, new Bgr(Color.Red), 3);
 Bitmap c = Frame.ToBitmap();
 Bitmap bmp = new Bitmap(face.rect.Width, face.rect.Height);
 Graphics g = Graphics.FromImage(bmp);
 g.DrawImage(c, 0, 0, face.rect, GraphicsUnit.Pixel);
 pictureBox2.Image = bmp;
 bmp.Save("myface"+DateTime.Now.Second.ToString() + ".jpeg");
 }
 pictureBox1.Image = nextFrame.ToBitmap();
}
 }
}

Now see what code does

using (Image<Bgr, byte> Frame = webcamvideo.QueryFrame())

it takes a frame from current web cam video stream. after .1 seconds it gets another frame because of timer function interval set to 100, if we want to fastened it we have to use threading.

if (Frame != null)

by this code we certain that we get a picture and can start analysis.

This analysis can divide in two steps.

• detact face
• extract the face

In detect face step we first make the image in greyscale using the following code

Image<Gray, byte> grayframe = Frame.Convert<Gray, byte>();

Now we take a special variable ‘var’ which a special type of Emgu.cv.ml. it can takes all the face location . and we use grayframe.DetectHaarCascade() function which take our template file , edggedetection algorithom type, scale factor and size of the part to except . So the code is like

var faces =grayframe.DetectHaarCascade( facedetection , 1.4, 4,

HAAR_DETECTION_TYPE.DO_CANNY_PRUNING,

new Size(Frame.Width/8, Frame.Height/8)

)[0];

now we have to create extracted image of face. “faces” has all of faces information if multiple face have. To extract we use

foreach (var face in faces)
{
Frame.Draw(face.rect, new Bgr(Color.Red), 3);
Bitmap c = Frame.ToBitmap();
Bitmap bmp = new Bitmap(face.rect.Width, face.rect.Height);
Graphics g = Graphics.FromImage(bmp);
g.DrawImage(c, 0, 0, face.rect, GraphicsUnit.Pixel);
pictureBox2.Image = bmp;
bmp.Save("myface"+DateTime.Now.Second.ToString() + ".jpeg");
 }

here frame.draw() draw the rectangle in the main frame.Graphics g is used to draw the extracted face in new image file which is bmp.

So using this way we can extract the face from live video like this image

now we have to replace the image with our selected image to select image u have to press the button
and browse the image after select the image my software will check that it has a face or not, if it has face it only keep the extracted face and if it had no extracted face it will keep the full image here is the code for that.

OpenFileDialog o = new OpenFileDialog();
            if (DialogResult.OK == o.ShowDialog())
            {

                Bitmap iss = new Bitmap(o.FileName);
                Image nextFrame = new Image(iss);
                 Image grayframe = nextFrame.Convert();
                    var face = grayframe.DetectHaarCascade(
                                    haar, 1.4, 4,
                                    HAAR_DETECTION_TYPE.DO_CANNY_PRUNING,
                                    new Size(nextFrame.Width / 8, nextFrame.Height / 8)
                                    )[0];
                 //   MessageBox.Show("df");

                    if (face.Length > 0)
                    {
                        Bitmap c = nextFrame.ToBitmap();
                        Bitmap bmp = new Bitmap(face[0].rect.Width, face[0].rect.Height);
                        Graphics g = Graphics.FromImage(bmp);
                        g.DrawImage(c, 0, 0, face[0].rect, GraphicsUnit.Pixel);
                        pictureBox3.Image = bmp;
                    }
                    else
                        pictureBox3.Image = iss;
                             }

the explanation of above coding is same as previous extraction coding. Now for replace face with another face u have to checked the check box named change face,
here i write simple below code here j used as checkbox status,

  private void timer1_Tick(object sender, EventArgs e)
        {
            using (Image nextFrame = cap.QueryFrame())
            {
                if (nextFrame != null)
                {

                    Image grayframe = nextFrame.Convert();
                    var faces =
                            grayframe.DetectHaarCascade(haar, 1.4,4,
                                    HAAR_DETECTION_TYPE.FIND_BIGGEST_OBJECT,
                                    new Size(nextFrame.Width / 8, nextFrame.Height / 8)
                                    )[0];
                 //   MessageBox.Show("df");

  if (faces.Length > 0)
  {
      if (j == true)
      {
          nextFrame.Draw(faces[0].rect, new Bgr(Color.Red), 3);
          Bitmap c = nextFrame.ToBitmap();
          Bitmap bmp = new Bitmap(faces[0].rect.Width, faces[0].rect.Height);
          Graphics g = Graphics.FromImage(bmp);
          g.DrawImage(c, 0, 0, faces[0].rect, GraphicsUnit.Pixel);
          //  g.Dispose();
          pictureBox2.Image = bmp;
          PictureBox p = new PictureBox();
          p.Image = bmp;
          p.Height = bmp.Height;
          p.Width = bmp.Width;
          imageList1.Images.Add(bmp);
          flowLayoutPanel1.Controls.Add(p);
          listView1.Items.Add(index.ToString(), index);
          pictureBox1.Image = nextFrame.ToBitmap();
      }
                        else
                        {
                            Image ia = nextFrame.Clone().ToBitmap();
                            Graphics g = Graphics.FromImage(ia);
                            g.DrawImage(pictureBox3.Image, faces[0].rect);
                            pictureBox1.Image = ia;

                        }
                        index++;

                    }
                    if (faces.Length == 0)
                    { pictureBox1.Image = nextFrame.ToBitmap(); }
                }
            }

        }

check checkbox and see the result take the fun:)
the code to replace face is

 Image ia = nextFrame.Clone().ToBitmap();
                            Graphics g = Graphics.FromImage(ia);
                            g.DrawImage(pictureBox3.Image, faces[0].rect);//change faces
                            pictureBox1.Image = ia;

now just move ur face and see another one face is moving 🙂 one thing if u wear a glasses in ur eyes my face detection will not work.the complete source code is here