Detect and track objects in live webcam video based on color and size by c#

In my last post i explain how EuclideanColorFiltering work . here i use that in my webcam for track and detect i made a realtime tracker could be made where color and objects size could also be changed in real time. and can draw movement of that objects in a bitmap with that color. i use some part of their code, although i use separate color filter for more accuracy.

First how my software work first can start tracking by default color black and color range of 120 but u can change in real time to color change press that a color dialog will come , then select color, remember to press OK and see the solidcolor box that Ur required color is present or not.

To select the range and define minimum size of the object see next image, there are 3 numeric updown list for that. objectsize calculated in pixel

now about the views, there are 4 view, 3 of them is Aforge video source control and another one is a picturebox, first one show normal video, second one show all detected object, 3rd box show only the biggest object and fourth one just draw a ‘0’ sign in biggest object location, see below image for more clearence

i also add a richtextbox which showing the pixel of biggestrectangle position .now about the code for connection i use normal aforge connection code,

for that u need to download aforge dll, and  also add a very good videocontrol for showing video u can add it in ur toolbox. just click freespace on toolbox chooseitems-> browse the dll -> add the control

using AForge;
using AForge.Video.DirectShow;

then initialize

  videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);

                if (videoDevices.Count == 0)
                    throw new ApplicationException();

               foreach (FilterInfo device in videoDevices)
                {
VideoCaptureDevice videoSource = new VideoCaptureDevice(device.MonikerString);
 videoSource.DesiredFrameSize = new Size(320, 240);
 videoSource.DesiredFrameRate = 15;
videoSourcePlayer1.VideoSource = videoSource;
 videoSourcePlayer1.Start();
               }

for videoplayercontrol add a event in its misc

named

 private void videoSourcePlayer1_NewFrame(object sender, ref Bitmap image)
        {//work with image
}

in that function i apply my color detection code by writing eucladian filtering and then use blobcounter to extract data to see how its work u can see my this post

the code is simple for second videoframe i used

  private void videoSourcePlayer1_NewFrame(object sender, ref Bitmap image)
        {
            Bitmap objectsImage = null;
            Bitmap mImage = null;
            mImage=(Bitmap)image.Clone();
           filter.CenterColor = Color.FromArgb(color.ToArgb());
            filter.Radius =(short)range;

            objectsImage = image;
            filter.ApplyInPlace(objectsImage);

            BitmapData objectsData = objectsImage.LockBits(new Rectangle(0, 0, image.Width, image.Height),
            ImageLockMode.ReadOnly, image.PixelFormat);
            UnmanagedImage grayImage = grayscaleFilter.Apply(new UnmanagedImage(objectsData));
            objectsImage.UnlockBits(objectsData);

            blobCounter.ProcessImage(grayImage);
            Rectangle[] rects = blobCounter.GetObjectRectangles();

            if (rects.Length > 0)
            {

                foreach (Rectangle objectRect in rects)
                {
                    Graphics g = Graphics.FromImage(mImage);
                    using (Pen pen = new Pen(Color.FromArgb(160, 255, 160), 5))
                    {
                        g.DrawRectangle(pen, objectRect);
                    }

                    g.Dispose();
                }
            }

            image = mImage;
        }

and for second videoframe i used belowcode

 private void videoSourcePlayer3_NewFrame(object sender, ref Bitmap image)
        {
            Bitmap objectsImage = null;

                  // set center colol and radius
                  filter.CenterColor = Color.FromArgb(color.ToArgb());
                  filter.Radius = (short)range;
                  // apply the filter
                  objectsImage = image;
                  filter.ApplyInPlace(image);

            // lock image for further processing
            BitmapData objectsData = objectsImage.LockBits(new Rectangle(0, 0, image.Width, image.Height),
                ImageLockMode.ReadOnly, image.PixelFormat);

            // grayscaling
            UnmanagedImage grayImage = grayscaleFilter.Apply(new UnmanagedImage(objectsData));

            // unlock image
            objectsImage.UnlockBits(objectsData);

            // locate blobs
            blobCounter.ProcessImage(grayImage);
            Rectangle[] rects = blobCounter.GetObjectRectangles();

            if (rects.Length > 0)
            {
                Rectangle objectRect = rects[0];

                // draw rectangle around derected object
                Graphics g = Graphics.FromImage(image);

                using (Pen pen = new Pen(Color.FromArgb(160, 255, 160), 5))
                {
                    g.DrawRectangle(pen, objectRect);
                }
              g.Dispose();
                int objectX = objectRect.X + objectRect.Width / 2 - image.Width / 2;
                int objectY = image.Height / 2 - (objectRect.Y + objectRect.Height / 2);
                ParameterizedThreadStart t = new ParameterizedThreadStart(p);
               Thread aa = new Thread(t);
               aa.Start(rects[0]);
            }
            Graphics g1 = Graphics.FromImage(image);
            Pen pen1 = new Pen(Color.FromArgb(160, 255, 160), 3);
            g1.DrawLine(pen1,image.Width/2,0,image.Width/2,image.Width);
            g1.DrawLine(pen1, image.Width , image.Height / 2, 0, image.Height / 2);
            g1.Dispose();
       }

the thread where i used to draw position and show that thread worked like

 void p(object r)
   {
       try
       {
       Bitmap b = new Bitmap(pictureBox1.Image);
       Rectangle a = (Rectangle)r;
       Pen pen1 = new Pen(Color.FromArgb(160, 255, 160), 3);
       Graphics g2 = Graphics.FromImage(b);
       pen1 = new Pen(color, 3);
       SolidBrush b5 = new SolidBrush(color);
        Font f = new Font(Font, FontStyle.Bold);
       g2.DrawString("o", f, b5, a.Location);
       g2.Dispose();
       pictureBox1.Image = (System.Drawing.Image)b;
       this.Invoke((MethodInvoker)delegate
           {
               richTextBox1.Text = a.Location.ToString() + "\n" + richTextBox1.Text + "\n"; ;
           });
       }
       catch (Exception faa)
       {
           Thread.CurrentThread.Abort();
       }
      Thread.CurrentThread.Abort();
   }

i have to use invoke to write in textbox because of cross threading.

u can change color size any time u want, the color of drawing point will also be change

thank u for reading this long article, The full code uploaded here.

after few moderation u can use it in ur code for many type of application

thank u all.

see update- at this post

Recognize face orientation using a new approach based on eye position and size by c#

Author: Md. Salahuddin Yousuf & Kishor datta gupta

We are trying to develop a method to recognize face orientation using OPENCV harr detection and neural network  . The process can be done without neural network but we use neural network for better performance and trying implement a new technique in pattern recognition.

Our technique is depend on 4 features of a face – They are

• Right Eye
• Left Eye
• Mouth
• Nose

We use respective distance and size of these feature’s for finding the orientation of the faces. We draw some pattern from these distance in a image file, and then send that image file to neural network.

The algorithm is

To see the patterns it can explain better how it works

Our pattern just draw some lines by using positions and length of feature items. if input image is like below

Then we get pattern like below

For coding we use C# languages. Its true c# is not good as matlab but we find it more applicable.
we also use EMGUCV which is a c# wrapper of OPENCV and For neural network we use BACKPROPAGTION Neural network which is entirely folowed the code of Murat Firat which u can get easily from his codeproject article.

Now starting the explanation how the code done. when a person show himself in a web camera or capture a image file send, it detect the human face from these image and extract only the faces . I explain the part of code before in here.

Now, we save detected face image’s in a folder and start to extract features. Process was simple , The code is

DirectoryInfo Folder;
Folder = new DirectoryInfo(filepath);
Images = Folder.GetFiles(".", SearchOption.TopDirectoryOnly);
string newPath = System.IO.Path.Combine(filepath, "profilelines");
System.IO.Directory.CreateDirectory(newPath);//creating folder for pattern
for (int i = 0; i < Images.Length; i++)
{
{
if (Images[i].FullName.Contains(".jpg") || Images[i].FullName.Contains(".jpeg"))
{//resizing images
Bitmap bmpResized = null;
Console.Write(Images[i].Name);
Bitmap bmp = new Bitmap(Images[i].FullName);
int resizedW = 320;
int resizedH = 320;
bmpResized = new Bitmap(bmp, resizedW, resizedH);
bmp.Dispose();
Images[i].Delete();
bmpResized.Save(filepath + "\\" + Images[i].Name, ImageFormat.Jpeg);
bmpResized.Dispose();
//create image files for pattern
Bitmap graph = new Bitmap(400, 400);
Graphics g = Graphics.FromImage(graph);
g.Clear(Color.White);
graph.Save(filepath + "\\profilelines\\" + Images[i].Name, ImageFormat.Jpeg);
}
}
}

Now code for extracting lefteye and draw pattern from left eye data
int inf = 0;
for (int i = 0; i < Images.Length; i++)
{
{
if (Images[i].FullName.Contains(".jpg") || Images[i].FullName.Contains(".jpeg"))
{
haar = new HaarCascade("haarcascade_mcs_lefteye.xml");
Bitmap bmpResized = null;
Bitmap bmp = new Bitmap(Images[i].FullName);
Image nextFrame = new Image(bmp);
Image grayframe = nextFrame.Convert();
var lefteye =
grayframe.DetectHaarCascade(haar, 1.4, 4,
HAAR_DETECTION_TYPE.FIND_BIGGEST_OBJECT,
new Size(nextFrame.Width / 8, nextFrame.Height / 8)
)[0];
if (lefteye.Length > 0)
{
{
if (lefteye[0].rect.Height * lefteye[0].rect.Width < 120 * 120 && lefteye[0].rect.Location.X < 130)
{
Bitmap face = new Bitmap(lefteye[0].rect.Height, lefteye[0].rect.Width);
Image ia = nextFrame.Clone().ToBitmap();
Graphics g = Graphics.FromImage(face);
g.DrawImage(ia, 0, 0, lefteye[0].rect, GraphicsUnit.Pixel);
//draw pattern
Bitmap b = new Bitmap(filepath + "\\profilelines\\" + Images[i].Name);
Graphics g2 = Graphics.FromImage(b);
Pen pen1 = new Pen(Color.Black, 3);
SolidBrush b5 = new SolidBrush(Color.Black);
int x = lefteye[0].rect.Location.X;
int y = lefteye[0].rect.Location.Y;
int h = lefteye[0].rect.Height;
int w = lefteye[0].rect.Width;
Point pp = new Point();
pp.X = Convert.ToInt32(x + w / 2);
pp.Y = Convert.ToInt32(y + h / 2);
Font f = new Font(Font, FontStyle.Bold);
Point p3 = new Point();
p3.X = Convert.ToInt32(x + w / 2);
p3.Y = Convert.ToInt32(y);
Point p4 = new Point();
p4.X = Convert.ToInt32(x + w / 2);
p4.Y = Convert.ToInt32(y + h);
Point p1 = new Point();
p1.X = Convert.ToInt32(x);
p1.Y = Convert.ToInt32(y + h / 2);
Point p2 = new Point();
p2.X = Convert.ToInt32(x + w);
p2.Y = Convert.ToInt32(y + h / 2);
Point[] pon = new Point[5];
pon[0] = pp; ;
pon[1] = p2;
pon[2] = p3;
pon[3] = p4;
pon[4] = p1;
g2.DrawPolygon(pen1, pon);
lefteyepositionx[inf] = len(pp.X, pp.Y, p2.X, p2.Y) + len(p2.X, p2.Y, p3.X, p3.Y) + len(p4.X, p4.Y, p3.X, p3.Y);
lefteyepositiony[inf] = lefteye[0].rect.Width;
b.Save(filepath + "\\profilelines\\a" + Images[i].Name, ImageFormat.Jpeg);
g2.Dispose();
b.Dispose();
File.Delete(filepath + "\\profilelines\\" + Images[i].Name);
File.Move((filepath + "\\profilelines\\a" + Images[i].Name), (filepath + "\\profilelines\\" + Images[i].Name));
}
}
}
bmp.Dispose();

write same kind of code for righteye, nose and mouth just changing the xml template. u will get all the template from emgucv site.

it is the code for draw the pattern, in next part i will write the code of neural network and explain how neural network trained by these pattern and recognizing it

C# tips : How to draw on a picturebox image using mouse by c#

if u want to draw something on a picture u can use microsoft paint, but if you want do it by ur c# code, i think this article will help you.

i make a win form application to draw by mouse. u can select the pensize as u wish. It is only a tricks between three mouse event on a picture they are

• mouse up
• mouse down
• mouse move

to use the software to u have to insert an image to picture box then chose color and size of brush. then start to draw by mouse press on image. like below image

after inserting u can draw as u like

the coding is fairly easy, u have to check mouse event is pressed or not, the write a rectangle in mouse position. like below



 bool draw = false;
   int s = 3;
  Color color = Color.Red;
 private void pictureBox1_MouseDown(object sender, MouseEventArgs e)
        {
            draw = true;
            Graphics g = Graphics.FromImage(mainimage);
            Pen pen1 = new Pen(color, 4);
            g.DrawRectangle(pen1, e.X, e.Y, 2, 2);
            g.Save();
            pictureBox1.Image = mainimage;
        }
        private void pictureBox1_MouseUp(object sender, MouseEventArgs e)
        {
            draw = false;
        }

        private void pictureBox1_MouseMove(object sender, MouseEventArgs e)
        {
            if (draw)
            {
                Graphics g = Graphics.FromImage(mainimage);
                SolidBrush brush = new SolidBrush(color);
                g.FillRectangle(brush, e.X, e.Y, s, s);
                g.Save();
                pictureBox1.Image = mainimage;
            }
        }

u can get full source code from here

How to allow unsafe code in visual studio 2010 c#

For many reason we have to use unsafe code like in pointer in dotnet framework , Visual studio cant build the project unless the allow unsafe code option enabled. To enable this option select ur project name solution explorer click right mousebutton go to the properties like below Now Click the build tab and check the allow unsafe code option. like below Now u can compile unsafe code as much as u wish. thank u

Convert a image file in byte pointer data and apply algorithom C#

If u want to use pointer in c# u have to use “unsafe” keyword. and then we can use byte pointer. In BitmapData we Can get data as a byte in different format. In my first

Bitmap pbmp=new Bitmap("fillocation");
int r = 0;
int g = 0;
int b = 0;
int a = 0;
BitmapData data = pbmp.LockBits(new Rectangle(0, 0, pbmp.Width, pbmp.Height), ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);

byte* imgPtr = (byte*)(data.Scan0);

pbmp.UnlockBits(data);
for (int x = 0; x < pbmp.Width; x++)
{
for (int y = 0; y < pbmp.Height; y++)
{
a = (int)*imgPtr;
imgPtr++;
r = (int)*imgPtr;
imgPtr++;
g = (int)*imgPtr;
imgPtr++;
b = (int)*imgPtr;
imgPtr++;

}
imgPtr += data.Stride - pbmp.Height * 4;
}

example u can see i use

PixelFormat.Format32bppArgb

as bitmap data. So every pixel data come in 4 byte first byte contain alpha value, then sequencely red , green and blue value.

byte* imgPtr = (byte*)(data.Scan0);

here the first byte of image pointed by the pointer imgptr.
then we start two loop in innar loop every byte continously represent alpha, red , green, blue, alpha red, green , blue……………………….. values of pixel. To jump out second line first pixel we used

imgPtr += data.Stride - pbmp.Height* 4;

here it was multiply by 4 because of our pixelformat was “PixelFormat.Format32bppArgb”.
now if u want to take 4 bytes in a value name color u can use this code

BitmapData data = pbmp.LockBits(new Rectangle(0, 0, pbmp.Width, pbmp.Height), ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);

byte* imgPtr = (byte*)(data.Scan0);

pbmp.UnlockBits(data);
for (int x = 0; x < pbmp.Width; x++)
{
for (int y = 0; y < pbmp.Height; y++)
{
Color c = Color.FromArgb((int)*imgPtr);
imgPtr=imgPtr+4;

}
imgPtr += data.Stride - pbmp.Height * 4;
}

Now if u use another pixelformat like “PixelFormat.Format24bppRgb” where only Red, green and blue present u code will change as

Bitmap pbmp=new Bitmap("fillocation");
int r = 0;
int g = 0;
int b = 0;

BitmapData data = pbmp.LockBits(new Rectangle(0, 0, pbmp.Width, pbmp.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);

byte* imgPtr = (byte*)(data.Scan0);

pbmp.UnlockBits(data);
for (int x = 0; x < pbmp.Width; x++)
{
for (int y = 0; y < pbmp.Height; y++)
{

r = (int)*imgPtr;
imgPtr++;
g = (int)*imgPtr;
imgPtr++;
b = (int)*imgPtr;
imgPtr++;

}
imgPtr += data.Stride - pbmp.Height * 3;
}

So for different type of pixel format u have to try differnt calculation for byte pointer.
and if are not familiar with how to use byte pointer just write ur function like that,
private unsafe Color pointerimagefunc(Bitmap pbmp){.........}
and make sure in ur build properties allow unsafe code is checked

c# tips: execute windows shutdown ,restart ,logof command in c#

i make c# function for these command

private void execute_command(String Comande)
{
if (Comande.CompareTo("shutdown") == 1)
{
System.Diagnostics.Process.Start("shutdown", "-s");
}
else if (Comande.CompareTo("restart") == 1)
{
System.Diagnostics.Process.Start("shutdown", "-r");
}
else if (Comande.CompareTo("logoff") == 1)
{
System.Diagnostics.Process.Start("shutdown", "-l");
}
else if (Comande.CompareTo("abort") == 1)
{
System.Diagnostics.Process.Start("shutdown", "-a");
}
}

Controlling PC And Remote Access via lan in c#

Author
Mahamud Hasan
Wasif Islam
kishor datta gupta

we  develop a software which perform basic PC controlling operations and  complete remote access using C#. With our software we can establish a connection to any computer via the LAN in just a few seconds and remotely control this computer just as if you were sitting in front of it.

our software can able to do following tasks

• We can connect server pc and client pc.
• Maximum 5 clients can connect to the server.
• Server and client can transfer data with each other.
• Only server can tranfer file to the client.
• Server pc can shutdown, restart and logoff to the client pc.
• Using remote access server pc can see the desktop of the client.
• Server pc can browse client pc and install any kind of software.
• Server pc can access to the computer to be serviced and it can perform any action even if no-one is sitting in front of it.

Features:Simple implementation of a TCP client server relationship

When the server program is run, it will indicate at which IP it is running and the port it is listening to. Using Tcp listener server can listen its ip address and port. “Dns.GetHostByName(Dns.GetHostName()).AddressList[0];” –Using this code we get the server ip address. At the same time a thread will be running. This thread will waiting for client connection.

Now run the client program and write down the server ip address. Then press the “Connect” button and it establishes a connection with the server.

When a connection is established the server will display the client IP address from where it has accepted the connection and client will ask for the string which is to be transmitted to the server. The server on reciept of the string will display it, send an acknowledgement which will be recieved by the client.

The client can be either run from the same machine as the server or from a different machine. If run from a different machine then a network connection should exist between the machines running the server and client programs.

When any client connect with the server then from the server display a combobox named “Client IP List” contains all client IP.

When connection is established between server and client then server and client can transfer message with each other.

When server send message then client recieve this message. At the same time client can send message to the server from the Message textbox.Then server also recieve data from the client. Server can transfer file to the client. Using Networkstream server can transfer file. When we press File Transfer button then a file dialoug box is open. Then select a file and send it.

There are controll operations that the server will perform to the client. If server want to shut down the client pc then server can select the shut down button and activate it. Then client pc will shut down. Server can do three operations if it want. These are Shutdown , restart and log off.

We use Remote Desktop a great deal. Using remote access server can browse the client PC. All kind of operation can be performed by the server when remote is active,

For that we get a help from a codeproject article . Thanks to it we make a very effective remote access,

We will be using AxMSTSCLib an ActiveX component in our program to connect to the remote computer. It’s not that hard to build a remote desktop application in .NET. Microsoft has a “Microsoft RDP client control” ActiveX control that we will be using in our application.We will start by creating a Windows application in the Visual Studio IDE. Add a reference to “Microsoft Terminal Services Control Type Library” from the COM tab. This will add MSTSCLib.dll to the project.

To add MSTSC to the toolbox, right click the toolbox and select “Choose Items…”. Now add “Microsoft Terminal Services control from the COM tab.

Drag the newly added control from toolbox to the form.

Here is how we write the Connect button click event.

rdp.Server = txtServer.Text;
rdp.UserName = txtUserName.Text;
IMsTscNonScriptable secured = (IMsTscNonScriptable)rdp.GetOcx();
secured.ClearTextPassword = txtPassword.Text;
rdp.Connect();

Now assign the properties (Server, UserName) of RDP control with the textbox values. Here’s how easy it is to login to remote machine. However there is one catch, there is no direct method in RDP control through which you can pass the username and password to login to the remote desktop.  Due to security reasons, you have to implement an interface (IMsTscNonScriptable) to cast it separately.
IMsTscNonScriptable secured = IMsTscNonScriptable)rdp.GetOcx();
secured.ClearTextPassword = txtPassword.Text;

To disconnect from the remote desktop session, we just need to call the Disconnect() method.

Before disconnecting, we want to ensure that the connection is still available. We don’t want to disconnect if it is already disconnected.

if (rdp.Connected.ToString() == "1")
 rdp.Disconnect();

Though we have developed our software using the platform independent programming language C# but our software is not totally platform independent at all. Because our software only run in the .net framework version 3.5. But we are tried our best to do that. There are a few limitations in our software cause it can remotely connect to only one client pc at a time.

Full source code of this software is here

Create a desktop video recorder in C#

We have to record our desktop activity for make various reason, like make a demo for an application, or making video tutorial for an application etc.

the prototype application is like

This project can be done in 3 steps

• taking screenshot of ur desktop continously
• Make an video file using the images

So first step taking screenshot of ur desktop continously

so first we have to take the screen shot of the desktop, for that we have to write the code
int ScreenWidth = Screen.PrimaryScreen.Bounds.Width;
int ScreenHeight = Screen.PrimaryScreen.Bounds.Height;
Graphics g;
Bitmap b = new Bitmap(ScreenWidth, ScreenHeight);
g = Graphics.FromImage(b);
g.CopyFromScreen(Point.Empty, Point.Empty, Screen.PrimaryScreen.Bounds.Size);

Now we have to make avi file from these images, for that we aviwrapper u can download it from here.

now about making avi file from these images , the code i used is like
AviManager aviManager = new AviManager(@"..\..\Output\output.avi", false);
VideoStream aviStream = null;
aviStream = aviManager.AddVideoStream(true,4, bi);
aviStream.AddFrame(b);


the link for full source code is here