How to process video frames in different thread

Question

2.50/5 (2 votes)

See more:

Hello Everyone,

Background
I am working on an application that requires some image-processing on video stream and display the original video and processed video side-by-side.

Situtation
Below is the event handler when a new frame is received from camera.
pictureBox1is where the original video is shown.
GetInputImage() function will steal a from pictureBox1,so that some image processing can be performed on that frame.

C#

private void camera_NewFrame(object sender, ref Bitmap image)
{
    if (!isReadingPictureBox)
    {
       if (pictureBox1.Image != null)
           pictureBox1.Image.Dispose();
       pictureBox1.Image = (Bitmap)image.Clone();
    }
}

private void GetInputImage()
{
    if (inputImage != null)
        inputImage.Dispose();

    isReadingPictureBox = true;
    if (pictureBox1.Image != null)
        inputImage = new Bitmap(pictureBox1.Image);
    isReadingPictureBox = false;
}

The image processing is heavy and takes time to process a single image. Thus the frame rate of output video is expected to be very less that the original.

Application must show the original video without getting affected by the image processing. So I want to perform the image processing in a different thread.

C#

private void ProcessThread(some args)
{
    GetInputImage();
    if (inputImage != null) {
        // Do Image Processing on "inputImage"
        // Show result in pictureBox2
    }
}

Questions

[1] Is the method of grabbing a frame (above), OK ? Or the one below is better ?

C#

private void camera_NewFrame(object sender, ref Bitmap image)
{
    pictureBox1.Image = image; // picturBox1's image will not be read for processing

    if(!isReadingInputImage) {
        if (inputImage != null)
            inputImage.Dispose();
        inputImage = (Bitmap)image.Clone(); // GetInputImage() is not required now.
    }
}

[2] How to make the ProcessThread(), run indefinitely ? Is this(below) approach OK ?

C#

private void ProcessThread(some args)
{
    do {
        GetInputImage();
        if (inputImage != null) {
            // Do Image Processing on inputImage;
            // Show result in pictureBox2
        }
    }while(someCondition);
}

Or should I trigger a processing event for each frame within camera_NewFrame() func ?

Posted 3-Apr-15 18:14pm

Abhishrek

Updated 3-Apr-15 18:22pm

v2

Add a Solution

Comments

Sergey Alexandrovich Kryukov 4-Apr-15 0:47am

PictureBox — not serious. Normally, video is implemented using DirectX...
Why doing all that?
—SA

2 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Sergey Alexandrovich Kryukov · Accepted Answer · 2015-04-03T19:02:00

Solution 1

Please see my comment to the question. In principle, what you do is possible, but quite questionable in terms of performance and timing. I want just address the main obstacle: it cannot work like that. In a non-UI thread, you need to use Control.Invoke.

You cannot call anything related to UI from non-UI thread. Instead, you need to use the method Invoke or BeginInvoke of System.Windows.Threading.Dispatcher (for both Forms or WPF) or System.Windows.Forms.Control (Forms only).

You will find detailed explanation of how it works and code samples in my past answers:
Control.Invoke() vs. Control.BeginInvoke()[^],
Problem with Treeview Scanner And MD5[^].

See also more references on threading:
How to get a keydown event to operate on a different thread in vb.net[^],
Control events not firing after enable disable + multithreading[^].

—SA

Posted 3-Apr-15 19:02pm

Sergey Alexandrovich Kryukov

Comments

Abhishrek 7-Apr-15 9:41am

Thank a lot @Sergey for a prompt reply.
The Solution you proposed might work, but I am a rookie and don't know much about Contol.Invoke etc. I tried to understand what it does but felt that this is not what i was looking for. I came up with a solution with background worker and it worked fine.
(But thanks anyways, to make me realize that I need to study more)

Sergey Alexandrovich Kryukov 7-Apr-15 9:45am

If you don't know about it, get to knowing it by reading my other answers I referenced. If you need clarification, please ask some follow-up questions.
—SA

Abhishrek 7-Apr-15 10:09am

@Sergey, Ofcourse I read your other answers but only text based explanation (without a sample usage code) is difficult to understand for dummies (like me). Had you given some sample usage code for the Control.Invoke, I would've understood it by now.
Update: I just noticed that you provided link to MS knowledge database, it has sample codes. I will study it now. Thanks :)

Sergey Alexandrovich Kryukov 7-Apr-15 10:13am

First, you are supposed to learn how to understand things by description. I do have some source code for an article, but the article itself is not written, it would be a big one, answering many questions at once. If you want, I'll notify you, but it will take time. I have more unfinished articles in my queue than published ones.
—SA

Abhishrek 7-Apr-15 10:25am

I noticed that you provided link to MS knowledge database in your answers. It has sample codes. I looked at it and understood the concept right away. So, Yes, the method you proposed should work. Thanks :)
However, I now realize that "background worker" is doing the same thing (calling a function asynchronously) in an easy to use syntax. So the solution I proposed seems OK as well.

Sergey Alexandrovich Kryukov 7-Apr-15 10:31am

That's right. If you read it and simply ask what is not clear, I'll gladly answer. Please understand: you will need the invocation mechanism anyway, in many cases, with BackgroundWorker, too...
—SA

Abhishrek 7-Apr-15 10:11am

By the way, what do you think about the solution i proposed. Is it not the proper way ? (its working fine though)

Sergey Alexandrovich Kryukov 7-Apr-15 10:20am

"Working" doesn't mean good. There are problems. It's redundant, and one obvious problem is: it's not a good idea to create object in one stack frame and dispose in another one, even a different method. Good idea is: creator disposes.

I have a bigger questions though: may I know what's is that in those frames? If this is some graphics rendered by your code (lines, shapes, etc.), it would be all badly wrong. So, what is that?

—SA

Abhishrek 7-Apr-15 10:32am

The solution I proposed, is only READING the Bitmap (inputImage) in another thread (background worker thread). "inputImage" is created and destroyed by the main thread (within camera_NewFrame() function) only.

The frames are raw Bitmap images provided by camera (read using DirectX by AForge .Net Image processing Library)

Sergey Alexandrovich Kryukov 7-Apr-15 10:36am

Okay then. But it raises another problem: why not using available MediaPlayer? It is very efficient, uses DirectX, unlike your code? But if you need to do some frame processing on the fly, it can be a problem...
—SA

Abhishrek 7-Apr-15 10:51am

I am using many other features provided by "AForge .Net library" and its convenient to use its inbuilt video capture with the other features, so I am not using MediaPlayer (or DirectX directly).

Sergey Alexandrovich Kryukov 7-Apr-15 10:53am

Understood. That was my guess.
—SA

Abhishrek · Accepted Answer · 2015-04-07T03:35:00

I solved the problem myself using background worker. Posting here for reference.

C#

private void camera_NewFrame(object sender, ref Bitmap image)
{
    pictureBox1.Image = image;
    if (backgroundWorker1.IsBusy != true)
    {
        lock (locker)
        {
            if (inputImage != null)
                inputImage.Dispose();
            inputImage = (Bitmap)image.Clone();
        }
        backgroundWorker1.RunWorkerAsync();
    }
}

private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
    lock (locker)
    {
         baseImage = (Bitmap)inputImage.Clone();
    }
    // Do Image processing here on "baseImage"
    // show result in pictureBox2
}