Click here to Skip to main content
15,884,537 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
I want to add some graphs/charts within a video from webcam, in a WinForm(C# .Net) application. What is the best way to add some images dynamically to a video in WinForms/WPF application? Please check the following image to get a better idea about what I'm trying to achieve. I want to achieve this on live streaming video from webcam, like Google hangout effects.

Basically what I want to do is blending two videos, or blending image and video.

http://www.nzembassy.com/sites/default/files/bigstock-businessmen-hand-touch-screen-55533974_0.jpg[^]

If the question feels too broad, correct technical terms to do some research will be a great help, as I' a newbie in video processing. I have already found terms like Digital compositing, alpha blending, watermarking etc. Also some frameworks/APIs like AForge.NET and Splicer. I found some useful reference here: From glyph recognition to augmented reality[^], and I'm referring it now. Is there any latest technology which came after this? (This article is bit old)

I have done days of research on this. I have found many libraries like OpenCV, EMGU, ImageMagic and many more, but I couldn't find something which I'm looking for. Or else I'm finding difficulties in getting a starting point. What I want to add is not simple images, I want to add working graphs and charts with animation and 3D rotations. Any guidance will be appreciated. Thank you.
Posted
Updated 18-Jan-15 20:59pm
v4
Comments
Sergey Alexandrovich Kryukov 19-Jan-15 2:47am    
Do you want to modify the video file or perhaps just overlay some graphic on top of it? If you want to modify video, do you want to do it on the fly or not? If you want just overlay, would you opt for WPF (I would not waste time for Forms in this case, too much bothering for questionable results)?
—SA
Paulson Geo 19-Jan-15 2:54am    
Thank you for your speedy reply.

I just wanted to overlay some graphics on top of it. It is basically for a UI within a webcam video or augmented reality kind of thing, to enable users to interact with graphs by detecting their hands movement.
Sergey Alexandrovich Kryukov 19-Jan-15 3:31am    
Think about switching to WPF. Then you won't have any problems with that at all. With Forms, I would not bother. GDI is not designed to work well with DirectX. Possible, but why? It would contaminate the project, compromise system compatibility, maintenance, everything...
—SA
Paulson Geo 19-Jan-15 3:50am    
I'm still in the design phase of the project, so I'll go for WPF then. Could you please provide some reference if you are familiar with adding overlays to video in WPF apps? Please have a look at the following video to get a clear idea about what I'm trying to achieve: http://youtu.be/gEt8e701w-c

I want to have 2D/3D charts on the screen like the circles in the video.
Sergey Alexandrovich Kryukov 19-Jan-15 12:14pm    
Basically, in WPF any UI element can be semi-transparent having any other see-through UI element behind. With Forms, video is a "foreign" element; it would not allow to show anything in front of it. Let me think at it a bit...
—SA

1 solution

Paulson George wrote:
I'm still in the design phase of the project, so I'll go for WPF then. Could you please provide some reference if you are familiar with adding overlays to video in WPF apps?
Sure. First, some introduction:

In the past, I saw my colleague making some primitive frame to show some metadata in from of video in System.Windows.Forms, to show some metadata, name coordinates, etc. If you do it with usual graphics, the video underneath "wipes" graphics. This person dug into DirectX, managed to utilize "DirectX Overlay Manager", rendered that stupid frame with text in that manager, succeeded. Forms use GDI+ which itself does no work well with DirectX. DirectX and GDI try to ignore each other.

WPF is itself based on DirectX, so, when it became available, years ago, I tried to experiment with that first, expecting that I can just put some UI elements in front of video, without any efforts, and see both video and those element. Naturally, this is what happened.

I needed some time to think how to show it on a simplest possible example. So, look at this WPF window:
HTML
<Window x:Class="OverVideo.Ui.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="MainWindow" Height="300" Width="300">
    <Grid>
        <MediaElement Source="test.avi"/>
        <Canvas Opacity="0.6">
            <Rectangle Canvas.Left="20" Canvas.Top="15"
                Width="80" Height="140" Fill="Red" Stroke="Black"/>
        </Canvas>
    </Grid>
</Window>

Just for demonstration, I put two UI elements one on top of another, in one of the simplest possible way: I put an instance of Grid on the window's logical tree top, with two children: MediaElement and Grid. This is just one possible ways to superimpose two or more elements: by default, these two children of Grid instance has the same values of dependency property Canvas.Row and Canvas.Column and, hence, take the same position. Another way would be putting some elements on the instance of Canvas.

So, why Canvas? Because this is element allows free positioning of elements on it. In other words, this is something which is usually used for vector graphics. This is not typical for WPF, where all elements are automatically aligned according its position in logical tree and all those margin and padding settings. For demonstration purpose, I put just one element on the canvas: a semi-transparent Rectangle (see Opacity attribute). Consider this rectangle as a prototype for your chart element. :-)

Note that the Opacity of the Canvas instance itself is default (1, full opacity), but you can still see through it, where there is not children. Should you set Opacity for Canvas, it would be applied to all its elements, so you would have some combined opacity.

And yes, I just copied some video file "test.avi" to the output directory, just for simplicity. So, when I start the application, video starts to play immediately, and you can see the portion of video behind the semi-transparent rectangle. Viola!

Now, as all the children of the Canvas instance are now on top of the logical tree, you can add any mouse and keyboard event handlers immediately to them, so you can move them around, transform, and so on. I think this is all you need to achieve your goals.

You can use different approaches for putting UI elements on top of video. You can use Grid as I just demonstrated, you can put video on the Canvas, too, with some other elements on top, and you can consider using such thing as adorners: http://msdn.microsoft.com/en-us/library/ms743737%28v=vs.110%29.aspx[^].

Please see:
http://msdn.microsoft.com/en-us/library/system.windows.controls.mediaelement%28VS.95%29.aspx[^],
http://msdn.microsoft.com/en-us/library/system.windows.controls.canvas%28v=vs.110%29.aspx[^].

My little problem was to give you the idea with video and at least some introduction to WPF, but this was not so easy, in part, because this is not the most typical use of WPF. So don't take my introduction serious, learn it yourself.

See also:
http://msdn.microsoft.com/en-us/library/ms754130%28v=vs.110%29.aspx[^],
http://msdn.microsoft.com/en-us/library/ms750441%28v=vs.110%29.aspx[^],
http://msdn.microsoft.com/en-us/library/ms752914%28v=vs.110%29.aspx[^],
http://msdn.microsoft.com/en-us/library/ms753391%28v=vs.110%29.aspx[^],
http://msdn.microsoft.com/en-us/library/ms745058%28v=vs.110%29.aspx[^],
http://wpftutorial.net/LayoutProperties.html[^].

For charting, you can use, for example, Microsoft Charts controls for your version of .NET.
Some links on charting:
http://www.c-sharpcorner.com/uploadfile/mahesh/charting-in-wpf[^],
https://code.msdn.microsoft.com/windowsapps/Chart-Control-in-WPF-c9727c28[^],
http://wpf.codeplex.com/releases/view/40535[^].

This CodeProject article can also be useful: WPF Toolkit Charting Controls (Line, Bar, Area, Pie, Column Series) Demo[^].

—SA
 
Share this answer
 
v9
Comments
Paulson Geo 20-Jan-15 1:57am    
Dear SA Kryukov,

Thank you very much for your support and effort. I really appreciate the way in which you have given me a detailed solution. I'll follow it and let you know the progress.

By the way, I have one more question. Would you suggest some kind of screen capture or some other solution if I need to record the whole video?

Thanks again,
Paulson George
Sergey Alexandrovich Kryukov 20-Jan-15 2:11am    
You are very welcome.
Will accept the answer formally?

<stroke>I answered on screen capture, Solution 2.

I did not try but would be able to solve it; basically, this is quite solvable, because both Forms and WFP provide functions to copy desktop to an image, and then you can assembly images in video, for example, FFMpeg can be used; there are even .NET wrappers for FFMpeg... Or you can find some ready-to-use solution.

You could ask a separate question on this forum...

—SA

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900