Kinect Bing Map AJAX Control 6.3 with WPF

• Download source - 234.8 KB

Introduction 

This article describes how we can use Microsoft Kinect SDK along with WPF Webbrowser control to make a desktop application which shows Bing Map 3D using Microsoft Virtual Earth AJAX Control 6.3.

Kinect is a motion sensing input device by Microsoft for the Xbox 360 video game console and Windows. Based around a peripheral for the Xbox 360 console, it enables users to control and interact with the Xbox 360 without the need to touch a game controller, but through a natural user interface using gestures and spoken commands. The project is aimed at broadening the Xbox 360's audience beyond its typical gamer base. A version for Windows was released on February 1, 2012.  

This project uses Kinect to detect some postures and on the basis of that it zooms in/out the 3D map.

I decided to write this article after successfully integrating the Kinect with Bing Map 3D or Bing Map with Birdeye View. I had searched the Internet for such a solution and it seemed that one of the solutions was to use Bing Map Silverlight Control. Then to integrate Kinect with Silverlight needs some tricks as well, like using the TCP COM for communication between Silverlight and WPF or importing the Microsoft.Research.Kinect.dll file using Pinvoke etc. But I did not use one of these methods. 

Initially I also tried the Bing Map WPF Control v1 which was successfully integrated with Kinect. But since it is recently released in Jan 2012 it does not supports Streetside or Birdeye View. I asked this thing on the Microsoft MSDN website here: 

Microsft MSDN

Background  

Kinect SDK can be easily integrated with WPF. Hence, basic application runs on wpf where the Microsft Kinect SDK is integrated. A WPF web browser control is added here which invokes javascript, whenever required, on a webpage. The webpage uses the AJAX control of Bing Map (formerly Virtual Earth) 6.3 to display the map and perform specific tasks whenever Kinect events are fired from wpf.

Using the code  

Pre-requisites

1. Kinect for Windows SDK Beta 2. This can be downloaded from here: Download Kinect for Windows SDK Beta 2
2. To run the sample code you need to install Bing Map 3D. It can be downloaded from here: Download Bing Map 3D

Although, Microsoft has announced that it will discontinue services of 3D maps. The ajax control can be used for several other features such as the 2D Streetside and Birdeye view. It has pretty good documentation here: http://msdn.microsoft.com/en-us/library/bb429619.aspx.

3. In the sample coding4fun v1.1.0.0 has been used which can be downloaded from: Coding For Fun library

Solution Architecture

The source code has two projects within the solution. One has the WPF application while the other has the ASP .NET web application. The main files to consider in the solution are: 

1) MainWindow.xaml

2) MainWindow.xaml.cs

3) Default.aspx

Source Code Description:

Xaml:  

<WebBrowser x:Name="myWebBrowser" Width="1197" Height="900" 
   Canvas.Top="-183" Canvas.Left="-7" Margin="0,166,0,0"></WebBrowser>

This adds the web browser control to the window.

MainWindow.xaml.cs:

First we navigate our WPF browser to a page, with the following code:

Uri uri = new Uri("http://localhost:59453/Default.aspx", UriKind.RelativeOrAbsolute);
this.myWebBrowser.Navigate(uri);

The first line sets the Uri which tells the wpf browser where to navigate.

DO NOT FORGET TO CHANGE THE LOCALHOST PORT. 59453 is on my PC yours will be different.

To identify the port you can run the aspx file independently by right-clicking on the default.aspx in the WebApplication1 project and select View in Browser. Then see the URL for port or copy paste the whole URL.

The second line actually directs the browser to the page.

Now lets take a look at the Kinect posture detection code. This is in fact the bridge between WPF and Ajax Map Control.

private void detectPosture()
{
    var scaledJointRight = skeleton.Joints[JointID.HandRight].ScaleTo(640, 480, .5f, .5f);
    var scaledJointLeft = skeleton.Joints[JointID.HandLeft].ScaleTo(640, 480, .5f, .5f);
    var scaledJointShoulder = skeleton.Joints[JointID.ShoulderRight].ScaleTo(640, 480, .5f, .5f);

    if (scaledJointRight.Position.Y < scaledJointShoulder.Position.Y && 
        scaledJointLeft.Position.Y > scaledJointShoulder.Position.Y && rightEnabled == true)
    {
        myWebBrowser.InvokeScript("zoomIn");
        rightEnabled = false;
    }
    else if (scaledJointLeft.Position.Y < scaledJointShoulder.Position.Y && 
         scaledJointRight.Position.Y > scaledJointShoulder.Position.Y && leftEnabled == true)
    {
        myWebBrowser.InvokeScript("zoomOut");
        leftEnabled = false;
    }
    else if (scaledJointLeft.Position.Y > scaledJointShoulder.Position.Y && 
         scaledJointRight.Position.Y > scaledJointShoulder.Position.Y)
    {
        rightEnabled = true;
        leftEnabled = true;
    }
}

As the name of the function suggests it detects the postures. When right hand is above the shoulder and left hand is below the shoulder it invokes the function zoomIn from the javascript. Same happens for left hand but this time zoomOut is called instead. And when both hands are below shoulder the boolean values are set to true so that it doesnt zoomIn or zoomOut again and again. Hence,  one needs to raise appropriate hand to zoomIn or zoomOut and then bring it down to do it again. 

I have made these postures just for showing it here. I have created many postures and gestures like that so one can create their own gestures.

This is my first Article for codeproject. I am sorry if I have missed anything or not explained anything in detail.  

Feel free to ask any questions.