Connecting Intel® RealSense™ 3D Camera With the Intel® Edison - JavaScript

Get access to the new Intel® IoT Developer Kit, a complete hardware and software solution that allows developers to create exciting new solutions with the Intel® Galileo and Intel® Edison boards. Visit the Intel® Developer Zone for IoT.

Introduction

Intel® has introduced many great innovation. One is Intel® RealSense™ technology with the Intel® RealSense™ 3D Cameras and SDK; we can create applications that can perform gesture detection, facial analysis, background segmentation, voice recognition, voice synthesis, and more, the SDK provides more support to software framework that is CSharp, Java, JavaScript, Processing and Unity. The developer can jump off to get started creating applications with ease based on framework they are familiar with.

Another cool thing from Intel® technology is the Intel® Edison platform. An IoT developer can build prototypes that gather sensor information or send control to other devices and create innovative products. Intel® Edison already has integrated Wi-Fi and Bluetooth Low-energy* (LE) so we can connect to the internet and create a real Internet of Things solutions.

A great article using Intel® RealSense™ Technology in combination with the Intel® Edison development platform was written by Peter Ma. In this article there are two examples of such applications. One uses the Intel® RealSense™ 3D Camera as input and the Intel® Edison board as output. The SDK triggers an LED light on the board. In the second example, which uses the Intel® Edison board as input and the Intel® RealSense™ 3D Camera as output, voice synthesis is used to speak the sensor data from the board.

That project inspired me to create similar thing based on JavaScript. In this article we will be using Intel® RealSense™ technology JavaScript Framework to send hand gesture data to a server and the server will send the data to trigger the buzzer and LED connected by Intel® Edison.

Hardware

• Intel® Edison board with the Arduino* breakout board
• Seeed Grove* – Starter Kit Plus - Intel® IoT Edition
• 4th generation (or later) Intel® Core™ processor
• 8GB free hard disk space
• USB 3.0
• An Intel® RealSense™ 3D Camera F200 (system-integrated or peripheral version)
• An Server equiped with Node.js

Software

• Intel® XDK IoT Edition : https://software.intel.com/en-us/getting-started-with-the-intel-xdk-iot-edition#launch
• Intel® RealSense™ SDK R4 : https://software.intel.com/en-us/intel-realsense-sdk

Step 1 : Setup the Intel® RealSense™

Setup for the Intel® RealSense™ is very simple. First connect the Intel® RealSense™ 3D F200 Camera to a USB 3.0 Port. Install Camera driver and Intel® RealSense™ SDK : https://software.intel.com/en-us/intel-realsense-sdk/download

After SDK is installed, navigate to the installation location and open a JavaScript sample at RSSDK\framework\JavaScript\FF_HandsViewer\FF_HandsViewer.html using your browser and it will do some checks.

Hit the link on your browser to install Intel® RealSense™ SDK Webapp Runtime then restart your browser and try opening the file again. Then try raising your hand in front of the camera and if everything goes well, you will see Hand Data visualized on the browser.

Take notice on bottom text Gesture:{"timeStamp": ,"handId": ,"state": ,"frameNumber": ,"name":"spreadfinger"} , we will send "name":"spreadfingers" to Server.

• Copying RSSDK\framework\JavaScript\FF_HandsViewer Folder to another location
• Copying realsense.js located at RSSDK\framework\common\JavaScript and place it on FF_HandsViewer Folder
• Creating edisonconnect.js files to receive Gesture Data and Send to Server, *note : please change your referring server IP’s addresses

• Edit sample.js file to intercept Gesture Data and pass it to edisonconnect.js

• Edit FF_HandsViewer.html file to re-locate realsense.js file path, linking socket.io and edisonconnect.js script

Until this step we are ready to send Gesture Data to Server. The next step is to setup server and Intel® Edison

Step 2 : Setup the Server

I assume we have already setup and Debian Linux based Server (i'm using Ubuntu Distro's) and we gone setup node.js server on that machine.

• Remote your server and login trough SSH
• Updating server

• Installing Nodejs

• Installing npm

• Installing socket.io express

• Creating file server.js and index.html

• Editing server.js file using vim

• Write this script bellow

• Last, running the scripts if you see "listening at port 1337" then you are safe

Step 3 : Setup the Intel® Edison

Intel® Edison is easy to deploy and clear documentation is provided by Intel®. You can find it at:

• Getting started with Intel® Edison : https://software.intel.com/en-us/iot/library/edison-getting-started
• Seeed Grove* – Starter Kit Plus - Intel® IoT Edition : http://www.seeedstudio.com/depot/Grove-starter-kit-plus-Intel-IoT-Edition-for-Intel-Galileo-Gen-2-and-Edison-p-1978.html
• Assemble Intel® Edison on the Arduino board : https://software.intel.com/en-us/articles/assemble-intel-edison-on-the-arduino-board
• Running the Integrated Installer : https://software.intel.com/en-us/get-started-edison-windows-step2
• Setup a Serial Terminal Connectoin : https://software.intel.com/en-us/get-started-edison-windows-step3
• Connecting Intel® Edison to Wi-Fi : https://software.intel.com/en-us/get-started-edison-windows-step4
• Install and configure Intel® XDK IoT Edition : https://software.intel.com/en-us/getting-started-with-the-intel-xdk-iot-edition#launch

Ok, now it's time to place a code inside Intel® Edison, this code is to connecting to Server and listening to any broadcast from Server. If there is Gesture Data Received then Intel® Edison will trigger Digital Pins to On/Off.

• Let's start by opening Intel® XDK IoT Edition and creating a new project from Templates. We will use Digital Write Template

• Edit package.json adding dependencies socket.io-client

• Edit main.js to connect to Server and check if Gesture Data name "spreadfingers" then trigger Digital Pins2 and Digital Pins8 state to 1 / On if other Gesture Data Received change back to 0 / Off state. * please change your referring server IP’s addresses, if you want to change the Pins make change on mraa.Gpio(selectedpins)

Finally, Install/Build your project then Run, *make sure Intel® Edison device has connected

Now let's try to make sure the server is Up and Running and Intel® Realsense™ Laptop/Computer and Intel® Edison are connected to the Internet.

You can see action here.

Conclusion

This is a very simple project, modifying Intel® RealSense™ JavaScript Framework Sample Script to send captured Gesture Data to Node.js Server. Server will broadcast Gesture Data to any socket client. Intel® Edison installed with socket.io-client listening broadcast from server and if Gesture Data name was "spreadfingers" it will trigger Digital Pins to change state. The same way is possible creating project using Face Data, Blob Data and Voice Recognition based on JavaScript Framework.

So What Will You Make?

Aulia Faqih - Intel® Software Innovator

Intel® RealSense Innovator based in Yogyakarta, Indonesia, currently lecturing at UIN Sunan Kalijaga Yogyakarta. Love playing with Galileo / Edison, Web and all geek things.

Intel® Developer Zone for IoT

Start inventing today with the Intel® IoT Developer Program which offers knowledge, tools, kits and a community of experts to quickly and easily turn your innovative ideas into IoT Solutions.

Dream it, Build it with the Intel® IoT Developer Kit for Intel® Edison and Intel® Galileo platforms. These kits are versatile, performance-optimized and fully integrated end-to-end IoT solutions supporting a variety of programming environments, tools, security, cloud connectivity and hardware.

For more resources and to learn how the new Intel® IoT Developer Kit v1.0 can help streamline your IoT projects:

• Download the Intel® IoT Developer Kit
• Visit the Intel® Developer Zone for IoT
• Attend one of our Roadshows for hands-on training in creating your own IoT projects