Smart Glasses to Help the Blind, With Pivothead LiveModPro and Intel Edison

Get access to the new Intel® IoT Developer Kit, a complete hardware and software solution that allows developers to create exciting new solutions with the Intel® Galileo and Intel® Edison boards. Visit the Intel® Developer Zone for IoT.

You want to do cool computer vision tricks with Intel Edison ? Yes, but why not work on a project to help the blind and put your coding skills to good use !

My project was to perform basic but robust computer vision tricks to help a blind person. Tricks such as :

• detect barcodes with zbar, get the description from a local source or from internet databases, read the text with epseak
• detect colors at the center of the frame, find the html and pantone names, read the name
• find colors that would look good with this color, to select matching clothes, and read them
• find faces in front of you, say where they are compared to you and how far. read the text in stereo to give spatial information on where the face is located

First step is to pick a camera. Pivothead SMART glasses were an obvious choice. Image quality is awesome, hardware codec too, there's cheap and super light. Turns out they have an Intel Edison extension card called "LiveModPro" to do computer vision in the glasses, on battery. Perfect for my project.

Steps :

• flash edison with build v3
• setup the board with internet access
• install repositories from repo.opkg.netupgrade node, but not all the packages (kernel would not work well if you do)
• install packages : opkg install fswebcam nano espeak ffmpeg-x264-presets gps-utils htop git lighttpd ofono opencv opencv-dev opencv-staticdev opencv-apps opencv-dev opencv-samples opencv-samples-dev python-opencv python-pip python-numpy zbar mjpg-streamer gstreamer1.0-plugins-good-interleave gstreamer1.0-plugins-good-audiofx
• install node packages : npm install -g fs sleep tinycolor2 array-unique striptags color-namer color-scheme onecolor util request shelljs-nodecli linux-input-device canvas okrabyte ocra.js
• unpack the source from http://dl.free.fr/l31NwuXWp
• setup webserver on port 81, and root folder as /home/root/www/ by editing the file /etc/lighttpd.conf
• test the webcam :fswebcam -d /dev/video0 /home/root/www/shot.png and use your browser to see the file http://EDISON_IP:81/shot.png
• optional : setup a bluetooth headset or use headphones connected to the jack of the glasses

Then :

• go to pivothead-intel and launch one of the demos, with a command like : node demo_barcode.js
• you can hear the messages, or go in your web browser to see the debug interface (frame grab, faces detected, audio messages, ...)

Enjoy the video demo video and code comments !