Create a Cross-browser Touch-based Joystick with Hand.js

I currently work on several gaming projects for modern browsers, as well as for Windows Store apps. Some of these are based on HTML5, so as to simplify multi-devices support. I then began looking for a way to address user inputs on all platforms - Windows 8 and Windows RT, Windows Phone 8, iPad, Android & FirefoxOS.

As you’ve may have read in my previous article on Unifying Touch and Mouse across browsers with Pointer Events, Internet Explorer 10 on Windows 8, Windows RT and Windows Phone 8 implement the Pointer Events model we’ve submitted to the W3C. IE10 on Windows 7 has partial support for these as well. In order to address in a unified way this Pointer Events model and the one implemented in WebKit based browsers, we’re going to use David Catuhe's HandJS library. Check out his blog post on HandJS as a Polyfill for supporting pointer events on every browser. The idea is to target the Pointer model and the library will propagate the touch events to all platforms specifics.

Once I had all the technical pieces, I was looking for a great way to implement a virtual touch joystick in my game. I’m not a huge fan of arrow keys buttons for a touchscreen. On the other hand, the virtual analogic pad are often not very well placed. But I’ve finally discovered that Seb Lee-Delisle had already considered this and has created an awesome concept, described in Multi-touch game controller in JavaScript/HTML5 for iPad. The code is available on GitHub here: JSTouchController

The idea was to take his code and refactor its touch components to target the Pointer model instead of the original WebKit Touch approach. While working on this several months ago, I discovered that Boris Smus from Google had already started to do something similar, while working on his own library Pointer.js, as described in his article Generalized input on the cross-device web. However, at that time, Boris was mimicking an old version of the IE10 Pointer Events implementation and his library wasn’t working in IE10. That’s why we've decided to work on our own version. Indeed, David’s library is currently targets the latest, and very recent W3C version of pointer events, which is currently in last call draft status. If you’re having a look at both librairies, you’ll see also that HandJS is using some different approaches in several parts of the code. We will then use HandJS in this article to build our touch joystick.

Sample 1: Pointers tracker

This sample helps you tracking the various inputs on the screen. It tracks and follow the various fingers pressing the canvas element. It’s based on Seb’s sample available on GitHub here: Touches.html

Thanks to Hand.js, we’re going to make it compatible for all browsers. It’s even also going to track the stylus and/or the mouse based on the type of hardware you’re currently testing on!

The same webpage provides the very same result under Chrome on Windows 8 or on an iOS/Android/FirefoxOS devices (except that pen is only supported by IE10). Thanks to HandJS, write it once and it will run everywhere!

As you just saw in the video, the cyan pointers are of type "TOUCH" whereas the red one is of type "MOUSE". If you have a touch screen, you can experience the same result by testing this page embedded in this iframe:

This sample works fine on a Windows 8/RT touch device, a Windows Phone 8, an iPad/iPhone or Android/FirefoxOS device! If you don’t have a touch device, HandJS will automatically fallback to mouse. You should then be able to track at least 1 pointer with your mouse.

Let’s see how to obtain this result in a unified way. All the code lives in Touches.js

In this code, I register the pointerdown/move/up event as described in my introduction article on MSPointer Events. In the pointerdown handler, I’m catching the ID, the X and Y coordinates, as well as the type of pointers (touch, pen or mouse) inside an object generated on the fly pushed in the pointers collection object. This collection is indexed by the id of the pointers. The collection object is described in Collection.js. The draw() function is then enumerating this collection to draw some cyan/red/lime circles based on the position of the pointer as well as its type, when you touching the screen. It also adds some text on each circle’s side to display the pointer’s details. The pointermove handler is responsible for updating the coordinates of the associated pointer in the collection and the pointerup/out simply removes it from the collection. Hand.JS makes this code compatible by propagating pointerdown/move/up/out to the associated MSPointerDown/Move/Up/Out on IE10 events and to the touchstart/move/end events for WebKit’s browsers.

"use strict";

// shim layer with setTimeout fallback
// use this to requestAnimationFrame across browsers
window.requestAnimFrame = (function () {
    return window.requestAnimationFrame ||
    window.webkitRequestAnimationFrame ||
    window.mozRequestAnimationFrame ||
    window.oRequestAnimationFrame ||
    window.msRequestAnimationFrame ||
    function (callback) {
        window.setTimeout(callback, 1000 / 60);
    };
})();

var pointers; // collections of pointers (regardless of input type)

var canvas,
c; // c is the canvas' context 2D

document.addEventListener("DOMContentLoaded", init);

// resize the canvas if user rotates slate or resizes browser window
window.onorientationchange = resetCanvas;
window.onresize = resetCanvas;

function init() {
    setupCanvas();
    pointers = new Collection();
    canvas.addEventListener('pointerdown', onPointerDown, false);
    canvas.addEventListener('pointermove', onPointerMove, false);
    canvas.addEventListener('pointerup', onPointerUp, false);
    canvas.addEventListener('pointerout', onPointerUp, false);
    requestAnimFrame(draw);
}

function resetCanvas(e) {
    // resize the canvas - but remember - this clears the canvas too.
    canvas.width = window.innerWidth;
    canvas.height = window.innerHeight;

    //make sure we scroll to the top left.
    window.scrollTo(0, 0);
}

function draw() {
    c.clearRect(0, 0, canvas.width, canvas.height);

    pointers.forEach(function (pointer) {
        c.beginPath();
        c.fillStyle = "white";
        c.fillText(pointer.type + " id : " + pointer.identifier + " x:" + pointer.x + " y:" + 
                           pointer.y, pointer.x + 30, pointer.y - 30);

        c.beginPath();
        c.strokeStyle = pointer.color;
        c.lineWidth = "6";
        c.arc(pointer.x, pointer.y, 40, 0, Math.PI * 2, true);
        c.stroke();
    });

    requestAnimFrame(draw);
}
// on detecting pointer events, create the pointer object to add to the collection 
// for different input type, show different color and text
function createPointerObject(event) {
    var type;
    var color;
    switch (event.pointerType) {
        case event.POINTER_TYPE_MOUSE:
            type = "MOUSE";
            color = "red";
            break;
        case event.POINTER_TYPE_PEN:
            type = "PEN";
            color = "lime";
            break;
        case event.POINTER_TYPE_TOUCH:
            type = "TOUCH";
            color = "cyan";
            break;
    }
    return { identifier: event.pointerId, x: event.clientX, y: event.clientY, type: type, color: color };
}

function onPointerDown(e) {
    pointers.add(e.pointerId, createPointerObject(e));
}

function onPointerMove(e) {
    if (pointers.item(e.pointerId)) {
        pointers.item(e.pointerId).x = e.clientX;
        pointers.item(e.pointerId).y = e.clientY;
    }
}

function onPointerUp(e) {
    pointers.remove(e.pointerId);
}

function setupCanvas() {
    canvas = document.getElementById('canvasSurface');
    c = canvas.getContext('2d');
    canvas.width = window.innerWidth;
    canvas.height = window.innerHeight;
    c.strokeStyle = "#ffffff";
    c.lineWidth = 2;
}

If you wish, you can view the complete source code here.

Sample 2: Video game controller with a simple spaceship game

Let’s now take a look at the sample I was the most interested in - a analog touch-pad for HTML5 games. The user should be able to touch anywhere on the left side of the screen. At this position, the canvas will display a simple but very efficient direction pad. Moving your finger, while still pressed, will update the virtual touch pad and will move a simple spaceship. Touching the right side of the screen will display some red circles and those circles will generate some bullets getting out of the spaceship. Once again, It’s based on Seb’s sample available on GitHub here: TouchControl.html.

If you have a touchscreen, you can give this game a try in the frame below:

If not, you will only be able to move the ship using your mouse by clicking on the left of the screen or firing by clicking on the right side, but you won’t be able to achieve both actions simultaneously. As you can see, HandJS is providing a mouse fallback if the browser or platform doesn’t support touch.

Note: The iPad seems to suffer from an unknown bug which prevents this second iframe to work correctly. Open the sample directly in another tab to make it works on your iPad.

Let’s see again how to obtain this result in a unified way. All the code lives this time in TouchControl.js, which you can find here.

// shim layer with setTimeout fallback
// use this to requestAnimationFrame across browsers
window.requestAnimFrame = (function () {
    return window.requestAnimationFrame ||
    window.webkitRequestAnimationFrame ||
    window.mozRequestAnimationFrame ||
    window.oRequestAnimationFrame ||
    window.msRequestAnimationFrame ||
    function (callback) {
        window.setTimeout(callback, 1000 / 60);
    };
})();

var canvas,
c, // c is the canvas' context 2D
container,
halfWidth,
halfHeight,
leftPointerID = -1,
leftPointerPos = new Vector2(0, 0),
leftPointerStartPos = new Vector2(0, 0),
leftVector = new Vector2(0, 0);
// halfWidth and halfHeight are use to separate the screen 
// so that we can decide whether to respond to touch joystick or bullet code
// Vector2 is a custom class for two-dimensional vectors
// the leftPointerStartPos is what we will us to place the joystick
// we will track joystick movement relative to it, using leftPointerPos
// in the Vector2 type object leftVector
// We will compute direction and speed of the spaceship based on value of leftVector

var pointers; // collections of pointers
var ship;
bullets = [],
spareBullets = [];

document.addEventListener("DOMContentLoaded", init);

// resize the canvas if user rotates slate or resizes browser window
window.onorientationchange = resetCanvas;
window.onresize = resetCanvas;

// associate the custom handJS events with their corresponding even listeners
function init() {
    setupCanvas();
    pointers = new Collection();
    ship = new ShipMoving(halfWidth, halfHeight);
    document.body.appendChild(ship.canvas);
    canvas.addEventListener('pointerdown', onPointerDown, false);
    canvas.addEventListener('pointermove', onPointerMove, false);
    canvas.addEventListener('pointerup', onPointerUp, false);
    canvas.addEventListener('pointerout', onPointerUp, false);
    requestAnimFrame(draw);
}

function resetCanvas(e) {
    // resize the canvas - but remember - this clears the canvas too. 
    canvas.width = window.innerWidth;
    canvas.height = window.innerHeight;

    halfWidth = canvas.width / 2;
    halfHeight = canvas.height / 2;

    //make sure we scroll to the top left. 
    window.scrollTo(0, 0);
}

function draw() {
    c.clearRect(0, 0, canvas.width, canvas.height);

    ship.targetVel.copyFrom(leftVector);
    ship.targetVel.multiplyEq(0.15);
    ship.update();

   // if the ship moves offscreen left, redraw it re-entering from the right, and so on
    with (ship.pos) {
        if (x < 0) x = canvas.width;
        else if (x > canvas.width) x = 0;
        if (y < 0) y = canvas.height;
        else if (y > canvas.height) y = 0;
    }

    ship.draw();

    for (var i = 0; i < bullets.length; i++) {
        var bullet = bullets[i];
        if (!bullet.enabled) continue;
        bullet.update();
        bullet.draw(c);
        if (!bullet.enabled) {
            spareBullets.push(bullet);

        }
    }


    pointers.forEach(function (pointer) {
        if (pointer.identifier == leftPointerID) {
            c.beginPath();
            c.strokeStyle = "cyan";
            c.lineWidth = 6;
            c.arc(leftPointerStartPos.x, leftPointerStartPos.y, 40, 0, Math.PI * 2, true);
            c.stroke();
            c.beginPath();
            c.strokeStyle = "cyan";
            c.lineWidth = 2;
            c.arc(leftPointerStartPos.x, leftPointerStartPos.y, 60, 0, Math.PI * 2, true);
            c.stroke();
            c.beginPath();
            c.strokeStyle = "cyan";
            c.arc(leftPointerPos.x, leftPointerPos.y, 40, 0, Math.PI * 2, true);
            c.stroke();

        } else {

            c.beginPath();
            c.fillStyle = "white";
            c.fillText("type : " + pointer.type + " id : " + pointer.identifier + " x:" + pointer.x + 
                       " y:" + pointer.y, pointer.x + 30, pointer.y - 30);

            c.beginPath();
            c.strokeStyle = "red";
            c.lineWidth = "6";
            c.arc(pointer.x, pointer.y, 40, 0, Math.PI * 2, true);
            c.stroke();
        }
    });

    requestAnimFrame(draw);
}

function makeBullet() {
    var bullet;

    if (spareBullets.length > 0) {

        bullet = spareBullets.pop();
        bullet.reset(ship.pos.x, ship.pos.y, ship.angle);

    } else {

        bullet = new Bullet(ship.pos.x, ship.pos.y, ship.angle);
        bullets.push(bullet);

    }

   // make the bullet speed relative to the ship speed
    bullet.vel.plusEq(ship.vel);
}

function givePointerType(event) {
    switch (event.pointerType) {
        case event.POINTER_TYPE_MOUSE:
            return "MOUSE";
            break;
        case event.POINTER_TYPE_PEN:
            return "PEN";
            break;
        case event.POINTER_TYPE_TOUCH:
            return "TOUCH";
            break;
    }
}

// here is where we handle the logic for whether to present a joystick or bullet-firing experience
// only if there is no already existing joystick, and the user presses down on the left of the screen
// we should begin joystick calculations using leftVector
// else fire bullets

function onPointerDown(e) {
    var newPointer = { identifier: e.pointerId, x: e.clientX, y: e.clientY, type: givePointerType(e) };
    if ((leftPointerID < 0) && (e.clientX < halfWidth)) {
        leftPointerID = e.pointerId;
        leftPointerStartPos.reset(e.clientX, e.clientY);
        leftPointerPos.copyFrom(leftPointerStartPos);
        leftVector.reset(0, 0);
    }
    else {
        makeBullet();

    }
    pointers.add(e.pointerId, newPointer);
}

// track motion of pointer to alter ships direction and speed
function onPointerMove(e) {
    if (leftPointerID == e.pointerId) {
        leftPointerPos.reset(e.clientX, e.clientY);
        leftVector.copyFrom(leftPointerPos);
        leftVector.minusEq(leftPointerStartPos);
    }
    else {
        if (pointers.item(e.pointerId)) {
            pointers.item(e.pointerId).x = e.clientX;
            pointers.item(e.pointerId).y = e.clientY;
        }
    }
}

// release the joystick when user lifts up his finger/mouse
function onPointerUp(e) {
    if (leftPointerID == e.pointerId) {
        leftPointerID = -1;
        leftVector.reset(0, 0);

    }
    leftVector.reset(0, 0);

    pointers.remove(e.pointerId);
}

function setupCanvas() {
    canvas = document.getElementById('canvasSurfaceGame');
    c = canvas.getContext('2d');
    resetCanvas();
    c.strokeStyle = "#ffffff";
    c.lineWidth = 2;
}

Thanks to the great work done by Seb Lee-Delisle and David Catuhe, you now have all the pieces needed to implement your own virtual touch joypad for your HTML5 games. The result will work on all touch devices supporting HTML5!

This article is part of the HTML5 tech series from the Internet Explorer team. Try-out the concepts in this article with 3 months of free BrowserStack cross-browser testing @ http://modern.IE.

David Rousset is a Developer Evangelist at Microsoft, specializing in HTML5 and web development. This article originally appeared on his MSDN blog, Coding4Fun on 22th Feb 2013.You can follow him @davrous on Twitter.