Click here to Skip to main content
15,867,756 members
Articles / Programming Languages / Javascript
Article

Ultrabook™ Device and Tablet Windows* Touch Developer Guide

24 Apr 2013CPOL20 min read 24.9K   7  
This Guide contains information about the APIs that application developers need to use when they are developing apps targeted for Ultrabook™ devices.

This article is in the Product Showcase section for our sponsors at CodeProject. These articles are intended to provide you with information on products and services that we consider useful and of value to developers.

Contents

Abstract

This Guide contains information about the APIs that application developers need to use when they are developing apps targeted for Ultrabook™ devices. It covers information for developing apps that run on Windows* 7, Windows* 8 Desktop, Windows* Modern UI, as well as apps that run in web browsers.

This guide also provides descriptions of common user expectations for gestures and provides some guidelines for developing satisfying touch interfaces.

Introduction

Now that there are many touch-capable Ultrabook™ devices available, software developers want to create software that is designed for touch input. For an app to be fully touch-enabled, it must meet three criteria:

  • The most frequently used controls are at least 40X40 pixels, big enough to be uniquely indicated by a fingertip.
  • Relevant gestures are supported (panning, zoom, rotate, two-finger tap, press, and tap), and the effect occurs at the point of contact.
  • The app provides smooth, responsive visual feedback while panning, zooming, and rotating so that it feels highly interactive.

Whether developers are writing a new app or want to update an existing app to include touch support, this guide will help them in their journey.

For more information on writing touch-enabled apps, refer to the MSDN article, Touch Interaction Design.

Hardware Requirements

Consumers have a variety of touch-capable devices to choose from. Developers have a challenging task when attempting to develop apps that feel natural on each device. In general, touch-enabled apps should be designed to run on any of their targeted devices while taking full advantage of the touch capabilities of each. Ultrabook devices represent the high-end of touch-capability and should be the design point for an app’s touch interface.

Ultrabook devices with touchscreens have been available since 2011:

  • 2nd generation Intel® Core™ Processor Family (codenamed Sandy Bridge). These systems launched with Windows 7 and some have touchscreens.
  • 3rd generation Intel® Core™ Processor Family (codenamed Ivy Bridge). These systems also launched with Windows 7 first, but are now being sold with Windows 8. We will see more of these having touch capability in 2013 and beyond, especially since many of the newer designs are convertibles.

The rest of this Developer Guide assumes that the target platform is a touch-capable Ultrabook system. Software designed for this device can be adapted to other touch-enabled devices that run the same OS.

Operating Environments for Ultrabook Devices

Designing apps today requires careful considerations of what environment customers most often use as well as which environment an app is best suited for. Whether an app is targeted for Windows 8 Desktop or as a Windows Store app, the developer needs to understand which interfaces are applicable.

If an app is targeted for the Windows 8 UI (also known as a Windows Store app), then it must use the WinRT APIs. If an app is to run in the Windows 8 Desktop environment, there are more choices: the legacy APIs from Windows 7 and the new Windows 8 APIs for touch. These interfaces will be discussed further in the sections below. Other options exist for developing web apps. Touch interfaces available for web browsers are also discussed below.

Touch Interactions: Pointer, Gesture, and Manipulation

There are varying levels of interpretation of touch input. Pointer events are the most basic because they represent individual points of touch contact. Gesture and Manipulation events are built upon that foundation. Gesture events provide an easy way to capture simple tap-and-hold gestures. Manipulation events are for touch interactions that use physical gestures to emulate physical manipulation of UI elements. Manipulation events provide a more natural experience when the user interacts with UI elements on the screen. The available touch interfaces have varying levels of support for these three levels of interpretation.

Read more about Gestures, Manipulations and Interactions on MSDN

Pointer Interactions

A pointer event is a single, unique input or “contact” from an input device such as a mouse, stylus, single finger, or multiple fingers. When a contact is made, the system creates a pointer when it is first detected and then it is destroyed when the pointer leaves the detection range or is canceled. In the case of multi-touch input, each contact is a unique pointer. Table 1 shows the interfaces for retrieving basic pointer events that are available to Ultrabook devices running Windows 7 and Windows 8.

Table 1. Pointer APIs available for Ultrabook Devices

OS CompatibilityTouch InterfaceRemarks

Windows* 7 (Desktop)

WM_TOUCH

  • Also compatible with the Windows 8 Desktop environment.
  • Maximum number of simultaneous touches limited by hardware.
  • No built-in gesture recognition.
  • Must call RegisterTouchWindow since WM_TOUCH messages are not sent by default.

Windows 8 only (Desktop)

WM_POINTER

  • Applicable only to the Windows 8 Desktop environment.
  • By default Windows 8 animations and interaction feedback is generated and available for further processing.

Windows Modern UI

PointerPoint

  • Applicable only for Windows Store apps.

Touch Interfaces available for Windows 7 and Windows 8:

Refer to Guidelines for common user interactions on MSDN.

Windows 7 and Windows 8 Desktop Touch Interface: WM_TOUCH

The WM_TOUCH message can be used to indicate that one or more pointers, such as a finger or pen, have made contact on the screen.

Sample Code:

Guidance:

Windows 8 Desktop Touch Interface: WM_POINTER

The WM_POINTER messages are part of the Direct Manipulation APIs and are specific to the Windows 8 Desktop. This interface can be used to capture individual touch pointers as well as Gestures and Manipulations. The WM_POINTER messages will be discussed further in the section on Manipulation and Gesture Interactions.

Reference on MSDN: Direct Manipulation APIs

Windows Modern UI Touch Interface: PointerPoint

The PointerPoint class is part of the Windows Runtime environment and is compatible only with Windows Store apps. It provides basic properties for the input pointer associated with a single mouse, stylus, or touch contact. MSDN has sample code that can help developers get started working with the PointerPoint interface.

Sample Code on MSDN: Input: XAML user input events sample

Manipulation and Gesture Interactions

Gesture events are used to handle static-finger interactions such as tapping and press-and-hold. Double-tap and right-tap are derived from these basic gestures:

  • Gestures: the physical act or motion performed on or by the input device that can be one or more fingers, a stylus, or a mouse.
  • Manipulation: the immediate, ongoing response an object has to a gesture. For example, the slide gesture causes an object to move in some way.
  • Interactions: how a manipulation is interpreted and the command or action that results from the manipulation. For example, both the slide and swipe gestures are similar but the results vary according to whether a distance threshold is exceeded.

Table 2. Basic gestures used for manipulation interactions

GestureTypeDescription

Press and Hold

Static Gesture

A single contact is detected and does not move. Press and hold causes detailed information or teaching visuals to be displayed without a commitment to an action.

Tap

Static Gesture

One finger touches the screen and lifts up immediately.

Turn

Manipulation Gesture

Two or more fingers touch the screen and move in a clockwise or counter-clockwise direction.

Slide

Manipulation Gesture

One or more fingers touch the screen and move in the same direction (also called Panning)

Swipe

Manipulation Gesture

One or more fingers touch the screen and move a short distance in the same direction.

Pinch

Manipulation Gesture

Two or more fingers touch the screen and move closer together or farther apart.

Stretch

Manipulation Gesture

Two or more fingers touch the screen and move further apart.

Table 3. Gesture Interfaces available for Ultrabook™ Devices

OS CompatibilityGESTURE InterfaceRemarks

Windows 7

Windows 8 (Desktop)

WM_TOUCH + IManipulationProcessor

  • This combination gives the developer the functionality of the WM_POINTER API that is available only to Windows 8 desktop.
  • Maximum Touch points dictated by hardware.

Windows 7

Windows 8 (Desktop)

WM_GESTURE + GESTUREINFO structure

  • Maximum of two simultaneous touch points
  • No simultaneous gestures
  • If the app requires more complex manipulations than what is available from the WM_GESTURE message, a custom gesture recognizer needs to be written using the WM_TOUCH interface.

Windows 8 (Desktop)

WM_POINTER

  • Gesture interactions result from the use of the Direct Manipulation APIs, which take in a stream of the pointer input messages.

Windows Modern UI

PointerPoint

  • Gesture interactions result from the use of GestureRecognizer, which takes the output from PointerPoint.

References on MSDN:

Table 4. Standard expected interactions and consequences for touch interactions

InteractionsDescription

Press and Hold to learn

Causes detailed information or teaching visuals.

Tap for primary action

Invokes a primary action, for example launching an application or executing a command.

Slide to pan

Used primarily for panning interactions but can also be used for moving, drawing, or writing. Can also be used to target small, densely packed elements by scrubbing (sliding the finger over related object such as radio buttons).

Swipe to select, command, and move

Sliding the finger a short distance, perpendicular to the panning direction, selects objects in a list or grid.

Pinch and stretch to zoom

Not only used for resizing, this interaction also enables jumping to the beginning, end, or anywhere within the content with Semantic Zoom. A SemanticZoom control results in a zoomed out view for showing groups of items and quick ways to go back to them.

Turn to rotate

Rotating with two or more fingers causes an object to rotate.

Swipe from edge for app commands

App commands are revealed by swiping from the bottom or top edge of the screen.

Swipe from edge for system commends

Swiping from the right edge of the screen shows the “charms” that are used for system commands. Swiping from the left edge results in cycling through currently running apps and sliding from the top edge toward the bottom of the screen closes the app. Sliding from the top edge down and to the left or right edge snaps the current app to that side of the screen.

Interpreting Manipulation and Gesture Interactions for Windows 7 Desktop

The IManipulationProcessor interface can be used in conjunction with the WM_TOUCH API to provide a way to add translation, rotation, scaling, and inertia to UI objects. This combination provides functionality similar to the gesture recognizing features of WM_POINTER. Once the Manipulation Processor is enabled, manipulation starts as soon as a touch gesture is initiated.

Sample Code:

WM_GESTURE messages have a structure called GESTUREINFO that is available for the interpretation of gestures and manipulations. The MSDN web page for WM_GESTURE shows an example of how to obtain gesture-specific information using the GESTUREINFO structure

Note that WM_GESTURE has limitations, such as the maximum number of simultaneous touch inputs is only two and it does not support simultaneous gestures. For apps that require more capability but still need to support Windows 7 desktop, use the WM_TOUCH interface and either write a custom gesture recognizer, as detailed in the section Custom Gesture Recognition below, or use the Manipulation Processor interface with WM_TOUCH.

Sample Code on Intel Developer Zone (WM_GESTURE API + GESTUREINFO: Sample Application: Touch for Desktop

For more information on writing touch-enabled apps, refer to the MSDN article: Touch Interaction Design.

Handling Manipulation and Gesture Interactions for Windows 8 Desktop Apps

Applications targeted only for the Windows 8 Desktop can use the Direct Manipulation APIs (WM_POINTER messages). The pointer messages are passed to an internal Interaction Context object that performs recognition on the manipulation without the need to implement a custom gesture recognizer. There is a callback infrastructure where all interactions involving tracked contacts are managed.

Direct Manipulation is designed to handle both manipulation and gesture interactions and supports two models for processing input:

  1. Automatic/Independent: Window messages are automatically intercepted by Direct Manipulation on the delegate thread and handled without running application code making it independent of the application.
  2. Manual/Dependent: Window messages are received by the window procedure running in the UI thread, which then calls Direct Manipulation to process the message making it dependent on the application.

Gestures can be captured by initializing Direct Manipulation and preparing the system for input processing.

Refer to the Quickstart: Direct Manipulation on MSDN for an outline of the API calls required to accomplish typical tasks when working with Direct Manipulation.

Handling Manipulation and Gesture Interactions in Windows 8 Modern UI

The GestureRecognizer API is used to handle pointer input to process manipulation and gesture events. Each object returned by the PointerPoint method is used to feed pointer data to the GestureRecognizer. The gesture recognizer listens for and handles the pointer input and processes the static gesture events. For an example of how to create a GestureRecognizer object and then enable manipulation gesture events on that object see the MSDN GestureRecognizer web page (referenced below.)

References on MSDN:

Sample Code on Intel Developer Zone: Sample Application: Touch for Windows* Store.

Custom Gesture Recognition

When possible, use the built-in gesture recognizers (see Table 3). If the provided gesture and manipulation interfaces do not provide the functionality that is needed, or maybe the app needs to disambiguate between taps and gestures more rapidly, it may be necessary to write custom gesture recognition software. If this is the case, customers expect an intuitive experience involving direct interaction with the UI elements in the app. It is best to base custom interactions on the standard controls to keep user actions consistent and discoverable. Custom interactions should only be used if there is a clear, well-defined requirement and basic interactions don't support the app’s desired functionality. See Table 4 for the list of common and expected interactions and consequences for touch interactions.

Code Sample on Intel Developer Zone (WM_TOUCH with custom gesture recognition): Touch for Windows Desktop

Touch Support in Web Browsers

Touch input is also available to apps running in web browsers, with varying degrees of support depending on the browser. Since web browser capabilities change rapidly, it is generally better to detect supported features instead of specific browsers. Feature detection has proven to be a more effective technique once the determination has been made if it is Internet Explorer (IE) 10, a browser built on Webkit*, or a different browser that requires support. Feature detection requires less maintenance for the following reasons:

  • New browsers get released and existing browsers are often updated. Existing code may not factor in the new browser versions. Updated browsers may support standards and features that were not supported when the browser detection code was designed.
  • New devices frequently include new versions of browsers, and so browser detection code must be reviewed continually to support the new browsers. Creating customized implementations for each browser can become extremely complicated.
  • Many browsers support the ability to modify the user-agent string making browser detection difficult to accurately identify.

WebKit powers Apple Safari* and Google Chrome*, and soon Opera will move their browser over to use it. Internet Explorer 10 does not use WebKit; however, both WebKit and IE 10 are built on top of the Document Object Model (DOM) Level 3 Core Specification. To review the standards associated with touch events, refer to the standard’s Touch Events Version 1, dated January 2013.

References:

IE 10 has its own touch interfaces that must be called for processing touch events. Use the navigator object with the userAgent property to determine if the browser supports the desired features. The following example indicates that the browser is Internet Explorer.

Usage:

JavaScript
<script type="text/JavaScript">
If (navigator.userAgent.indexOf("MSIE")>0)
    { 
         // Run custom code for Internet Explorer.
    }
</script>
Figure 1. Snippet for determining if browser is Internet Explorer*

Use the hasFeature method to determine if specific features are supported in the browser. For example, here is how to determine if a browser supports touch events (this works for IE 10 as well):

JavaScript
var touchSupported = document.implementation.hasFeature("touch-events","3.0");

Where “touch-events” is the feature that we are checking for and “3.0” is the DOM specification level that we are interested in. An app can then listen for the following touch events: touchstart, touchend, touchmove, and touchcancel

Reference: hasFeature method

To process touch events using a WebKit-based browser (Chrome, Safari, etc.), simply set up the following three events to cover the main input states:

JavaScript
canvas.addEventListener( ‘touchstart’, onTouchStart, false );
canvas.addEventListener( ‘touchmove’, onTouchMove, false);
canvas.addEventListener( ‘touchend’, onTouchEnd, false);

For Internet Explorer, reference the MSPointer event instead:

JavaScript
canvas.addEventListener( ‘MSPointerDown’, onTouchStart, false );
canvas.addEventListener( ‘MSPointerMove’, onTouchMove, false);
canvas.addEventListener( ‘MSPointerUp’, onTouchEnd, false);

Similarly, there are gesture event listeners as well. They are gestureStart, gestureChange, and gestureEnd for the non IE 10 browsers.

Download sample code handling DOM pointer events on MSDN: Input DOM pointer event handling sample.

Internet Explorer 10 and its Compatibility with Windows 7

While IE 10 does not use WebKit, it is built on top of the DOM Level 3 Events, HTML5, and Progress Event standards. This section provides information about IE 10 and how it interacts on Windows 7.

Standards:

Internet Explorer 10 on Windows 7 handles touch and pen input as simulated mouse input for the following Document Object Model (DOM) events:

  • MSPointerCancel
  • MSPointerDown
  • MSPointerMove
  • MSPointerOver
  • MSPointerUp

IE 10 on Windows 7 will not fire any of the following DOM Gesture events:

  • MSGestureChange
  • MSGestureEnd
  • MSGestureHold
  • MSGestureStart
  • MSGestureTap
  • MSManipulationStateChanged

Table 5. Touch Interfaces for Internet Explorer* 10

InterfaceWindows* 7
MSVS 2010
Windows 8
MSVS 2012 (Desktop)
Windows 8 Modern UIRemarks

MSGESTURE

NO

Yes

Yes

  • Get high level gestures such as hold, pan, and tap easily without capturing every pointer event individually

MSPOINTER

Yes

YES

Yes

  • Part of DOM Object Model (DOM) Core
  • Methods getCurrentPoint and getIntermediatePoints both retrieve a collection of PointerPoint objects and are available only on Windows 8.

For more information on developing touch-enabled web apps for IE 10 (MSDN): Internet Explorer 10 Guide for Developers

Sample code on MSDN: Input: Manipulations and gestures (JavaScript)

Identifying Touch Capability

Whether an app is a native app or a web app, the developer will want to add a check for hardware touch capability so that the app can configure its UI appropriately. Use the following methods to test for touch capability.

Windows 7 and Windows 8 Desktop

Apps targeting Windows 7 or for Windows 8 Desktop can call GetSystemMetrics with SM_DIGITIZER as the argument. The following code snippet is part of a Touch sample that can be downloaded from the Intel Developer Zone: Touch for Windows Desktop

C++
void SDKSample::DeviceCaps::Touch::TouchGetSettings_Click(Platform::Object^ sender, Windows::UI::Xaml::RoutedEventArgs^ e)
{
    Button^ b = safe_cast<Button^>(sender);
    if (b != nullptr)
    {
        TouchCapabilities^ pTouchCapabilities = ref new TouchCapabilities();
        Platform::String^ Buffer;

        Buffer = "There is " + (pTouchCapabilities->TouchPresent != 0 ? "a" : "no") + " 
              digitizer presentn";
        Buffer += "The digitizer supports " + pTouchCapabilities->Contacts.ToString() + " 
              contactsn";

        TouchOutputTextBlock->Text = Buffer;
    }
}
Figure 3. Windows*UI Example for Identifying Touch Capability

Web apps

For Internet Explorer, use the msMaxTouchPoints property described as follows:

Test for touch capable hardware


If (navigator.msMaxTouchPoints) {…}

Test for multi-touch capable hardware


If (navigator.msMaxTouchPoints >1) {…}

Get the maximum number of touch points the hardware supports


Var touchPoints = navigator.msMaxTouchPoints;

For Chrome and Safari, use the following (same as above but replace msMaxTouchPoints with maxTouchPoints):

JavaScript
var result = navigator.maxTouchPoints;

It can be somewhat tricky to test for touch devices generically from web apps. While some functions work well on some browsers, others indicate that touch is present when it is not, i.e., if the browser itself supports touch, it may report that touch is available even if the device is not touch-capable.

Note that MaxTouchPoints will return 0 in IE 10 (Desktop) running on Windows 7.

References:

UI design for Touch-Enabled Ultrabook Devices

Apps designed for Ultrabook devices may need to process gestures such as taps, pans, zooms, etc. Apps that are touch-enabled may do little with the raw pointer data except to pass it to gesture detection.

New applications should be designed with the expectation that touch will be the primary input method. Mouse and stylus support require no additional work; however, software developers should consider several other factors when designing touch-optimized apps.

Table 6. Considerations for Touch-Enabled Apps

References:

C++
// Check for Touch support
// Get the Touch Capabilities by calling GetSystemMetrics
BYTE digitizerStatus = (BYTE)GetSystemMetrics(SM_DIGITIZER);
// Hardware touch capability (0x80); readiness (0x40)
if ((digitizerStatus & (0x80 + 0x40)) != 0) //Stack Ready + MultiTouch
{
    RegisterTouchWindow(m_pWindow->GetHWnd(), TWF_WANTPALM);
}
Figure 2. Windows* 7 Example for identifying touch capability

Note that GetSystemMetrics can be used to find out what is the maximum number of touch points that are available:

C++
BYTE nInputs = (BYTE)GetSystemMetrics(SM_MAXIMUMTOUCHES);

Windows 8 (Windows Store apps)

Determine touch capabilities for Windows Store apps by using the TouchCapabilities class. The following code snippet can be found in the code sample on MSDN that demonstrates its use: Input: Device capabilities sample.

References:

FactorTouchMouse/Stylus

Precision

  • Contact area for fingertip is much larger than a single x-y coordinate.
  • The shape of the contact area changes with the movement
  • There is no mouse cursor to help with targeting
  • Mouse/Stylus gives a precise x-y coordinate
  • Keyboard focus is explicit

Human Anatomy

  • Fingertip movements are imprecise
  • Some areas on the touch surface may be difficult to reach
  • Objects may be obscured by one or more fingertips
  • Straight-line motion with the mouse/stylus are easier to perform
  • Mouse/Stylus can reach any part of the screen
  • Indirect input devices do not cause obstruction

Object state

  • Touch uses a two-state model. The touch surface is either touched or not. There is no hover state that can trigger additional visual feedback.
  • Three states are available: on, off, hover (focus)

Rich interaction

  • Multi-touch – multiple input points (fingertips) are available.
  • Supports only a single input point.

Software developers should supply appropriate visual feedback during interactions so that users can recognize, learn, and adapt to how their interactions are interpreted by both the app and by the OS. Visual feedback is important for users to let them know if their interactions are successful, so they can improve their sense of control. It can help reduce errors and help users understand the system and input device.

Resources for Developing Touch Applications

Related Articles on Intel Developer Zone:

  1. Comparing Touch Coding Techniques – Windows 8 Desktop Touch Sample
  2. Exploring Touch Samples for Windows* 8 apps
  3. Touch Code Sample for Windows* 8 Store
  4. Touch Code Sample for Windows* 8 Desktop
  5. Porting Win32 Apps to Windows 8 Desktop
  6. Real-Time Strategy Game with Touch Screen

Related Articles on MSDN

  1. Windows 7 Touch Input Programming Guide
  2. Architectural Overview (Windows 7)
  3. Troubleshooting Applications
  4. Adding Manipulation Support in Unmanaged Code
  5. Windows Touch Samples
  6. Build Advanced Touch Apps in Windows 8* (Video)
  7. Windows 8 SDK
  8. Input: Touch hit testing sample
  9. Desktop App Development Documentation (Windows)
  10. Windows Touch Gestures Overview (Windows)
  11. Getting Started with Windows Touch Messages (Windows)
  12. Get PointerTouchInfo function (Windows)
  13. (MSDN) Internet Explorer 10 Guide for Developers

Summary

App developers, who want to develop touch-enabled apps, whether their apps will run natively or are targeted as web apps, need to have a clear understanding of which APIs are available to them. This guide covered the interfaces available for the following environments: Windows 7, Windows 8 Desktop, Windows Modern UI, as well as apps running in web browsers. While Gestures and Manipulations are possible in Windows 7, app developers may find that the Windows 8 APIs (those targeted for the Desktop and/or for the Modern UI) offer the best options for automatic gesture recognition.

App developers who are writing touch-enabled web apps need to check their code for IE 10 since IE 10 has its own interface that must be used to process touch, gestures, and manipulation. Other Webkit-based browsers are also based on the DOM Level 3 standards and have touch and gesture event support.

This guide also covered descriptions of common Gesture and Manipulation interactions and provided some guidelines for developing touch-enabled apps.

This article was written by Gael Hofemeier. Gael is an Evangelist Application Engineer at Intel Corporation. Her focus is providing technical content that developers writing software for Intel® Architecture need. In addition to writing content, she also moderates the Business Client Forum on the Intel Developer Zone.

See Gael's Blog Author Page

Intel, the Intel logo, Ultrabook, and Core are trademarks of Intel Corporation in the US and/or other countries.

Copyright © 2013 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
United States United States
You may know us for our processors. But we do so much more. Intel invents at the boundaries of technology to make amazing experiences possible for business and society, and for every person on Earth.

Harnessing the capability of the cloud, the ubiquity of the Internet of Things, the latest advances in memory and programmable solutions, and the promise of always-on 5G connectivity, Intel is disrupting industries and solving global challenges. Leading on policy, diversity, inclusion, education and sustainability, we create value for our stockholders, customers and society.
This is a Organisation

42 members

Comments and Discussions

 
-- There are no messages in this forum --