Adding Mouse Gesture Functionality to Your .NET Application Within Seconds

• Download demo project - 57.75 KB
• Download source - 75.33 KB

Introduction

Mouse gesture commands enrich the UI of an application. They are very easy to learn and intuitive for the user. There are some postings on CodeProject that address mouse gestures in one way or the other way (unfortunately, I didn't get the AI ones running), so I thought to offer my solution to the CP community. I first saw a mouse gestures functionality in the very early 90s in a CAD kernel package written in C that I used in a project.

This article gives you the possibility to add mouse gesture functionality in your .NET project in a very easy way. It offers the following advanteges:

• definition of the mouse buttons to mask and to trigger mouse gesture capturing
• definition of the keyboard modifier buttons (shift, ctrl, alt)
• definition of different controls to allow start capturing mouse gesture
• definition of own mouse gesture commands
• visual feedback to the user while capturing the mouse gesture
• Manager Form to manage (teach, delete, copy, import, export, properties) the mouse gestures
• simple usage in any control (form, panel, group, etc)

• raising an event to respond to when a mouse gesture is entered

The Idea

It doesn't really need to be said but first we have to capture the mouse movement/position. Once done, these x/y positions can be analysed to find an appropriate mouse gesture command.

The idea of finding a mouse gesture is as follows:

1. Finding the bounding rectangle of the mouse movement
2. Dividing this rectangle into the same number of rectangular fields of the same size (width and height are devided by the same number) — in other words: laying a square grid over this rectangle
3. Number the fields of the grid from top/left to bottom/down from left to right starting at 0
4. Tracking the path/fields in which the mouse moved, starting on MouseDown and stopping on MouseUp event
5. Convert the field numbers/the path into a key
6. Lookup the command for the found key

Example:

Path: 0, 1, 2, 3, 4, 5, 4, 10, 9, 15, 14, 20, 19, 18, 24, 30, 31, 32, 33, 34, 35
Key: ABCDEFEKJPOUTSYefghij

With this solution a specific command has several keys — one key for every possible path for a mouse gesture. That means the more fields we divide the bounding rectangle into, the more keys a command needs to be found. A more complicated mouse gesture consists of a longer path which ends in more possible keys.

As we can see, a very important point is the number of rectangles we divide the bounding rectangle into. If the number is too small, we will not have enough fields to differentiate similar mouse gestures. To illustrate this, imagine the smallest possible divider equal to 1 to divide each side of the bounding rectangle; in other words, not to divide the bounding rectangle. So every mouse gesture's path is 0, no matter if it's just a point or millions of movements. On the other side, if we divide the rectangle into too many fields, there are lots of path's which are very similar.

I made several tests with different dividers for the bounding rectangle and finaly used 6 as the best avarage which gets 36 fields.

The key should be as short as possible, because we will have many of them for one command as more complex a mouse gesture is. Because mouse gestures are not limited to a certain length or movements, we don't know how long the key can get. The simplest solution is to use a string that consists of the Base64 encoded field numbers — the path.

The Implementation

Class MouseGestureData

Class MouseGestureData has no code, it just holds the variables that are the same for all MouseGesture instances. It is implemented with the singleton pattern as Microsoft suggests on MSDN. It has a private constructor and the member properties can be accessed using the public static readonly Instance variable.

Class MouseGesture

Class MouseGesture is the main class and does all the important work. The only constructor takes two arguments. The first one is a reference to a Control object. This object is the parent control in which the mouse gesture is working in. The constructor registers to the MouseUp, MouseMove and MouseDown events of the parent control and all its containing controls that are allowed. Argument two is a List<Type> list that tells which type of controls are allowed. If it is null, the list returned by method GetDefaultAllowedControls() is used. The idea is to allow all kind of "container controls" meaning that these are controls that "show" some sort of the parent control's background and not having selectable elements and not being data entering controls. The default allows Label, GroupBox, PictureBox, ProgressBar, ScrollableControl and TabControl.

In the class MouseGesture, you can define different properties needed to start capturing the mouse positions and to trigger mouse gesture commands. MouseButtonTrigger defines the mouse button that must be pressed down to trigger/start and stop capturing the mouse positions. This means the mouse button that raises the MouseDown and MouseUp events must be equal to this property. Also all mouse button states must be equal to MouseButtonMask and the modifier keys (ctrl, shift, alt) must be equal to ModifierKeyMask when staring the capturing.

private void OnMouseDown( object sender, MouseEventArgs e )
{
    // only enter capturing if...
    if( !_bCapturing &&                                     // ...not capturing in
                                                            // progress
        e.Button == _data.MouseButtonTrigger &&             // ...the button pressed last
                                                            // to trigger
        Control.MouseButtons == _data.MouseButtonMask &&    // ...this/these button/s
                                                            // pressed together
        Control.ModifierKeys == _data.ModifierKeyMask       // ...this/these modifier
                                                            // key/s (shift/ctrl/alt)
      )
    {
        // capturing mouse positions starts here
        ...
    }
}

The default is only the right mouse button to be pressed. But with the additional properties MouseButtonMask and ModifierKeyMask, we could, for instance, start capturing when 1st the left mouse button is pressed and kept and 2nd the right mouse key is pressed:

MouseButtonTrigger = MouseButtons.Right;
MouseButtonMask = MouseBottons.Left | MouseButtons.Right

Or the ctrl-key plus the right mouse botton is pressed:

MouseButtonTrigger = MouseButtons.Right;
MouseButtonMask = MouseButtons.Right
ModifierKeyMask = Keys.Control;

It is possible to give the user a visual feedback when the mouse positions are captured. It is done in three levels. The first level is that we show a window on which we draw the mouse gesture. This window has no frame, but we setup the Opacity and BackColor to show the user that the imput window has changed. Property WindowAppearance defines how to show the window:

• None No capturing positions and/or window effects are shown.
• FullScreenOpaque An opaque window covers the full screen on which the capturing points/lines are drawn. This means the capturing points/lines can be drawn outside the parend window.
• ParentOpaque An opaque window covers the parent window on which the capturing points/lines are drawn. This means the capturing points/lines can only be drawn inside the parend window.
• ParentClear The parent window is not changed on which the capturing points/lines are drawn. This means the capturing points/lines can only be drawn inside the parend window.

The second level for the visual feedback is drawing the mouse positions and is defined in property GestureAppearance.

The third level is some properties that define if the bounding rectangle, the grid and the path of the mouse gesture should be drawn and its colors. There is also a property that defines for how much longer the capturing window should be shown after capturing the mouse position is finished.

The analysis of the captured mouse positions is straightforward, there isn't really any tricky code behind it. But there are two properties to mention. Property MinimumMovement defines a minimum number of pixels the mouse has to be moved to create a key and lookup for an appropriate command. Property UnidimensionalLimit defines the minimum width or height for a field in which the bounding rectangle is divided to. This is done because it is almost impossible to enter an orthogonal movement (in x or y direction) with a mouse. This was the only part where I saw a possibility to reduce the number of keys for a command.

Once capturing in the mouse positions is started and the MouseButtonTrigger is released, on MouseUp event, the mouse positions are analysed. After this MouseGesture raises event MouseGestureEntered, no matter if it found a key and it's appropriate command. It even raises the event if it was an invalid mouse movement. MouseGestureEventArgs of delegate MouseGestureEventHandler has the following properties:

• Key is the key of the entered mouse gesture. It is string.Empty if no key was found.
• Command is the appropriate command to the found key. It is string.Empty if no command was found.
• Bounds is the bounding rectangle of the entered mouse gesture. This is always relative to the upper left corner of the parent control that was passed in the constructor.
• Control is the control in which the mouse gesture started. The idea of this is to be able to handle different commands depending on where the user started drawing the mouse gesture.

These values of the last MouseGestureEntered event can be read from the MouseGesture object properties starting with "Last".

Class Commands

This class holds a Dictionary with the key/command pairs. It hides the dictionary to be able to make checks on the operations; i.e. not throwing an exception if a key already exists. It also implements reading from and writing to file in XML format. It's done in a structured way (one command with multiple keys) instead of a flat way (one entry for each key/command pair).

Class MouseGestureManager

You can draw mouse gestures on the manager form on all "container controls" (as explained above) plus the list control on the second tab. The allowed controls are set to default. When an event raises, it is visualized in the group box bellow the tabs: Red means no valid mouse gesture, green means a valied mouse gesture was entered and its key generated, but no appropriate command was found, whereas blue means that a command was found.

Tab Parameter allows the setting of all the possible properties of MouseGestureData. You can draw mouse gestures on the tab. Play around with the parameters to immediately see the effects.

Tab Keys & Commands allows you to manage everything around keys and commands. Select or type in a command in the drop down combo box Commands to see all appropriate keys in the list bellow it. Group box Selected key shows the current/selected key as a stretched thumb and lets you delete it from the list.

Button Delete all asks for a confirmation before deleting all keys/commands from the list.
Buttons Import and Export do just what'd you expect from them.
Button Copy Cmd allows you to copy all keys of the selected command to a new one, while mirroring or rotating them.

Probably the most important button is the Add Mouse Gesture at the top right border of the tab. It is a button style check box and when pressed, every valid mouse gesture you draw on tab Keys & Commands (must be visible) is added to the current command. It is immediately selected on the list box, so you can delete if right afterwards if you are not happy with it. If you want to add a new command, just enter its name in the drop down combo box and start drawing the mouse gestures. (Unfortunately, the first character typed in the combo box is not acceppted, IMO a .NET problem).

Consecutive cmd in group box Total counters is increased everytime a valid command is found and is set to 0 if not. It can be used while teaching new mouse gestures to get an idea of how god the system already is.

How to Use the Code

Using the DcamMouseGesture functionality just needs a few lines of code after adding a reference to DcamMouseGesture.dll in your project:

1. Refer to namespace DcamMouseGesture
2. Load a file with the commands and keys once in your application
3. For each Form you want to use mouse gestures, instantiate an object of class MouseGesture and register to the MouseGestureEvent event
4. In your registered MouseGestureEventHandler, handle the commands you want

That's it! Here is an example with the minimum of steps needed (unnecessary VS code deleted):

// a) Refer to namespace "DcamMouseGesture"
using DcamMouseGesture;

namespace WindowsFormsApplication
{
    public partial class Form1 : Form
    {
        MouseGesture _mg;

        public Form1()
        {
            InitializeComponent();

            // b) Load a file with the commands and keys once in your application
            MouseGestureData.Instance.Commands.ReadFile( 
                Environment.CurrentDirectory + @"\MouseGestureCommands.xml" );

            // c) For each Form you want to use mouse gestures...
            _mg = new MouseGesture( this, null ); 
            _mg.MouseGestureEntered += new MouseGestureEventHandler( 
                OnMouseGestureEntered );
        }

        private void OnMouseGestureEntered( object sender, MouseGestureEventArgs e )
        {
            // d) In your registered MouseGestureEventHandler, handle the commands
            // you want
            MessageBox.Show( string.Format( "OnMouseGestureEntered:\n" +
                                            "   Command:\t{0}\n" +
                                            "   Key:\t\t{1}\n" +
                                            "   Control:\t\t{2}\n" +
                                            "   Bounds:\t\t{3}", 
                                            e.Command, e.Key, e.Control,
                                            e.Bounds.ToString() ) );
        }

    }
}

The downloadable source code is a .NET 2.0 solution created in Visual Studion 2008 Standard Edition. I used the new feature auto-implemented proberties wherever possible. So if you want to use the source code in an older version of Visual Studio, you have to declare these as member variables and it's appropriate properties.

History

2008-05-12 - V1.0

• first post