WPF Embedded Touch Keyboard

• Download control - 38.5 KB

Introduction

I have been working on a project to build an application on a Microsoft Windows system that is dedicated to the application. The touch screen is rather small and the entire surface is dedicated to the application. Therefore, there is really only need for one window. There is a physical keyboard, but it would be desirable to not require the keyboard, or allow the user to just use the screen to input data, like on a smart phone. To expedite maximizing the effective use of the screen, especially since the screen should be easily read from a distance, using a design that goes to a special screen for editing. This allows the screen real estate to be optimized for reading for a distance, and then using a different screen design optimized to allow editing of values with a visible on-screen keyboard.

The design I was working with used a separate window for the touch screen, and all the solutions I found on the web also used a separate keyboard window. This is really no better than using the default touch screen keyboard provided by Microsoft, which is actually not really that good since it still does not really support special symbols well.

I ended up using one of the solutions I found online, in combination with the one currently in the solution. I still heavily modified the code since it still needed to be put into a control in the same window as the controls for the values being edited. There were a lot of things I did not like about the code, and it seemed to make the coding harder than if I rewrote it. Ended up very much sampling the code-behind for the keyboard (which is derived from the Grid with no associated XAML), but left the code-behind for the keys (derived from the Button) and the keyboard sections (again derived from the Grid) alone. This includes a separate set of classes to differentiate the key functionality depending on key type. This was a set of classes that inherited from several different classes to differentiate the functionality of the normal key, the single use modifier key such as the shift key, and the toggle modifier key such as caps lock key. There were also some other derived classes to handle other specialized functionality.

I wanted to have the styles for the keys to use a defined style, so, to minimize change, had a separate style for the normal key and the key down (for the Shift and Caps Lock keys).

This worked quite well, but really I was not at all happy with the design for the key functionality since it seemed to be overly complex, and did not really take advantage of a good state pattern since there was a lot of state dependent functionality implemented in the keyboard class, and thought that the use of a separate class to differentiate button types and wanted to use the Key Button as the base class, and derive from that class to create the custom functionality. I also wanted to use the ToggleButton for the base class instead of the Button so that only one Style was required.

The Keyboard Class

The heart of the design is the creation of the keys:

public override void BeginInit()
{
 SetValue(FocusManager.IsFocusScopeProperty, true);

 var mainSection = new OnScreenKeyboardSection();
 var mainKeys = new ObservableCollection<OnScreenKey>
 {
  new OnScreenKeyNormal(0, 00, new[] {"`", "~"}, KeyTextUpdateDelegateFunctions.CapsAndSpecial),
      .
      .
      .
  new OnScreenKeySpecial(4, 02,  string.Empty, " "){GridWidth = new GridLength(5, GridUnitType.Star)},
 };

 mainSection.Keys = mainKeys;
 mainSection.SetValue(ColumnProperty, 0);
 _sections.Add(mainSection);
 ColumnDefinitions.Add(new ColumnDefinition { Width = new GridLength(3, GridUnitType.Star) });
 Children.Add(mainSection);
 _allOnScreenKeys.AddRange(mainSection.Keys);
 Loaded += OnScreenKeyboard_Loaded;
 base.BeginInit();
}

The abstract class for all the keys in the OnScreenKey, from which both the OnScreenKeyNormal and the OnScreemKeySpecial, are derived, and the OnScreenKey is derived from a ToggleButton. As each key is created, it is assigned to the Grid.Column and a Grid.Row. The keyboard (OnScreenKeyboard), which is derived from the Grid, contains one or more of another class derived from the Grid, the OnScreenKeyboardSection. The keys are added to an ObservableCollection because the OnScreenKeyboardSection Keys collection needs the CollectionChanged event to allow monitor changes in the collection if keys are ever changed dynamically. Once all the keys are added to the OnScreenKeyboardSection, theGrid.Column for the section is assigned, a ColumnDefinition in the OnScreenKeyboard is added for the Column, and the OnScreenKeyboardSection is added to the Children of the OnScreenKeyboard. Then, all the keys are added to the collection of all keys so that all keys can be notified when there is a change that will affect the state of the keys, such as when the Shift key is pressed.

After all the OnScreenKeyboardSection instances have been added to the OnScreenKeyboard, there is a subscription made to Loaded event:

private void OnScreenKeyboard_Loaded(object sender, RoutedEventArgs e)
{
 _allOnScreenKeys.ForEach(x =>
 {
  x.Style = ToggleButtonStyle;
  x.OnScreenKeyPressEvent += OnScreenKeyPressEventHandler;
 });
}

The Loaded event handler is responsible for setting the ToggleButton Style to all of the OnScreenKey instances (cannot do it earlier because the Style is not available until the Loaded event). It also subscribes to the OnScreenKeyPressEvent of all the keys to allow the effect of the key press to be executed:

void OnScreenKeyPressEventHandler(object sender, OnScreenKeyPressEventArgs e)
{
 if (e.StateModifier == null)
 {
  e.Execute(_activeControl);
  var singleInstance = _activeKeyModifiers.Where(i => i.SingleInstance).Select(k => k).ToList();
  singleInstance.ForEach(j => _activeKeyModifiers.Remove(j));
 }
 else
 {
  var dups = _activeKeyModifiers.Where(i => i.ModifierType == e.StateModifier.ModifierType)
          .Select(k => k).ToList();
  dups.ForEach(j => _activeKeyModifiers.Remove(j));
  if (e.StateModifier.Clear == false)
   _activeKeyModifiers.Add(e.StateModifier);
 }
 _allOnScreenKeys.ForEach(i => i.Update(_activeKeyModifiers));
}

There are basically two cases that need to be handled on the KeyPress, when a modifier key is pressed and a normal key press.

If it is a normal key press, then need to tell the OnScreenKey that was activated to execute, passing the currently selected control to the OnScreenKey Execute method. Then check to see if any single instance modifiers exist, and clear any out of the modifiers that do.

If a modifier key is pressed, need to check to see if that modifier type is already in the the modifiers, and removing them if they are. Then add the new modifier to the collection of modifiers.

Finally, for both cases, call the Update method on all OnScreenKey instances with the current collection of key modifiers. This allows each key to change its caption in responce to the set of modifiers, and any modifiers OnScreenKey instances whose state is affected by the current set of modifiers to change to the appropriate toggle state.

The class used to pass the argument when a key is pressed is as follows:

internal class OnScreenKeyPressEventArgs
 {
  public delegate void ExecuteKeyPress(FrameworkElement frameworkElement);

  public OnScreenKeyStateModifier StateModifier { get; }

  internal OnScreenKeyPressEventArgs(OnScreenKeyStateModifier stateModifier)
  {
   StateModifier = stateModifier;
  }

  public OnScreenKeyPressEventArgs(ExecuteKeyPress execute)
  {
   Execute = execute;
  }

  public ExecuteKeyPress Execute { get; }
 }

I have found that I quite often regret not creating a specific class for the argument when I create an event since retrofitting is somewhat painful, so generally try to create an associated argument class for the event. Since I was using a class for the arguments anyway, I included the Execute method in the OnScreenKeyPressEventArgs class instead of having to casting the class to use the Execute method. I also used it to pass the key modifier. As can be seen, there was either an Execute method or a StateModifier passed in the event argument.

The Key Class

All of the keys are based on an abstract class called OnScreenKey that is based on the ToggleButton class. The constructor takes arguments to specify the Grid.Row, Grid.Column, an array of values that will be display as the caption for the key, and, optionally, the function to execute when the Execute method is called. The constructor sets these values, including the current string to display on the key cap, and subscribes to the Click event:

protected OnScreenKey(int row, int column, string[] values,
  Func<IEnumerable<OnScreenKeyStateModifier>, int> valueIndexFunction = null,
  ExecuteDelegate executeFunction = null)
{
  _values = values;
  _valueIndexFunction = valueIndexFunction ?? KeyTextUpdateDelegateFunctions.FirstInArray;
  _executeFunction = executeFunction ?? ExecuteDelegateFunctions.DefaultExecuteDelegate;
  Content = Value = values[0];
  GridRow = row;
  GridColumn = column;
  Click += KeyPressEventHandler;
}

The GridRow and GridColumn properties just bind to the Grid.RowProperty and Grid.ColumnProperty using the GetValue and SetValue methods. The KeyPressEventHandler is an abstract method. There is also a GridWidth property that is used by the OnScreenKeyboardSection for the relative key width.  

The public Execute method is called with the currently selected FrameworkElement by the OnScreenKeyboard class after it has received the initiation of the key Click event, and executes the ExecuteDelegateFunction in passed into the constructor, or the default ExecuteFunction if the value is null. This ExecuteFunction usually does the execution of the key press such as simulating a KeyPress on a TextBox.

public delegate void ExecuteDelegate(OnScreenKey button, FrameworkElement frameworkElement);

An example of the ExecuteDelegate, and also the default ExecuteDelegate is the method to handle an ordinary key press of a key that represents a character:

public static ExecuteDelegate DefaultExecuteDelegate = (key, frameworkElement) =>
{
 if (frameworkElement is TextBoxBase)
 {
  var textBoxBase = (TextBox)frameworkElement;
  var start = textBoxBase.Text.Substring(0, textBoxBase.SelectionStart);
  var end = textBoxBase.Text.Substring(textBoxBase.SelectionStart + textBoxBase.SelectionLength);
  textBoxBase.Text = start + key.Value + end;
  textBoxBase.SelectionStart = start.Length + 1;
  textBoxBase.SelectionLength = 0;
 }
 else if (frameworkElement is PasswordBox)
 {
  var passwordBoxBase = (PasswordBox)frameworkElement;
  passwordBoxBase.Password = passwordBoxBase.Password + key.Value;
 }
};

This method has a reference to the OnScreenKey and the FrameworkElement that currently has keyboard focus. The method is intended to support a key that, when pressed, is supposed to represent the press of a single character key such as the "a" key. The method handles two different types of FrameworkElements: the TextBoxBase and the PasswordBox. For the TextBoxBase, the Value, which would be the character that is currently displayed as the ToggleButton content, either inserted in the current SelectionStart location if the SelectionLength is zero, or replaces the currrent selection with the character. For a PasswordBox, only append is supported since the PasswordBox does not have the SelectionStart or SelectionLength properties.

The DefaultExecuteDelegate is used for most keys, but there are a lot of keys that require special one-time functionality. Examples of these keys would be the backspace, tab, delete, return, etc.  I could have created an inherited class for each of these cases but that would mean that there were a lot of derived classes with only minor differences.

Another important method is the Update method. It is called on all keys after a key press, unlike the Execute method which in only called on the instance that initiated the key press event. It updated the modifiers, and then updates any states, such as displaying the right caption for the keys that have been activated such as the Shift and Caps Lock keys, and, for modifier (Shift) and toggle keys (Caps Lock), put the ToggleButton IsChecked property in the right state.

public delegate int CaptionUpdateDelegate(IEnumerable<OnScreenKeyStateModifier> values);

An example of the CaptionUpdateDelegate, and also the default ExecuteDelegate is the method to handle an ordinary key press of a key that represents a character:

public static int CapsAndSpecial(IEnumerable<OnScreenKeyStateModifier> values)
{
 var capValue = values.Any(i => i.ModifierType == OnScreenKeyModifierType.Shift) ? 1 : 0;
 var specialValue =values.Any(i => i.ModifierType == OnScreenKeyModifierType.Special) ? 2 : 0;
 return capValue + specialValue;
}

Basically this function returns a "0" if neither a Shift nor Special ModifierType is included in the modifier values, "1" if just the Shift exists, "2" if just the Special exists, and "3" if both exist. It is possible that there are not three values, and this is handled in Update method of the abstract OnScreenKey:

protected virtual void Update()
{
 Content = Value = GetCurrentValue(_valueIndexFunction(Modifiers));
}

protected string GetCurrentValue(int index)
{
 return _values[Math.Min(index, _values.Length - 1)];
}

The reason for the update function instead of having this included in the inherited class is that the keyboard could include a NumLock functionality, which I have not included in this sample. There could also be other special cases. This is also why I had this as the default. I did not implement it in the abstract class basically to keep this class cleaner. There are two other of these methods that have been implemented: NumberOnly, and FirstInArray. FirstInArray is used if there is only one caption, no matter what the modifiers are, such as for Shift, Space, etc.

Example of an Inherited OnScreenKey Class

The following code is an example of a key class that inherits from the OnScreenKey class.

public class OnScreenKeyNormal : OnScreenKey
{
 private readonly Func<IEnumerable<OnScreenKeyStateModifier>, int> _valueIndexFunction;

 internal OnScreenKeyNormal(int row, int column, string[] values,
     Func<IEnumerable<OnScreenKeyStateModifier>, int> valueIndexFunction)
   : base(row, column, values, valueIndexFunction) { }

 internal override void KeyPressEventHandler(object sender, RoutedEventArgs routedEventArgs)
 {
  IsChecked = false;
  OnClick(new OnScreenKeyPressEventArgs( Execute));
 }
}

As can be seen, about the only thing this class really does is set the IsChecked property to false to keep the ToggleButton from going to this state since it is only applicable for modifier keys and do not want to have the button ever to be in the depressed state.

Modifier Type Key Class

The class used for a single use modifier key such as the Shift key is as follows:

public class OnScreenKeyModifier : OnScreenKey
{
 private readonly OnScreenKeyModifierType _modifierType;
 public OnScreenKeyModifier(int row, int column, string[] values, OnScreenKeyModifierType modifierType) : base(row, column, values)
 {
  _modifierType = modifierType;
 }

 internal override void KeyPressEventHandler(object sender, RoutedEventArgs routedEventArgs)
 {
  OnClick(new OnScreenKeyPressEventArgs(new OnScreenKeyStateModifier(_modifierType, true,
             IsChecked == false)));
 }

 protected override void Update()
 {
  IsChecked = Modifiers.Any(i => i.ModifierType == _modifierType);
 }
}

In this class need to maintain the ModifierType so that the can be communicated to the OnScreenKeyboard class when a Click event occurs, and also to know when to reset. Both the Update and KeyPressEventHandler are overridden to accomplish this. In the KeyPressEventHandler, the OnScreenKeyStateModifier is passed in the OnScreenKeyPressEventArgs. The triggering of the Update cause the IsChecked state to be set if any of the ModifierTypes in the Modifiers is of the type of the saved _modifierType.

The colleciton of OnScreenKeyStateModifier instances contain information on the current state of the modifier keys. This class is also used to communicate to the OnScreenKeyboard class when a modifier key has been activated or deactivated, the distinction being specified by Clear property. The OnScreenKeyStateModifier contains three properties: the type of the modifier (shift, num lock, special), whether is is for one character press or not (SingleInstance p;roperty) and whether this modifier should be cleared (Clear property) from the collection or added to the collection.

public class OnScreenKeyStateModifier
{
 public OnScreenKeyStateModifier(OnScreenKeyModifierType modifierType, bool singleInstance, bool clear)
 {
  Clear = clear;
  ModifierType = modifierType;
  SingleInstance = singleInstance;
 }

 public OnScreenKeyModifierType ModifierType { get; }
 public bool SingleInstance { get; }
 public bool Clear { get; }
}

The big difference between the single instance class above and the toggle class is that the Update method checks if the SingleInstance property of the OnScreenKeyStateModifier before setting the IsChecked value to true. If the SingleInstance is set then the toggle class should not set its IsChecked property to true.

Support of ICommand

There is one hidden tidbit in the OnScreenKey abstract class and that is that it has a property that allows the key to be bound to an ICommand interface:

public string ClickCommand
{
 get { return _clickCommand; }
 set
 {
  _clickCommand = value;
  SetBinding(Button.CommandProperty, new Binding(_clickCommand)
  { RelativeSource = new RelativeSource(RelativeSourceMode.FindAncestor,
          typeof(OnScreenKeyboard), 1) });
 }
}
private string _clickCommand;

The property has to be set to the name of the DependencyProperty for the command that is specified OnScreenKeyboard code, and the propety has to be bound to a property in the XAML.

Binding has to be used for this since that is the only way to get the CanExecute to affect whether the key is enabled. There may be cases where the functionality is disabled, and it would be a lot more complex to add another property to enable the button.

Of course to make this work the OnScreenKeyboard class has to include a DependencyProperty for this ICommand and the command has to have a Binding assigned in the XAML.

Since the ClickCommand property is in the abstract OnScreenKey class, any key can have a command associated with it.

Using the Code

Below is the sample XAML for using the keyboard:

<keyboard:OnScreenKeyboard Height="300"
                           VerticalAlignment="Bottom"
                           ToggleButtonStyle="{StaticResource DefaultTouchToggleButtonStyle}"
                           ActiveContainer="{Binding ElementName=MainGrid}"
                           CancelCommand="{Binding EditorViewModel.CancelCommand}"
                           SaveCommand="{Binding EditorViewModel.SaveCommand}" />

The ToggleButtonStyle is the Style to be used for the keys, which are implemented using ToggleButtons. The most important binding is the ActiveContainer, which tells the keyboard which container to monitor for the GotKeyboardFocus event. If the UIElement that gets focus is a TextBoxBase or a PasswordBox, then the keyboard can be functional.

The Sample

History

• 02/26/2016: Initial version with minimal implementation detail
• 03/03/2016: Cleanup
• 03/04/2016: New code with ICommand keys implemented
• 03/05/2016: Code updates
• 03/09/2016: Updates and cleanup
• 03/11/2016: Minor code cleanup