• Download source code - 17.2 KB

Introduction

Sometimes you forced to do strange things. In this case I faced a task of implementing support of TabTip (Virtual Keyboard) in Silverlight in Windows 8/8.1. I found almost nothing on this subject, so I implemented my own solution for this problem and in this article I will describe challenges this task presented and my way of dealing with them. 

Prerequisite

• Elevated-trust.

In this article I will actively use COM automation and some other capabilities granted to us by elevated-trust. It doesn’t really matter if it will be in-browser or out-of-browser (I personally enabled it in-browser), but without elevated-trust working with TabTip is practically impossible. How to enable elevated-trust you can find in any serious book on Silverlight, or just applying your google skills, so I will not cover it here, but if you wonder it is pretty easy.

• Microsoft.CSharp

We need to add a reference to Microsoft.CSharp because to use COM automation we will need to use keyword dynamic.

• System.Windows.Interactivity

Main  logic will be encapsulated in custom behavior, for which we will need System.Windows.Interactivity reference.

Behavior

To implement code once and use it with every control that require TabTip I chose to do it using custom behavior. Class declaration look like this:

public class ControlTabTipBehavior : Behavior<Control>

As you should know, in behavior we need to override two methods, namely OnAttached and OnDetaching. 

protected override void OnAttached()
        {
            if (Application.Current.HasElevatedPermissions && AutomationFactory.IsAvailable)
            {
                AssociatedObject.LostFocus += AssociatedObject_LostFocus;
                AssociatedObject.GotFocus += AssociatedObject_GotFocus;
            }
        }

protected override void OnDetaching()
        {
            if (Application.Current.HasElevatedPermissions && AutomationFactory.IsAvailable)
            {
                base.OnDetaching();
                AssociatedObject.LostFocus -= AssociatedObject_LostFocus;
                AssociatedObject.GotFocus -= AssociatedObject_GotFocus;
            }
        }

Code will be changed in course of the article, but main thing you should gather from this methods is that things will happen in event handlers for GotFocus and LostFocus.

Open TabTip

We will be opening TabTip using COM automation and Shell.Application object’s ShellExecute method.

// This path is the same for both 32 and 64-bit installs of Windows
static string TabTipFilePath = @"C:\Program Files\Common Files\microsoft shared\ink\TabTip.exe";

static dynamic shellApplication = null;
static dynamic ShellApplication { get { return (shellApplication != null) ? shellApplication : shellApplication = GetShellApplication(); } }

private static dynamic GetShellApplication()
{
    return AutomationFactory.CreateObject("Shell.Application");
}

private static void OpenTabTip()
{
    ShellApplication.ShellExecute(TabTipFilePath, "", "", "open", 1);
}

Close TabTip

To close TabTip we will use some P/Invoke staff.

[DllImport("user32.dll")]
public static extern IntPtr FindWindow(String sClassName, String sAppName);

[return: MarshalAs(UnmanagedType.Bool)]
[DllImport("user32.dll", SetLastError = true)]
public static extern bool PostMessage(int hWnd, uint Msg, int wParam, int lParam);

private static void CloseTabTip()
{
    uint WM_SYSCOMMAND = 274;
    uint SC_CLOSE = 61536;
    IntPtr KeyboardWnd = FindWindow("IPTip_Main_Window", null);
    PostMessage(KeyboardWnd.ToInt32(), WM_SYSCOMMAND, (int)SC_CLOSE, 0);
}

Check If Keyboard Connected

As you could guess OpenTabTip will be called in GotFocus event handler, but should we open TabTip every time AssociatedObject gets focus? My guess is that user will need TabTip when hardware keyboard is not connected, otherwise it should remain closed. Hence the need for the next code:

static bool HardwareKeyboardConnected;

static dynamic wmiService = null;
static dynamic WMIService { get { return (wmiService != null) ? wmiService : wmiService = GetWMIService(); } }


private static dynamic GetWMIService()
{
    using (dynamic SWbemLocator = AutomationFactory.CreateObject("WbemScripting.SWbemLocator"))
    {
        SWbemLocator.Security_.ImpersonationLevel = 3;
        SWbemLocator.Security_.AuthenticationLevel = 4;
        return SWbemLocator.ConnectServer(".", @"root\cimv2");    
    }
}

private static void CheckIfHardwareKeyboardConnected()
{
    new Thread(() =>
    {
        dynamic keyboards = WMIService.ExecQuery(@"Select * from Win32_Keyboard");
        if (keyboards.Count() == 0)
            HardwareKeyboardConnected = false;
        else if (keyboards.Count() == 1) // Check for Tablets
        {
            foreach (dynamic keyboard in keyboards)
                HardwareKeyboardConnected = !(keyboard.Description == "Keyboard HID");
        }
        else
            HardwareKeyboardConnected = true;
    }).Start();
}

This code needs some explanation.

Firstly I start new Thread in CheckIfHardwareKeyboardConnected method. This is required because query for Win32_Keyboard will take some time to execute, and CheckIfHardwareKeyboardConnected will not be called only once. We will want to call it in AssociatedObject_GotFocus because user can connect keyboard at any time, as well as disconnect it. We will also call it once, when first control gets attached.

Secondly following code might raise some questions:

else if (keyboards.Count() == 1) // Check for Tablets
{
    foreach (dynamic keyboard in keyboards)
        HardwareKeyboardConnected = !(keyboard.Description == "Keyboard HID");
}

I found out that on windows tablets there is one keyboard that is always present. I don’t really know why that is, but we have to deal with it. Hence the need for that code. I distinct this keyboard by its description, but you might want to check if that description in your language is the same as I wrote it ("Keyboard HID").

Calculation and Animation

So now we can open TabTip when AssociatedObject gets focus and close it when focus is lost, pretty cool. But is it really enough? On tablets TabTip takes up half of the screen, and everything that is in lower half of it will be not visible. So what should we do? Calculate and Animate!  

Getting Visual Root For AssociatedObject

First thing first, we need to know, where AssociatedObject resides, namely is it in Application.Current.RootVisual or in some kind of child window.

public static DependencyObject GetVisualRoot(this DependencyObject element)
{
    DependencyObject RootCandidate = VisualTreeHelper.GetParent(element);
    if (RootCandidate != null)
        return RootCandidate.GetVisualRoot();
    else return element;
}

Calculating Y Offset For AssociatedObject

Now is the time to calculate where exactly our AssociatedObject is on Y axis, is it visible, and if not, by how much we will need to move visual root to make it visible.

static double RootVisualYOffset = 0;

private double GetNewRootVisualYOffsetForAssociatedObject()
{
    double NewRootVisualYOffset = RootVisualYOffset;
    double VisibleAreaTop = 0 - RootVisualYOffset;
    double VisibleAreaBottom = ((Application.Current.RootVisual as FrameworkElement).ActualHeight / 2) - RootVisualYOffset - 10;

    GeneralTransform gt = AssociatedObject.TransformToVisual(AssociatedObject.GetVisualRoot() as FrameworkElement);
    Point offset = gt.Transform(new Point(0, 0));
    double AssociatedObjectTop = offset.Y;
    double AssociatedObjectBottom = AssociatedObjectTop + AssociatedObject.ActualHeight;

    if ((AssociatedObjectBottom <= VisibleAreaBottom) && (AssociatedObjectTop >= VisibleAreaTop))
        return RootVisualYOffset;
    else if (AssociatedObjectBottom > VisibleAreaBottom)
    {
        double delta = (VisibleAreaBottom - AssociatedObjectBottom - 10);
        if (AssociatedObjectTop > (VisibleAreaTop - delta))
            NewRootVisualYOffset = RootVisualYOffset + delta;
        else
            NewRootVisualYOffset = RootVisualYOffset + (VisibleAreaTop - AssociatedObjectTop + 10);   
    }
    else if (AssociatedObjectTop < VisibleAreaTop)
        NewRootVisualYOffset = RootVisualYOffset + (VisibleAreaTop - AssociatedObjectTop + 10);
    return NewRootVisualYOffset;
}

Here we do some simple math, but there is some things I’d like to point out.

Calculations of VisibleAreaTop and VisibleAreaBottom done based on the assumption that browser is fullscreen and your application takes all available space. Also assumption is that this TabTip functionality is for tablets, which are relatively small, and on which TabTip takes half of the screen.

This assumptions work great in my case, but it is admittedly not an ideal solution. If you know how to determine what area of the application is really under the TabTip, please do write a comment or contact me any other way you find convenient. 

Animating

Now we need to actually move visual root to make users happy. So we will create a Storyboard, put it in VisualRoot.Resources dictionary and start the animation.

private static Storyboard GetMoveRootVisualStoryboard(FrameworkElement VisualRoot)
{
    if (VisualRoot.Resources.Contains("MoveRootVisualStoryboard"))
        return VisualRoot.Resources["MoveRootVisualStoryboard"] as Storyboard;
    else
    {
        Storyboard MoveRootVisualStoryboard = new Storyboard();
        MoveRootVisualStoryboard.Duration = new Duration(TimeSpan.FromSeconds(0.35));

        DoubleAnimation TranslateTransformAnimation = new DoubleAnimation();
        TranslateTransformAnimation.EasingFunction = new CircleEase { EasingMode = EasingMode.EaseOut };
        TranslateTransformAnimation.Duration = new Duration(TimeSpan.FromSeconds(0.35));

        MoveRootVisualStoryboard.Children.Add(TranslateTransformAnimation); 

        VisualRoot.RenderTransform = new TranslateTransform();
        Storyboard.SetTarget(TranslateTransformAnimation, VisualRoot);
        Storyboard.SetTargetProperty(TranslateTransformAnimation, new PropertyPath("(UIElement.RenderTransform).(TranslateTransform.Y)"));
        VisualRoot.Resources.Add("MoveRootVisualStoryboard", MoveRootVisualStoryboard);

        return MoveRootVisualStoryboard;
    }
}

private void MoveRootVisual(double To)
{
    Storyboard MoveRootVisualStoryboard = GetMoveRootVisualStoryboard(AssociatedObject.GetVisualRoot() as FrameworkElement);

    (MoveRootVisualStoryboard.Children.First() as DoubleAnimation).To = To;
    RootVisualYOffset = To;
    MoveRootVisualStoryboard.Begin();
}

Other Considerations

Main logic is there, but as always more things need to be done, to make user experience as enjoyable as we can.

Tracking Focus

In AssociatedObject_LostFocus we will call MoveRootVisual(0) to move layout to its original position, but what if focus was lost only because other AssociatedObject got it? In this case we will get TabTip moving quickly down and up, annoying our users. To avoid this we will write following:

static bool TextBoxFocused;

void AssociatedObject_LostFocus(object sender, RoutedEventArgs e)
{
    if (!HardwareKeyboardConnected)
    {
        TextBoxFocused = false;
        new Thread(() =>
            {
                // checking if another control got focus 
                Thread.Sleep(100);
                if (TextBoxFocused == false)
                {
                    CloseTabTip();
                    Dispatcher.BeginInvoke(() => MoveRootVisual(0));
                }
            }).Start();
    }
}

TextBoxFocused will be set to true in AssociatedObject_GotFocus. In AssociatedObject_LostFocus we wait 100 ms, and then check if any AssociatedObject is now focused.

TabTip Closed Event

Consider what will happen if AssociatedObject gets focus, TabTip opens, visual root moves, and after that user closes TabTip manually. AssociatedObject_LostFocus is not called, visual root stays where it was moved to.

So we need to know when TabTip is closed, even if it is done by user. To achieve that we will call StartTimerForKeyboardClosedEvent method in AssociatedObject_GotFocus. 

static Timer timer;

[DllImport("user32.dll", SetLastError = true)]
static extern UInt32 GetWindowLong(IntPtr hWnd, int nIndex);

public const int GWL_STYLE = -16;

public const uint KeyboardClosedStyle = 2617245696;

private void StartTimerForKeyboardClosedEvent()
{
    if (timer == null)
    {
        timer = new Timer((obj) =>
        {
            IntPtr KeyboardWnd = FindWindow("IPTip_Main_Window", null);
            if (KeyboardWnd.ToInt32() == 0 || GetWindowLong(KeyboardWnd, GWL_STYLE) == KeyboardClosedStyle)
            {
                Dispatcher.BeginInvoke(() => MoveRootVisual(0));
                timer.Dispose();
                timer = null;
            }
        }, null, 700, 50);
    }
}

Here we start timer and check if we can find TabTip window and if we can what is that windows style.

Check If TabTip Docked

TabTip can be docked and it is not so great for our purposes, because when TabTip is docked everything that is beneath it will not render. So when we move our visual root we will see some white space where our controls were supposed to be.

There is no really good way around this problem, but the least we can do, is warn our users about it.   

static bool tabTipDockedWarningShowed = false;

static dynamic wScriptShell = null;
static dynamic WScriptShell { get { return (wScriptShell != null) ? wScriptShell : wScriptShell = GetWScriptShell(); } }

private static dynamic GetWScriptShell()
{
    return AutomationFactory.CreateObject("WScript.Shell");
}

private void CheckIfTabTipDockedAndShowWarnng()
{
    if (tabTipDockedWarningShowed == false)
        new Thread(() =>
        {
            if (WScriptShell.RegRead(@"HKCU\Software\Microsoft\TabletTip\1.7\EdgeTargetDockedState") == 1)
            {
                tabTipDockedWarningShowed = true;
                Dispatcher.BeginInvoke(() => MessageBox.Show("It is not recomended to use virtual keyboard in docked state!"));
            }
        }).Start();
}

AssociatedObject_GotFocus

Now I can finally show code for AssociatedObject_GotFocus method.

void AssociatedObject_GotFocus(object sender, RoutedEventArgs e)
{
    AssociatedObject.Focus(); //to avoid losing focus

    CheckIfHardwareKeyboardConnected();

    if (!HardwareKeyboardConnected)
    {
        TextBoxFocused = true;

        CheckIfTabTipDockedAndShowWarnng();

        OpenTabTip();

        MoveRootVisual(GetNewRootVisualYOffsetForAssociatedObject());

        StartTimerForKeyboardClosedEvent();
    }
}

In this method only one line was not explained, and it is:

AssociatedObject.Focus(); //to avoid losing focus

It may look counterintuitive to set focus on control that just got focus, but when user clicks (or taps) on TextBox and he happens to click close to border of an element, focus will be received and then immediately lost.

To avoid this inconvenience we will reinforce focus with that line of code. 

Disposing COM Objects

As you may notice three COM objects was declared in course of this article with keyword static. This means that we would want to get rid of them only when this behavior will be detached from last AssociatedObject. To do that we will enhance our OnAttached and OnDetaching methods.

static int numberOfAttachedElements = 0;

protected override void OnAttached()
{
    if (Application.Current.HasElevatedPermissions && AutomationFactory.IsAvailable)
    {
        numberOfAttachedElements++;

        AssociatedObject.LostFocus += AssociatedObject_LostFocus;
        AssociatedObject.GotFocus += AssociatedObject_GotFocus;
    }
}

protected override void OnDetaching()
{
    if (Application.Current.HasElevatedPermissions && AutomationFactory.IsAvailable)
    {
        base.OnDetaching();
        AssociatedObject.LostFocus -= AssociatedObject_LostFocus;
        AssociatedObject.GotFocus -= AssociatedObject_GotFocus;

        numberOfAttachedElements--;

        if (numberOfAttachedElements == 0)
        {
            if (shellApplication != null)
                shellApplication.Dispose();

            if (wmiService != null)
                wmiService.Dispose();

            if (wScriptShell != null)
                wScriptShell.Dispose();
        }
    }
}

As you can see this was achieved with the use of numberOfAttachedElements static counter.

Attached Property

By now you can use that TabTip behavior simply attaching it to any control you want. But truth once been spoken by some woman, who wisely said: “ain't nobody got time for that!”. So what we want to do, is to create an attached property, which will attach our behavior for us. After that we can set this property once in style, and get this functionality throughout an application. This technique is widely known, so I will not give you code in the article, but it is in the attached source code. I will only write how can that one line of code with attached property look like in your style definition:

<Setter Property="Behaviours:ControlAttachedBehavior.ControlTabTipBehaviorEnabled" Value="True"/>