Approach to Implement Modal Windows for OpenGL/OpenTK in MONO/.NET for Serious Applications

Introduction

I want to discuss the options to prevent (or adjust) message processing for a main window, that is parent of a currently active modal window/dialog box. The options target to OpenTK on X11/Xlib.

This tip is a follow up of the tip Approach to Provide Modal UI Components (e.g. Dialogs) Without Blocking (and is based on its ComponentDispatcher/ DispatcherFrame approach).

Background

The typical expectations are (shaped by experience with Microsoft Windows) that a currently active modal window/dialog box shall prevent processing some of its parent window messages, e.g.:

• Obtaining the input focus (mouse, keyboard, gestures, ...)
• Hover effects
• Popups (menus, dropdowsn, tips/ballons, ...)
• Command invocation (button click, keyboard shortcut, ...)
• Maximize, minimize/iconify

And the currently active modal window/dialog box shall allow processing some other of its parent messages, e.g.:

• Move and resize, redraw, refresh
• Timer, network stack, device connection, ...
• Potential scrolling

While legacy Win32, MFC, Windows Forms and WPF (even today) use the Win32 API window style WS_DISABLED and/or the function EnableWindow(hwndParent, FALSE) / EnableWindow(hwndParent, TRUE), there is no ideal solution for X11/Xlib to disable a parent window as long as a modal child window/dialog box is active.

There are several recommendations to find across the Xlib documentation and programmer community forums:

1. Use XSetTransientForHint() to set the parent window's WM_TRANSIENT_FOR property and use a TransientShell for the modal child window/dialog box.
2. Set the modal child window/dialog box property _NET_WM_STATE_MODAL.
3. Set the modal child window/dialog box property _NET_WM_STATE_ABOVE property.
4. Set the modal child window/dialog box property _NET_WM_STATE_STAYS_ON_TOP (reported to be available for KDE only).

I've tested these recommendations on different platforms:

1. Open SUSE 11.3 32 bit edition; kernel Linux linux 2.6.34.7-0.5-desktop; X11 server X.Org 7.5_1.8.0-10.3.1
2. Open SUSE 12.3 64 bit edition; kernel linux-nscs.site 3.7.10-1.45-desktop; X11 server X.Org 7.6_1.13.2-1.29.1
3. Open SUSE LEAP 42 64 bit edition; kernel linux 4.1.20-11-default; X11 server X.Org 7.6_1.17.2-24.2

The results are unsatisfactory, especially the suppression of mouse/gesture input focus and command invocation (button click, keyboard shortcut, ...) for the parent window is not realized by any of the window properties.

Qt (KDE) solves this problem with another message loop (blocking the previously active one) for modal GUI classes (e. g. QDialog), that is invoked by the inherited method exec().

GTK+ solves this problem with another message loop as well (blocking the previously active one) for modal GUI classes (e.g. GtkDialog), that is invoked by the method gtk_dialog_run(). The gtk_dialog_run() method invokes g_main_loop_new() (see file gtkdialog.c).

Using the Code

Since a 100% blocking is not my objective (move and resize, redraw, refresh; timer, network stack, device connection, ...; potential scrolling shall be possible for the parent window) and modifying the OpenTK NativeWindow's ProcessEvents() method is not supported, I've chosen an approach that includes a filter into the event handler callbacks. My event handler callbacks are:

{
    ...

    _glWindow = new OpenTK.NativeWindow ();
    _context  = new GraphicsContext (_glMode, _glWindow.WindowInfo);

    ...

    _glWindow.Move              += HandleGlWindowMove;
    _glWindow.Resize            += HandleGlWindowResize;
    _glWindow.MouseMove         += HandleGlWindowMouseMove;
    _glWindow.MouseDown         += HandleGlWindowMouseDown;
    _glWindow.MouseUp           += HandleGlWindowMouseUp;
    _glWindow.MouseEnter        += HandleGlWindowMouseEnter;
    _glWindow.MouseLeave        += HandleGlWindowMouseLeave;

    _glWindow.FocusedChanged    += HandleGlWindowFocusedChanged;
    _glWindow.Closing           += WmCloseFromOpenTk;
    _glWindow.Closed            += WmDestroy;
    _glWindow.FreeLastRef       += FreeLastRefFromSourceTargetHandler;

    ...

    _context.MakeCurrent(_glWindow.WindowInfo);
    _glWindow.ProcessEvents ();

    ...
}

All my windows (no matter whether modal or not) share the same base class and process their own message loop. To prevent errors in an application with multiple OpenTK windows, the MakeCurrent() method shall be called before any other OpenTK calls are made to a window.

Modal windows are always children and all child windows register themselves to their parent window (managed within the OwnedWindowsInternal collection).

/// <summary>Handle the OpenGL NativeWindow MouseDown event.</summary>
/// <param name="sender">The control, the MouseDown event is assigned to.</param>
/// <param name="e">The event data.
/// <see cref="OpenTK.Input.MouseButtonEventArgs"/></param>
void HandleGlWindowMouseUp (object sender, OpenTK.Input.MouseButtonEventArgs e)
{
    // ######################################## For X11: ########################################
    // 1. Neither Xlib.XSetTransientForHint() nor Xlib.XChangeProperty() with _NET_WM_STATE_MODAL
    //    nor _NET_WM_STATE_ABOVE prevent message processing for the owner window, if window
    //    shall have modal child window/dialog box.
    // 2. GTK solves this with modified message loop -see "gtkdialog.c" method gtk_dialog_run().
    // 3. OpenTK's message loop NativeWindow.ProcessEvents() is hidden and can't be modified.
    //    Consequently the message handler are to be modified instead, to suppress unwanted event
    // processing.
    // ######################################## For X11: ########################################
    // In case of the component frame dispatcher is set to run in modal mode and at least one
    // child window exists, this is the parent window of a modal window/dialog box.
    if (ComponentDispatcher.CurrentDispatcher.IsThreadModal && this.OwnedWindowsInternal.Count>0)
        return;
        
    ...
}

This approach offers a very fine adjustment for command invocation (button click, keyboard shortcut, ...) to be suppressed or not.

Points of Interest

I was looking for an easy and flexible solution...

History

• 016/06/03: First version