Using Asynchronous Procedure Calls (APC) in Windows based C++ applications to perform GUI Update

• Download MFCAPCTest_-_exe.zip - 2.1 MB
• Download MFC_APC_Test.zip - 133.2 KB

Introduction

When you create Window based application in C++ (using raw win32 APIs or MFC for example) and your application performs non-trivial work, you quickly run into the problem that the work affects the responsiveness of the application. If the click of a button results in 1 second of processing being done inside the handler for the button click event, that means the entire application will not be able to respond to other events during that 1 second.

Clearly, this goes against the idea of good user interface design because it feels slow and unprofessional. People will start to think something is wrong, and may decide to kill your application. The solution is to offload the work to a background thread which can do the work while the user interface window can respond to new events such as system messages or user interaction.

That solves the 'time to perform the work' problem but introduces a new 'communicating with the user' problem. After the work is being done, or while the work is being done, the worker thread needs to be able to interact with the user interface. This could be to display results, an intermediate status, or to update the accessibility of the user interface itself. But one of the prime restrictions of user interface interactions is that under no circumstance is it allowed for a worker thread to touch user interface elements that are managed by the user interface thread. Doing so will cause bad things to happen such as crashes or data corruption.

What we need is a reliable, simple way to implement a way for worker threads to send updates to the user interface. As with many such problems, there is no '1' right solution. However, APCs (Asynchronous Procedure Calls) have a lot of benefits, so in this article I explore those. Note that this is a follow up on my first article about APCs. In that article I explained how they work and how to use them. In this article, I show a practical application.

Background

In real-world applications, it is typical for the end user to start an action with the click of a button, and then wait for / while the results come in. This can be anything. A machine manipulation, measurement sequence on some connected instrument, a complex database query, etc. This is something you'd typically send to another thread as a command, together with a set of data (if applicable) that serves as input data.

The worker thread can then take as much time as it needs to process the command, and send an update back when it's done. As a matter of convenience, we implement the dispatch of commands to the worker thread with APCs as well, simply because it's so elegant and convenient, and takes the pain of thread safety queueing out of our hands.

So let's get started

The threading code

We need a background worker thread so let's implement that first.

DWORD WorkerThreadFunc(void* context)
{
    HANDLE shutdown = (HANDLE)context;
    DWORD retVal = 0;

    while (retVal = WaitForSingleObjectEx(shutdown, INFINITE, TRUE) == WAIT_IO_COMPLETION)
        ;
    return retVal;
}

As you can see, this code could not be simpler because there isn't any. Remember from the previous article that Windows queues our work request internally, and executes it in place of whatever is going on in that thread. By using WaitForSingleObjectEx, we essentially have a thread that does absolutely nothing but wait for something to do. The 'Ex' in the function name indicates that we allow Windows to interrupt the wait.

The added beauty of this mechanism is not only that we don't have to write any of the scheduling code ourselves, but the worker thread doesn't have to know in advance what it will be doing, because we can schedule whatever we want on that thread.

The asynchronous task data

We've mentioned that we will be sending work requests to the background thread, and getting some kind of response back.

struct TaskData
{
    HANDLE hCaller;
    CMFCTestDlg* sourceDialog;
    FLOAT Value;
};

struct TaskResponse
{
    CMFCTestDlg* targetDialog;
    CString Value;
};

For the same of this example, we're going to send a floating point number to the background worker to do something with. The purpose of the hCaller parameter is obvious. If we want to use an APC to schedule a work item on the original user interface thread, we have to know which thread to send it to. The final parameter is a pointer to the window object where the update must be processed.

At first you may think: Why do we a pointer to the window when we already know where the return APC has to go (via hCaller). There are 2 reasons. First, from a technical pov, APC functions cannot be member functions. they are global functions (not bound to a specific object). So unless we use global variables in our code, the APC function has no way to know on which window object it needs to update the information.

But secondly, a Windows application such as an MDI (Multiple Document Interface) may have several active windows at the same time and in that case, neither worker thread nor the user interface thread would know the origin / destination window.

The task worker itself

The task worker is the function which gets executed on the background thread.

void PerformTask(void* context)
{
    TaskData* data = (TaskData*)context;
    TaskResponse* response = new TaskResponse;
    Sleep(500);
    response->targetDialog = data->sourceDialog;
    response->Value.Format(L"Processed APC with value %f\r\n", data->Value);
    if(QueueUserAPC(
        (PAPCFUNC)&ReportBack,
        data->hCaller,
        (ULONG_PTR)response))
    {
        //do error handling here
        delete response;
    }
    delete data;
}

The first part is easy enough to understand. We get the input data and create a response message. The 500 ms delay is meant to simulate work being done, after which the response data is filled in. Not that under no circumstance (*) are you allowed to do anything with the pointer to the window object that is passed back and forth. This would violate the premise that window objects get touched from another thread, and could cause data corruption or crashes.

When the response message is ready, it gets posted back to the original thread, together where the window object can be used alongside the task results to perform user interface updates. Note that this is a reasonably simple example. It is perfectly valid to have multiple possible functions for reporting back. In general this can lead to cleaner code.

For example, suppose an error occurred, then there is a use case for a ReportError function which could perform actions that only need to be performed when an error occurred, such as turning a status bar red. Having multiple functions to do things in specific cases removed the need for the ReportBack to implement complex processing to handle all cases.

(*) Technically, the restriction is only against touching anything 'window' related such as e.g. the contents of a text box or a background color, or any thread-unsafe data which may be touched by the user interface thread such as a CString object. In theory it is perfectly fine to access a file handle or a bool member variable because those don't affect the window infrastructure in any way. However, even in that case, I strongly argue against such things. Not only is it too easy to misjudge things, but you still have to deal with the 'normal' multithreading pitfalls such as -possibly- using a file handle in 2 different threads at the same time.

Most multi thread problems can be avoided by making shortcuts out of convenience. If something is needed in the worker thread, send it along with the task in a way that confers ownership. And when the task is done, it closes the resources or passes ownership back.

The task response

For the purpose of our example, the response is simple. There is a status window on screen, and we add the result of the latest task execution to that window. There isn't a whole lot to say here.

void ReportBack(void* context)
{
    TaskResponse* response = (TaskResponse*)context;
    response->targetDialog->logText = response->Value + response->targetDialog->logText;
    response->targetDialog->txtApcResult.SetWindowTextW(
        response->targetDialog->logText);
    delete response;
}

APC Execution

We use APCs for 2 things: offloading work to the background worker thread, and handling responses on the user interface thread.

Scheduling the work

If you read my first APC article, you'll see that this part is very similar. We have 2 buttons for scheduling work. The first schedules 1 work item, the other schedules 5.

void CMFCTestDlg::OnBnClickedStart()
{
    auto data = new TaskData;
    data->hCaller = hGuiThread;
    data->Value = fVal++;
    data->sourceDialog = this;
    if(QueueUserAPC((PAPCFUNC)&PerformTask, hWorkerThread, (ULONG_PTR)data))
    {
        //do error handling here
        delete data;
    }
}


void CMFCTestDlg::OnBnClickedStart5()
{
    for (int i = 0; i < 5; i++) {
        auto data = new TaskData;
        data->hCaller = hGuiThread;
        data->Value = fVal++;
        data->sourceDialog = this;
        if(QueueUserAPC((PAPCFUNC)&PerformTask, hWorkerThread, (ULONG_PTR)data))
        {
            //do error handling here
​​​​​​​            delete data;
        }
    }
}

Both options are very similar. We create a new task input data structure, populate the data, and then queue the worker function on the background thread, together with the input data.

Handling responses

With the discussion about task execution out of the way, we need to circle back to the beginning. Remember when we said that APCs only get executed when the target thread tells Windows that it is ready to be interrupted by performing a so called 'interruptible wait'.

In the worker thread this is easy enough because we can use WaitForSingleObjectEx to keep listening for incoming APCs. However, in the GUI thread, we have no such luck. Window applications are driven by a message pump, internally.

For reference, message loops typically look something like this:

while( (bRet = GetMessage( &msg, hWnd, 0, 0 )) != 0)
{ 
    if (bRet == -1)
    {
        // handle the error and possibly exit
    }
    else
    {
        TranslateMessage(&msg); 
        DispatchMessage(&msg); 
    }
}

The details are beyond the scope of this article, but suffice it to say that the application can get messages from the system itself and from user interactions. Every message gets handled swiftly, after which the application goes back to waiting. At first glance, there is no proper way for the GUI thread to put itself in an alertable state.

For GUI applications that are programmed in raw win32, we can replace GetMessage with MsgWaitForMultipleObjectsEx. For more details you can read the documentation but essentially, it provides an alertable equivalent to GetMessage.

These days it is rare / unheard of to program new applications in raw win32 so that chances that you'll ever do things this way are slim. It is much more likely you'll be using MFC or a comparable class library which hides the message loop deep underneath layers and layers of object hierarchy, where it is not meant to be touched. You could of course dig through all those layers and make a lot of changes, but that's not really something you should do. It's not necessary, and it negates a lot of the benefits in using standard frameworks.

Instead, we're going to cheat.

Enabling APC processing in the GUI thread

When you think about it, our only requirements are to not block the user interface thread by doing work on it, and processing the task responses in the user interface thread at some point. It doesn't say we have to do it instantly. When you take a step back and think about it, nothing in the user interface requires Realtime responses. A response time of 0 to 250 ms is plenty good enough, as long as the application itself is free to interact during that time.

We do this with a timer. Every window class that derives from CWnd has a SetTimer method that causes a WM_TIMER event to be fired on the message pump of the window. As a result, every nElapse milliseconds, the method which handles that event will be executed.

void CMFCTestDlg::OnTimer(UINT_PTR nIDEvent)
{
    SleepEx(0, TRUE);
    CDialogEx::OnTimer(nIDEvent);
}

And in the handler for that message, all we have to do is perform an alertable sleep of 0 ms. This will notify the system that the user interface thread is in a position to execute whatever APCs there may be for it, if any.

It is correct to say that this is not a perfect solution. After all, we need to process a timer message, a couple of times per second. Even if the amount of processing is trivial, it is not a perfect solution like MsgWaitForMultipleObjectsEx. But it's as close to it as we can get without having to rip the guts out of the MFC window handling. 

There are still an optimization you can do, if it should be a concern. It only makes sense to process APCs if you are expecting any. If there is no background processing being done, you can simply disable the timer!

Now in my case, for the sake of the example, I opted to implement a radio button with which you can switch between manually firing off APC processing and automatically processing them. In MFC, radio buttons generate individual events so this is what is implemented.

void CMFCTestDlg::OnBnClickedApcauto()
{
    rbnAuto.SetCheck(1);
    rbnMan.SetCheck(0);
    btnProcessAPC.EnableWindow(0);
    SetTimer(1, 250, NULL);
}

void CMFCTestDlg::OnBnClickedApcman()
{
    rbnAuto.SetCheck(0);
    rbnMan.SetCheck(1);
    btnProcessAPC.EnableWindow(1);
    KillTimer(1);
}

If we opt for automatic processing, we enable the timer with a 250 ms period, and disable the button for manually processing them. If we opt for manual processing, we kill the timer and enable the button to manually start APC processing. Manual processing is trivial:

void CMFCTestDlg::OnBnClickedButton2()
{
    SleepEx(0, TRUE);
}

From these examples, it is equally easy to see that you start the timer whenever you push work towards the background thread, and disable it when the work is done being processed by the GUI. Of course, you have to then account for the fact that there can be multiple work items, and you only want to disable the timer if ALL work is done.

The good news is that both starting the work, and handling the work response is done inside the GUI thread, and is therefore synchronized. You could simply use a member variable to increment and decrement the number of open tasks without needing to worry about syncrhonization.

Using the test application

The test application is simple

You can either start 1 or 5 tasks at once, and then decide if you want to process them manually or automatically. That's all there is to it!

Points of Interest

It feels like we had to write very little code to implement a robust worker thread solution with robust and thread safe work scheduling. And that's true. If you use APCs, Windows does the scheduling, context switching and work ordering for you. And by using them for handling the task responses in the user interface thread, you comply with Windows requirements, avoid crashes / data corruption, and don't have to worry about race conditions because all response handling is neatly in order, at a time when the user interface itself cannot do other things.

I did play with the idea of building an example application in raw win32 code in order to demonstrate MsgWaitForMultipleObjectsEx. I decided to go with MFC instead because if you're a C++ developer and working on window based applications, chances are that you use MFC or a similar framework. It's fairly safe that few if any projects these days are started in win32. 

I'd also like to note that in order to keep things simple, I did not implement error handling for the new operator or checking for NULL pointers. It goes without saying the in production code, you should always implement input validation, range checking, pointer validation, etc.

History

08DEC2023: 1st version.

19DEC2023: 2nd version. Added handling of QueueUserAPC error.