Learn Windows Workflow Foundation 4.5 through Unit Testing: Persistence with WorkflowApplication

Background

I had started to learn Windows Workflow Foundation sometime ago. I prefer to learn a major technology framework through systematic study rather then googling around. However, I found that most well written books and articles were published between 2006-2009, so outdated, particularly missing new features in .NET 4 and 4.5; and a few books published in recent years for WF 4.0 and 4.5 were poorly written. While I generally prefer systematic, dry and abstract study, this time I would make up some wet materials for studying.

Introduction

Supporting long running process is one of the things that make WF shine. And technically to support long running process against all odds, persisting the workflow is a key. For persistence, WF 3.5, 4.0 and 4.5 have quite significant differences. Googling around may return mix information, outdate ones and updated ones, quite confusing. In this article, I would like to present some unit testing cases for demonstrating the behaviors of persistence.

This is the 5th article in the series. And source code is available at https://github.com/zijianhuang/WorkflowDemo

Other articles in this series are given below:

• Learn Windows Workflow Foundation 4.5 through Unit Testing: CodeActivity
• Learn Windows Workflow Foundation 4.5 through Unit Testing: InvokeMethod and DynamicActivity
• Learn Windows Workflow Foundation 4.5 through Unit Testing: WorkflowApplication and WorkflowServiceHost

And this article about persistence is utilizing WorkflowApplication and WorkflowServiceHost since both have full access to WF runtime, particularly the persistence.

Using the Code

The source code is available at https://github.com/zijianhuang/WorkflowDemo.

Prerequsites:

1. Visual Studio 2015 Update 1 or Visual Studio 2013 Update 3
2. xUnit (included)
3. EssentialDiagnostics (included)
4. Workflow Persistence SQL database, with default local database WF.

Examples in this article are from a test class: WorkflowApplicationPersistenceTests.

WorkflowApplication with persistable workflow

Bookmark

public sealed class ReadLine : NativeActivity<string>
{
    public ReadLine()
    {
    }

    public InArgument<string> BookmarkName { get; set; }

    protected override bool CanInduceIdle
    {
        get
        {
            return true;
        }
    }

    protected override void Execute(NativeActivityContext context)
    {
        string name = this.BookmarkName.Get(context);

        if (name == null)
        {
            throw new ArgumentException(string.Format("ReadLine {0}: BookmarkName cannot be null", this.DisplayName), "BookmarkName");
        }

        context.CreateBookmark(name, new BookmarkCallback(OnReadComplete));
    }

    void OnReadComplete(NativeActivityContext context, Bookmark bookmark, object state)
    {
        string input = state as string;

        if (input == null)
        {
            throw new ArgumentException(string.Format("ReadLine {0}: ReadLine must be resumed with a non-null string"), "state");
        }

        context.SetValue(base.Result, input);
    }
}


    [Fact]
    public void TestPersistenceWithBookmark()
    {
        var x = 100;
        var y = 200;
        var t1 = new Variable<int>("t1");

        var plus = new Plus()
        {
            X = x,
            Y = y,
            Z = t1,  //So Output Z will be assigned to t1
        };
        var bookmarkName = NewBookmarkName();
        var a = new System.Activities.Statements.Sequence()
        {
            Variables =
                    {
                        t1
                    },
            Activities = {
                        new Multiply()
                        {
                            X=3, Y=7,
                        },

                        new ReadLine()
                        {
                            BookmarkName=bookmarkName,
                        },

                        plus,

                    },
        };

        bool completed1 = false;
        bool unloaded1 = false;
        bool isIdle = false;

        AutoResetEvent syncEvent = new AutoResetEvent(false);

        var app = new WorkflowApplication(a);
        app.InstanceStore = WFDefinitionStore.Instance.Store;
        app.PersistableIdle = (eventArgs) =>
        {
            return PersistableIdleAction.Unload;//so persist and unload
        };

        app.OnUnhandledException = (e) =>
        {

            return UnhandledExceptionAction.Abort;
        };

        app.Completed = delegate (WorkflowApplicationCompletedEventArgs e)
        {
            unloaded1 = true;

        };

        app.Aborted = (eventArgs) =>
        {

        };

        app.Unloaded = (eventArgs) =>
        {
            unloaded1 = true;
            syncEvent.Set();
        };

        app.Idle = e =>
        {
            Assert.Equal(1, e.Bookmarks.Count);
            isIdle = true;
        };

        //  app.Persist();//This is optional, since Workflow runtime will persist when the execution reach to ReadLine.
        var id = app.Id;
        app.Run();
        syncEvent.WaitOne();

        Assert.False(completed1);
        Assert.True(unloaded1);
        Assert.True(isIdle);
        //At this point, DB WF/InstancesTable has a new record, and the value of column BlockingBookmark contains the bookmarkName

        //Now to use a new WorkflowApplication to load the persisted instance.
        LoadWithBookmarkAndComplete(a, id, bookmarkName, "abc");
        //The record is now deleted by WF runtime.
    }

    static IDictionary<string, object> LoadWithBookmarkAndComplete(Activity workflowDefinition, Guid instanceId, string bookmarkName, string bookmarkValue)
    {
        bool completed2 = false;
        bool unloaded2 = false;
        AutoResetEvent syncEvent = new AutoResetEvent(false);
        IDictionary<string, object> outputs = null;

        var app2 = new WorkflowApplication(workflowDefinition)
        {
            Completed = e =>
            {
                if (e.CompletionState == ActivityInstanceState.Closed)
                {
                    outputs = e.Outputs;
                }
                completed2 = true;
            },

            Unloaded = e =>
            {
                unloaded2 = true;
                syncEvent.Set();
            },

            InstanceStore = WFDefinitionStore.Instance.Store,
        };

        app2.Load(instanceId);
        var br = app2.ResumeBookmark(bookmarkName, bookmarkValue);
        Assert.Equal(BookmarkResumptionResult.Success, br);

        syncEvent.WaitOne();

        Assert.True(completed2);
        Assert.True(unloaded2);

        return outputs;
    }

It is optional to explicitly call app.Persist(), since the WF runtime will persist the workflow when reaching activity ReadLine. And the workflow is unloaded and not completed, before running LoadAndComplete. Then another WorkflowApplication in this example is created, picking up the instance stored, and resume with the bookmark and the data expected.

Remarks:

In the case, you see the 2nd run of the workflow gets the workflowDefinition defined before the first run. In real world scenarios, sometimes it is not nice to have the definition hanging around in the application for the whole lifecycle. It is better to persist the definition somewhere, so the 2nd run just needs the instanceId and bookmarkName in order to resume from last breakpoint. In WF3.5, WF did support natively persisting the definition, however, this had been removed from WF4 and 4.5. So it is up to the application developers to persist the definitions.

Persist workflow definitions

When you are using Workflow Designer of Visual Studio, you probably notice that the custom made workflow is an XAML file which is transformed to C# codes at design time. And the C# codes is compiled when you build projects. And in WF, there's a function to transform a workflow definition in memory into XAML. So XAML is a natural way of persisting and reloading workflows.

Some helper functions to serialize and deserialize a workflow:

/// <summary>
 /// Persist activity for purpose of resuming later.
 /// </summary>
 /// <remarks>The functions here are for bookmark and long running only, not for general purpose.</remarks>
 /// <example>inspired by https://msdn.microsoft.com/en-us/library/ff458319%28v=vs.110%29.aspx</example>
 public static class ActivityPersistenceHelper
 {
     /// <summary>
     ///
     /// </summary>
     /// <param name="activity"></param>
     /// <param name="stream">External XML UTF8 stream. Caller is responsible to set position if the stream supports random seek, and to dispose.</param>
     public static void SaveActivity(Activity activity, Stream stream)
     {
         using (var streamWriter = new StreamWriter(stream, Encoding.UTF8, 512, true))
         using (var xw = ActivityXamlServices.CreateBuilderWriter(new System.Xaml.XamlXmlWriter(streamWriter, new System.Xaml.XamlSchemaContext())))
         {
             System.Xaml.XamlServices.Save(xw, activity);
         }
     }

     /// <summary>
     ///
     /// </summary>
     /// <param name="stream">XAML stream defining an Activity</param>
     /// <returns></returns>
     public static Activity LoadActivity(Stream stream)
     {
         var settings = new ActivityXamlServicesSettings()
         {
             CompileExpressions = true,
         };

         var activity = ActivityXamlServices.Load(stream, settings);
         return activity;
     }

     public static Activity LoadActivity(byte[] bytes)
     {
         var settings = new ActivityXamlServicesSettings()
         {
             CompileExpressions = true,
         };

         using (var stream = new MemoryStream(bytes))
         {
             var activity = ActivityXamlServices.Load(stream, settings);
             return activity;
         }
     }

Dictionary to map instance ID with workflow definition:

public class WFDefinitionStore
{
    private static readonly Lazy<WFDefinitionStore> lazy = new Lazy<WFDefinitionStore>(() => new WFDefinitionStore());

    public static WFDefinitionStore Instance { get { return lazy.Value; } }

    public WFDefinitionStore()
    {
        InstanceDefinitions = new System.Collections.Concurrent.ConcurrentDictionary<Guid, byte[]>();

        Store = new SqlWorkflowInstanceStore("Server =localhost; Initial Catalog = WF; Integrated Security = SSPI")
        {
            InstanceCompletionAction = InstanceCompletionAction.DeleteAll,
            InstanceEncodingOption = InstanceEncodingOption.GZip,

        };
    }

    public System.Collections.Concurrent.ConcurrentDictionary<Guid, byte[]> InstanceDefinitions { get; private set; }

Hints:

In real world applications, you may have other structures to persist the workflow definitions according to your technical requirements. Also, in a service app, there might be multiple instances of the same workflow definition, thus it is inefficient to store multiple copies of the same definition, so you may be designing a storage either in program logic or persistence that will store a definition once and support versioning. This dictionary is good enough for demo and small applications.

Workflow with Delay

This test case demonstrates:

• The basic behaviors of WorkflowApplication
• The performance of persistence
• Workflow definition persistence using the demo helper class above
• How to obtain result / OutArgument from the workflow

[Fact]
public void TestPersistenceWithDelayAndResult()
{
    var a = new Fonlow.Activities.Calculation();
    a.XX = 3;
    a.YY = 7;

    bool completed1 = false;
    bool unloaded1 = false;

    AutoResetEvent syncEvent = new AutoResetEvent(false);

    var app = new WorkflowApplication(a);
    app.InstanceStore = WFDefinitionStore.Instance.Store;
    app.PersistableIdle = (eventArgs) =>
    {
        return PersistableIdleAction.Unload;//so persist and unload
    };

    app.OnUnhandledException = (e) =>
    {
        Assert.True(false);
        return UnhandledExceptionAction.Abort;
    };

    app.Completed = delegate (WorkflowApplicationCompletedEventArgs e)
    {
        completed1 = true;
        Assert.True(false);
    };

    app.Aborted = (eventArgs) =>
    {
        Assert.True(false);
    };

    app.Unloaded = (eventArgs) =>
    {
        unloaded1 = true;
        syncEvent.Set();
    };

    var id = app.Id;
    stopwatch.Restart();
    stopwatch2.Restart();
    app.Run();
    syncEvent.WaitOne();

    stopwatch.Stop();
    Assert.True(stopwatch.ElapsedMilliseconds < 2500, String.Format("The first one is executed for {0} milliseconds", stopwatch.ElapsedMilliseconds));
    //the ellipsed time depends on the performance of the WF runtime when handling persistence. The first case of persistence is slow.

    Assert.False(completed1);
    Assert.True(unloaded1);

    stopwatch.Restart();
    var t = WFDefinitionStore.Instance.TryAdd(id, a);
    stopwatch.Stop();
    Trace.TraceInformation("It took {0} seconds to persist definition", stopwatch.Elapsed.TotalSeconds);

    //Now to use a new WorkflowApplication to load the persisted instance.
    LoadAndCompleteLongRunning(id);
}

void LoadAndCompleteLongRunning(Guid instanceId)
{
    bool completed2 = false;
    bool unloaded2 = false;
    AutoResetEvent syncEvent = new AutoResetEvent(false);

    var app2 = new WorkflowApplication(WFDefinitionStore.Instance[instanceId])
    {
        Completed = e =>
        {
            completed2 = true;
            var finalResult = (long)e.Outputs["Result"];
            Assert.Equal(21, finalResult);
        },

        Unloaded = e =>
        {
            unloaded2 = true;
            syncEvent.Set();
        },

        InstanceStore = WFDefinitionStore.Instance.Store,
    };

    stopwatch.Restart();
    app2.Load(instanceId);
    Trace.TraceInformation("It took {0} seconds to load workflow", stopwatch.Elapsed.TotalSeconds);

    app2.Run();
    syncEvent.WaitOne();
    stopwatch2.Stop();
    var seconds = stopwatch2.Elapsed.TotalSeconds;
    Assert.True(seconds > 3, String.Format("Activity execute for {0} seconds", seconds));//But if the long running process is fired and forgot, the late load and run may be completed immediately.

    Assert.True(completed2);
    Assert.True(unloaded2);

}

Remarks:

• The first instance of persistence is running slow, probably due to some warm-up in WF runtime or SQL connection. Please leave a comment if you know why.
• After a long running workflow persistable is persisted and unloaded, a house keeper program is needed to wake it up through checking the persistence layer regularly. This will be another big subject to be discussed in next articles.

Invalid Store Throws Exception

[Fact]
public void TestPersistWithWrongStoreThrows()
{
    var a = new Multiply()
    {
        X = 3,
        Y = 7,
    };

    AutoResetEvent syncEvent = new AutoResetEvent(false);

    var app = new WorkflowApplication(a);
    app.InstanceStore = new SqlWorkflowInstanceStore("Server =localhost; Initial Catalog = WFXXX; Integrated Security = SSPI");
    app.PersistableIdle = (eventArgs) =>
    {
        Assert.True(false, "quick action no need to persist");//lazy
        return PersistableIdleAction.Persist;
    };

    //None of the handlers should be running
    app.OnUnhandledException = (e) =>
    {
        Assert.True(false);
        return UnhandledExceptionAction.Abort;
    };

    var ex = Assert.Throws<System.Runtime.DurableInstancing.InstancePersistenceCommandException>
       (() => app.Persist(TimeSpan.FromSeconds(2)));

    Assert.NotNull(ex.InnerException);
    Assert.Equal(typeof(TimeoutException), ex.InnerException.GetType());
}

There could be many reasons why the instance store has problems, so your application needs to handle this exception nicely.