Scheduling Tasks with Revalee and MVC

• Download source from GitHub
• Download source - 1.9 MB

Introduction

So you're almost finished with the user registration portion of your MVC project, but you need to be able to send an email to each user 3 days before their 30-day trial plans are about to expire. Now what?

Well, you could create an out-of-band process that batches up expiring user accounts nightly and sends them an email message at 4:00 in the morning. But that has hassle written all over it. First, you have to create a job outside of your web application. Next, you have to schedule that job. Then, you have to handle any errors the overnight job generates and, of course, monitor the thing yourself especially when Ted (you know the guy who always reboots the server just as he's about to walk out the door) is in charge of that other server.

There is another way.

Use your web application. That's right: it already has access to the database, it can render & send outbound email messages (you already sent out a welcome message, right?), and it's part of your testing rig. Now before you freak out about having to keep your web application loaded for days straight or some other cockamamie scheme (aka. hack), I'll say it again: there is another way.

Background

The development group I work with has run into this issue so many times that we decided to write our own solution to this problem and make it an open source project. It's called Revalee—as in reveille: a signal to arise—and it is a Windows Service that freezes your web requests in carbonite (well, not really) until it's time to thaw them out. More technically, Revalee makes note of your original web requests and calls back your web application at a preset, future date and time.

Disclaimer: In case you missed it two sentences earlier, Revalee is a free, open source project written by the development team that I am a part of. It is available on GitHub and is covered by the MIT License. If you are interested, download it and check it out.

If you take a look at the GitHub project, you will see that project has three parts: Revalee.Service, Revalee.SampleSite, and Revalee.Client. The service is the Windows Service itself and it receives your original web requests and handles scheduling your future callback. The sample site is a mini-MVC project that illustrates how someone might use Revalee within their own MVC web application. (This article will attempt to cover this same topic.) Finally, the client part of the project is a client library that helps automate sending requests to the service.

First things first. Let's install the service.

Installing Revalee

Whether you download the GitHub project and compile the C# solution yourself or you simply download the precompiled files from the project website, installing Revalee is a simple affair. First, copy the following compiled files into your desired folder (say: C:\Program Files\Revalee\):

• Esent.Interop.dll
• License.txt
• Revalee.Service.exe
• Revalee.Service.exe.config

Next, install the Windows Service from a command prompt (with elevated Administrator rights naturally):

Revalee.Service.exe -install

That's it. You're done installing. One more thing: let's assume for the sake of this article that your server's IP Address is 172.31.46.200. (We'll need this later.)

As an aside, the Revalee service listens on port 46200 for scheduling requests. This is the service's default port number, but it can be changed to whatever you need it to be. However, keep port 46200 (or whatever port you've configured) in mind, especially if you have internal firewalls separating the server where Revalee is installed from your web server(s).

Using Revalee

Getting back to the problem at hand, first copy the Revalee.Client.dll file into your MVC project's bin folder. Next, add a reference to the Revalee.Client assembly to your MVC project.

There's also a Revalee.Client NuGet package to simplify this if you prefer that route.

As before, let's assume for the sake of this article that the controller, which will be handling your callback actions, is named ScheduledCallback. At the top of that controller in your MVC application, don't forget to add the namespace reference:

using Revalee.Client;

In your ScheduledCallback controller, create a new action called, say, SendExpirationMessage that had a single, integer parameter:

public ActionResult SendExpirationMessage(int userId)

This controller action will be what your future callback will request. Using the incoming userId, you will be able to lookup any information about the specified user from the database before you have to format your outbound expiration email message à la:

[AllowAnonymous]
[HttpPost]
public ActionResult SendExpirationMessage(int userId)
{
    // Validate the incoming request to ensure that this web app requested this callback
    if (!RevaleeRegistrar.ValidateCallback(this.Request))
    {
        // Return a '401 Unauthorized' response to the Revalee service if the callback doesn't validate
        return new HttpStatusCodeResult(HttpStatusCode.Unauthorized);
    }

    // Validate the incoming parameter
    if (userId <= 0)
    {
        // Either throw an exception, log an invalid (or malformed) request,
        //  or track something else appropriate for your application.
    }

    // Look up the user in your database.

    // Format the email message with user-specific information, like name, email address, etc.

    // Send the expiration email message to the user.

    // Return a '200 OK' response to the Revalee service after successfully processing the callback
    return new HttpStatusCodeResult(HttpStatusCode.OK);
}

Finally, create a helper method that you'll call when you need to schedule the future callback (27 days from now, in this case). This is the method that you'll call when the user registers on your site:

private void ScheduleExpirationMessage(int userId)
{
    // The server address where the Revalee service is installed
    string revaleeServiceHost = "172.31.46.200";

    // The callback will occur 27 days from now
    DateTimeOffset callbackTime = DateTimeOffset.Now.AddDays(27.0);

    // The url that will be called back, including userId
    Uri callbackUrl = new Uri(
        string.Format("http://mywebapp.com/ScheduledCallback/SendExpirationMessage/{0}", userId));

    // Register the callback request with the Revalee service
    RevaleeRegistrar.ScheduleCallback(revaleeServiceHost, callbackTime, callbackUrl);
}

In case you need to track it for your web application's own internal purposes, ScheduleCallback() returns a Guid. That way, you can reference the future callback using that unique identifier or simply cancel the callback (see below) if you need to.

And that's it. You've scheduled a future expiration email message without ever having to step outside of your Visual Studio development environment or your MVC application. Now that's cooler than Boba Fett. Well, almost.

So What's the Big Deal?

Wait, what? Oh, so you're like: I'm not using some open source crapware. Alright, let's roll up our sleeves and look under Revalee's hood a bit.

Revalee.Client

In the RevaleeRegistrar class, two methods highlight the simplicity of how you interact with the service:

private static string BuildScheduleRequestUrl(Uri serviceBaseUri, DateTime callbackUtcTime, Uri callbackUrl)
{
    return string.Format(
        "{0}://{1}/Schedule?CallbackTime={2:s}Z&CallbackUrl={3}",
        serviceBaseUri.Scheme,
        serviceBaseUri.Authority,
        callbackUtcTime,
        EscapeCallbackUrl(callbackUrl));
}

private static string BuildCancelRequestUrl(Uri serviceBaseUri, Guid callbackId, Uri callbackUrl)
{
    return string.Format(
        "{0}://{1}/Cancel?CallbackId={2:D}&CallbackUrl={3}",
        serviceBaseUri.Scheme,
        serviceBaseUri.Authority,
        callbackId,
        EscapeCallbackUrl(callbackUrl));
}

Naturally, there's a lot more code than this, but rendered down to its essence, you can either schedule a callback or cancel one (using the Guid you received when originally scheduling the callback). All of the URLs generated are used to call a REST web service. That's it. Easy, right?

Well...

Revalee.Service

The Revalee.Service project does the real heavy-lifting. Or to keep the Star Wars references going, this is where the Ugnaughts work (oh, and don't pretend you don't know who they are). More specifically, the Supervisor class manages it all, like Lobot (zing!):

using System;
using System.Diagnostics;
using System.Runtime.ConstrainedExecution;

namespace Revalee.Service
{
    internal sealed class Supervisor : CriticalFinalizerObject, IDisposable
    {
        private readonly ILoggingProvider _LoggingProvider;
        private readonly ConfigurationManager _ConfigurationManager;
        private readonly TelemetryManager _TelemetryManager;
        private readonly StateManager _StateManager;
        private readonly TimeManager _TimeManager;
        private readonly RequestManager _RequestManager;
        private readonly WorkManager _WorkManager;
        private readonly object _SyncRoot = new object();

        private bool _IsStarted;
        private bool _IsPaused;

        private Supervisor()
        {
            try
            {
                _LoggingProvider = new TraceListenerLoggingProvider();
                try
                {
                    _ConfigurationManager = new ConfigurationManager();
                    _TelemetryManager = new TelemetryManager();
                    _StateManager = new StateManager();
                    _TimeManager = new TimeManager();
                    _RequestManager = new RequestManager();
                    _WorkManager = new WorkManager();
                }
                catch (Exception ex2)
                {
                    try
                    {
                        _LoggingProvider.WriteEntry(
                            string.Format("{0} [Critical startup error.]", ex2.Message),
                            TraceEventType.Critical);
                    }
                    catch (Exception ex3)
                    {
                        Console.WriteLine("Could not write to the error log.");
                        Console.WriteLine("*  {0}", ex3.Message);
                    }

                    throw;
                }
            }
            catch (Exception ex1)
            {
                Console.WriteLine("Could not initialize logging subsystem.");
                Console.WriteLine("*  {0}", ex1.Message);
                throw;
            }
        }

        // ...
        // (Omitted for brevity; see source code for specifics)
        // ...

    }
}

As you can see, the Supervisor class instantiates numerous sub-manager classes each in charge of handling its own domain. They are:

Class	Responsibility
ConfigurationManager	Loads configuration settings from the Revalee.Service.exe.config file
TelemetryManager	Tracks activity via the Windows Performance Monitor
StateManager	Stores the callback requests
TimeManager	Wakes the service to process a callback
RequestManager	Handles incoming requests for callbacks
WorkManager	Performs the callback to your web application

And, yes, you saw good 'ole _SyncRoot to handle locking for the multi-threaded bits.

Extensible Storage Engine (aka. JET Blue)

A word or two about persistence. As a Windows Service, Revalee needs to be able manage data persistence, since a server can be rebooted at anytime (thanks, Ted!).

For this reason, Revalee uses the tried-and-true Extensible Storage Engine (ESE), also known as JET Blue. Per Wikipedia, "the ESE Runtime (ESENT.DLL) has shipped in every Windows release since Windows 2000". That's good, because it means Revalee can rely on having access to a database without having to install anything new. Fortunately, accessing ESE from .NET was made easy due to the existence of the ESENT Managed Interface open source project.

Data persistence in Revalee is handled in the EseTaskPersistenceProvider class. Highlighted below are the AddTask() and ListTasksDueBetween() methods that are used to interact with ESE via the Esent.Interop. Naturally, this class is rather long, but it's been edited down here for the sake of brevity:

using Microsoft.Isam.Esent.Interop;
using Microsoft.Isam.Esent.Interop.Windows7;
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading;

namespace Revalee.Service.EsePersistence
{
    internal class EseTaskPersistenceProvider : ITaskPersistenceProvider
    {
        private const string _DatabaseName = "RevaleeTasks";
        private const string _StorageEngineBaseName = "edb";
        private const int _ConnectionPoolSize = 10;

        private const string _TableNameCallbacks = "Callbacks";
        private const string _ColumnNameCallbackId = "CallbackId";
        private const string _ColumnNameCreatedTime = "CreatedTime";
        private const string _ColumnNameCallbackTime = "CallbackTime";
        private const string _ColumnNameCallbackUrl = "CallbackUrl";
        private const string _ColumnNameAttemptsRemaining = "AttemptsRemaining";
        private const string _ColumnNameAuthorizationCipher = "AuthorizationCipher";

        private Instance _EseInstance;
        private EseConnectionPool _ConnectionPool;
        private string _DatabasePath;

        private sealed class EseConnection : EsentResource
        {
            // (Omitted for brevity; see source code for specifics)
        }

        private sealed class EseConnectionPool : IDisposable
        {
            // (Omitted for brevity; see source code for specifics)
        }

        public void Open(string connectionString)
        {
            // (Omitted for brevity; see source code for specifics)
        }

        public void Close()
        {
            // (Omitted for brevity; see source code for specifics)
        }

        public RevaleeTask GetTask(Guid callbackId)
        {
            // (Omitted for brevity; see source code for specifics)
        }

        public void AddTask(RevaleeTask task)
        {
            if (task == null)
            {
                throw new ArgumentNullException("task");
            }

            if (_EseInstance == null)
            {
                throw new InvalidOperationException("Storage provider has not been opened.");
            }

            EseConnection connection = _ConnectionPool.OpenConnection();

            try
            {
                using (Table table = connection.GetTable(_TableNameCallbacks, OpenTableGrbit.Updatable))
                {
                    IDictionary<string, JET_COLUMNID> columnIds = connection.GetSchema(_TableNameCallbacks);

                    using (var transaction = new Transaction(connection))
                    {
                        using (var update = new Update(connection, table, JET_prep.Insert))
                        {
                            Api.SetColumn(
                                connection,
                                table,
                                columnIds[_ColumnNameCallbackId],
                                task.CallbackId);
                            Api.SetColumn(
                                connection,
                                table,
                                columnIds[_ColumnNameCreatedTime],
                                task.CreatedTime);
                            Api.SetColumn(
                                connection,
                                table,
                                columnIds[_ColumnNameCallbackTime],
                                task.CallbackTime);
                            Api.SetColumn(
                                connection,
                                table,
                                columnIds[_ColumnNameCallbackUrl],
                                task.CallbackUrl.OriginalString,
                                Encoding.Unicode);
                            Api.SetColumn(
                                connection,
                                table,
                                columnIds[_ColumnNameAttemptsRemaining],
                                task.AttemptsRemaining);

                            if (task.AuthorizationCipher != null)
                            {
                                Api.SetColumn(
                                    connection,
                                    table,
                                    columnIds[_ColumnNameAuthorizationCipher],
                                    task.AuthorizationCipher,
                                    Encoding.Unicode);
                            }

                            update.Save();
                        }

                        transaction.Commit(CommitTransactionGrbit.None);
                    }
                }
            }
            finally
            {
                _ConnectionPool.CloseConnection(connection);
            }
        }

        public void RemoveTask(RevaleeTask task)
        {
            // (Omitted for brevity; see source code for specifics)
        }

        public IEnumerable<revaleetask> ListAllTasks()
        {
            // (Omitted for brevity; see source code for specifics)
        }

        public IEnumerable<revaleetask> ListTasksDueBetween(DateTime startTime, DateTime endTime)
        {
            if (_EseInstance == null)
            {
                throw new InvalidOperationException("Storage provider has not been opened.");
            }

            DateTime rangeStartTime = NormalizeDateTime(startTime);
            DateTime rangeEndTime = NormalizeDateTime(endTime);

            // Inclusive Upper Limit does not work properly for the CLR DateTime type.
            // Add the smallest amount of time that the Esent engine will detect to include
            //  the ending range inclusively.
            rangeEndTime = rangeEndTime.AddMilliseconds(1.0);

            var taskList = new List<revaleetask>();

            EseConnection connection = this._ConnectionPool.OpenConnection();

            try
            {
                using (Table table = connection.GetTable(_TableNameCallbacks, OpenTableGrbit.DenyWrite
                                                                            | OpenTableGrbit.Preread
                                                                            | OpenTableGrbit.ReadOnly
                                                                            | OpenTableGrbit.Sequential))
                {
                    IDictionary<string,> columnIds = connection.GetSchema(_TableNameCallbacks);
                    Api.JetSetCurrentIndex(connection, table, "due");
                    Api.MakeKey(connection, table, rangeStartTime, MakeKeyGrbit.NewKey);

                    if (Api.TrySeek(connection, table, SeekGrbit.SeekGE))
                    {
                        Api.MakeKey(connection, table, rangeEndTime, MakeKeyGrbit.NewKey);
                        if (Api.TrySetIndexRange(connection, table, SetIndexRangeGrbit.RangeInclusive
                                                                  | SetIndexRangeGrbit.RangeUpperLimit))
                        {
                            JET_SESID jetSession = connection;
                            JET_TABLEID jetTable = table;
                            JET_COLUMNID jetColumnCallbackId = columnIds[_ColumnNameCallbackId];
                            JET_COLUMNID jetColumnCreatedTime = columnIds[_ColumnNameCreatedTime];
                            JET_COLUMNID jetColumnCallbackTime = columnIds[_ColumnNameCallbackTime];
                            JET_COLUMNID jetColumnCallbackUrl = columnIds[_ColumnNameCallbackUrl];
                            JET_COLUMNID jetColumnAttemptsRemaining = columnIds[_ColumnNameAttemptsRemaining];
                            JET_COLUMNID jetColumnAuthorizationCipher = columnIds[_ColumnNameAuthorizationCipher];

                            do
                            {
                                Guid? callbackId = Api.RetrieveColumnAsGuid(
                                    jetSession,
                                    jetTable,
                                    jetColumnCallbackId);
                                DateTime? createdTime = Api.RetrieveColumnAsDateTime(
                                    jetSession,
                                    jetTable,
                                    jetColumnCreatedTime);
                                DateTime? callbackTime = Api.RetrieveColumnAsDateTime(
                                    jetSession,
                                    jetTable,
                                    jetColumnCallbackTime);
                                string callbackUrl = Api.RetrieveColumnAsString(
                                    jetSession,
                                    jetTable,
                                    jetColumnCallbackUrl);
                                int? attemptsRemainingColumn = Api.RetrieveColumnAsInt32(
                                    jetSession,
                                    jetTable,
                                    jetColumnAttemptsRemaining);
                                string authorizationCipher = Api.RetrieveColumnAsString(
                                    jetSession,
                                    jetTable,
                                    jetColumnAuthorizationCipher);

                                Uri callbackUri = null;

                                if (callbackTime.HasValue
                                    && Uri.TryCreate(callbackUrl, UriKind.Absolute, out callbackUri)
                                    && createdTime.HasValue
                                    && callbackId.HasValue
                                    && attemptsRemainingColumn.HasValue)
                                {
                                    RevaleeTask revivedTask = RevaleeTask.Revive(
                                        DateTime.SpecifyKind(callbackTime.Value, DateTimeKind.Utc),
                                        callbackUri,
                                        DateTime.SpecifyKind(createdTime.Value, DateTimeKind.Utc),
                                        callbackId.Value,
                                        attemptsRemainingColumn.Value,
                                        string.IsNullOrEmpty(authorizationCipher) ? null : authorizationCipher);

                                    taskList.Add(revivedTask);
                                }
                            } while (Api.TryMoveNext(jetSession, jetTable));
                        }
                    }
                }
            }
            finally
            {
                _ConnectionPool.CloseConnection(connection);
            }

            return taskList;
        }

        private void CreateTaskTable(EseConnection connection)
        {
            // (Omitted for brevity; see source code for specifics)
        }

        private static DateTime NormalizeDateTime(DateTime time)
        {
            if (time.Kind == DateTimeKind.Local)
            {
                return time.ToUniversalTime();
            }
            else if (time.Kind == DateTimeKind.Utc)
            {
                return time;
            }
            else
            {
                return DateTime.SpecifyKind(time, DateTimeKind.Utc);
            }
        }

        ~EseTaskPersistenceProvider()
        {
            this.Dispose(false);
        }

        public void Dispose()
        {
            // Re-opening a disposed provider will cause an ObjectDisposedException,
            //  use Close() to re-open a provider.
            this.Dispose(true);
            GC.SuppressFinalize(this);
        }

        protected virtual void Dispose(bool isDisposing)
        {
            // (Omitted for brevity; see source code for specifics)
        }
    }
}

Using ESE for persistence worked out great for Revalee, since it's compact and fast.

Conclusion

As reviewed above, using Revalee with your MVC application to schedule future tasks is easy. It makes operations that are typically outside the scope of your web application instead a core part of your web application. The web callbacks received from Revalee promotes those requests to first-class status on par with everything else your MVC application does. As first-class processes, you never have to worry about your web application unloading without handling the requested actions. And that makes things like handling future expiration email messages a breeze.

Now go watch Star Wars Episode V: The Empire Strikes Back, you know you want to.

Further Reading

• Scheduling tasks with Revalee and MVC (Part 2)
• Scheduling recurring tasks with Revalee and MVC
• UrlValidator, a Project Widget used by Revalee

History

• [2014.Mar.04] Initial post.
• [2014.Mar.05] Removed errant text: "</string,>" from code block.
• [2014.May.19] Added 'Further Reading' section.
• [2014.May.23] Amended 'Further Reading' section with UrlValidator, a Project Widget used by Revalee.