Quartz.Net with Multiple Schedulers

• Download demo project - 823.2 KB
• Download source from GitHub

Introduction

This article is about using Quartz.Net in a more modular and extensive approach. When we're using Quartz.Net for job queuing and execution, there is a possibility that we might face issues of volume, memory usage and multithreading. This project provides the solution by creating a custom Quartz Schedulers which will pick up jobs from the queue and execute it. It gives us the flexibility to schedule job using the web application and decide which remote server should execute the job. Depending on the nature of the job, we can select the scheduler/listener which will be responsible for picking up the job from the queue and executing it.

Background

It will help you gain better understanding about architecture of the framework and how it is being used here. Quartz.Net has its own windows service that it provides that helps you offload some of the job to remote server. But if you plan on having a custom implementation, this article will be helpful.

Requirement

Before going through this article, please go over the basics of Quartz Scheduler. Before moving forward, you'll need to have SQL Server to get it to work.

Using the Code

The first step is to get the Quartz.Net package from Nuget.

The next step is to set up the database. Quartz requires a set of tables when we plan on using SQL Server as our datastore. You can find the SQL script in the project. Once the database is updated, we need to configure our Quartz. There are multiple ways of doing it like having it in Web.config/App.config, a separate

• Web.config/App.config
• Separate config file for Quartz
• Writing in class file as NameValueCollection

For simplicity, we're writing in the class file, i.e., QuartzConfiguration.cs. Here, we've two separate configuration pointing to the same database. The only difference between the two is the instance name. So while scheduling, we'll define which scheduler it needs to run under and that particular scheduler will select the job and run. All other schedulers will not execute it due to instance name difference.

public class QuartzConfiguration
{
    public static NameValueCollection RemoteConfig()
    {
        NameValueCollection configuration = new NameValueCollection
        {
            { "quartz.scheduler.instanceName", "RemoteServer" },
            { "quartz.scheduler.instanceId", "RemoteServer" },
            { "quartz.jobStore.type", "Quartz.Impl.AdoJobStore.JobStoreTX, Quartz" },
            { "quartz.jobStore.useProperties", "true" },
            { "quartz.jobStore.dataSource", "default" },
            { "quartz.jobStore.tablePrefix", "QRTZ_" },
            { "quartz.dataSource.default.connectionString",
              "Server=(servername);Database=(datbasename);Trusted_Connection=true;" },
            { "quartz.dataSource.default.provider", "SqlServer" },
            { "quartz.threadPool.threadCount", "1" },
            { "quartz.serializer.type", "binary" },
        };

        return configuration;
    }

    public static NameValueCollection LocalConfig()
    {
        NameValueCollection configuration = new NameValueCollection
        {
            { "quartz.scheduler.instanceName", "LocalServer" },
            { "quartz.scheduler.instanceId", "LocalServer" },
            { "quartz.jobStore.type", "Quartz.Impl.AdoJobStore.JobStoreTX, Quartz" },
            { "quartz.jobStore.useProperties", "true" },
            { "quartz.jobStore.dataSource", "default" },
            { "quartz.jobStore.tablePrefix", "QRTZ_" },
            { "quartz.dataSource.default.connectionString",
              "Server=(servername);Database=(datbasename);Trusted_Connection=true;" },
            { "quartz.dataSource.default.provider", "SqlServer" },
            { "quartz.threadPool.threadCount", "1" },
            { "quartz.serializer.type", "binary" },
        };

        return configuration;
        }
    }

So when we're scheduling the job, we'll use appropriate configuration. Another thing to be careful about is to start the scheduler. If the scheduler isn't running, it can schedule jobs on the queue, but it cannot execute it. In this example, I'm using web application to schedule the job only. I'm not starting the scheduler. Hence, my web app will never pick up any jobs from the queue as scheduler instance is not running.

How It Works

When you schedule a job using SQL server datastore, the job gets saved to [dbo].[QRTZ_JOB_DETAILS] table.

The highlighted section is the name of the scheduler it is meant to run under. So when I start my Remote Scheduler, it will only pick up the 2^nd and 3^rd jobs and not the first. It will only be picked up when I start Local Scheduler.

Here is the screenshot of the execution. If you notice, there are two jobs that were executed by Remote Scheduler and one by Local Scheduler.

If we're to use App.config or Web.Config to configure the schedulers, all we've to do is change the scheduler name in the file. The code will automatically start picking up jobs from the queue will the same scheduler name. It can be very beneficial in times when one of your schedulers is overloaded with jobs and you need additional resources to execute the jobs quicker. Without updating/replacing a single DLL, you can change the configuration of Quartz Schedulers and it will continue to work seamlessly.

Points of Interest

The most important thing that I learned that the Quartz.Net framework is very modular and flexible, making it easier to have a plug and play kind of components. Having a modular architecture is really helpful in times of working with larger jobs, volumes, long running jobs. As we're working on the SQL Server datastore, this concept can also be used across load balanced servers. The only catch is to make sure that the service has the required DLLs/binaries in its folder and the app.config file is regularly updated.