SaintModeCache.Net

Introduction

SaintModeCache.Net is a library inspired by Varnish's saint mode caching. Saint mode is when the cache continues to serve stale content after the cache has expired. This happens when the origin content becomes unavailable, and for as long as the cache cannot be refreshed.

The SaintModeCache library created for .NET works on a similar principle, but is applied at the data layer rather than Varnish's HTML caching.

It's useful when integrating with slow or unreliable data services that are frequently accessed, and protects the user experience from slow load times. Once cached, the data is kept in memory and refreshes on a single background thread when it expires. This keeps the site responsive at all times.

The key benefits of the library are as follows:

• Protects the user experience from slow running processes
• Enables stale data to be used until the cache can be refreshed
• Prevents cache misses from overloading dependent services or databases

Installation

To install SaintModeCache.Net, run the following command in the Package Manager Console.

PM> Install-Package SaintModeCache.Net

Demo Site

See an example of SaintModeCache.Net on the SaintCacheMode.Net Demo site.

This aggregates multiple RSS feeds which are refreshed in the background using the SaintModeCache. There is no other caching implemented and all requests are rendered on the server-side in real-time.

Caching You Is Easy

SaintModeCache.Net is easy to use and can be wrapped around any data access methods. It works like other caches, and uses MemoryCache under the hood.

To leverage saint mode, always provide an update delegate when requesting data from the cache. The delegate is only called when there's a cache miss. This ensures that the caller always receives data synchronously, although sometimes it may be stale. Well, you can't have everything!

First, setup a static reference to the cache instance to persist data in memory.

static SaintModeCache Cache = new SaintModeCache();

Next, start fetching and caching data with all the benefits of a saint.

var cacheKey = "customer123"; 
var cacheTimeInSeconds = 60; 
var customerModel = Cache.GetOrCreate(cacheKey, 
                          (key, cancelToken) => slowUnreliableService.GetCustomerModel(key),
                          cacheTimeInSeconds);

When the cache is empty, the method will block all callers and allow one thread to fetch the data from the slow unreliable service.

Once the update delegate completes, the cached data is returned immediately to all callers. After 60 seconds, the next request for the same cache key will trigger a background thread to update the cache but simultaneously return the currently cached value. After the cache is updated in the background, subsequent callers will receive the updated data.

Stay Thread Safe

The trick to SaintModeCache.Net is ensuring that only one thread can access the function for refreshing the cache. To achieve this, it triggers an on-demand refresh via a background thread, and continues to serve stale data to the caller. It uses locking to ensure only one refresh thread succeeds, and this prevents data sources from being overwhelmed when the cache expires and the site is under load. The locking is scoped to the current cache key which allows multiple data fetches in parallel.

If there is a cache miss because data isn't yet available, it blocks all requests for the same cache key until one thread provides the cacheable data for all requests to consume. Again this prevents data sources from being overwhelmed when under load, and ensures that the caller always receives data.

Locking Intern

The implementation for locking per cache key uses string.Intern and the cache key itself. This is a very powerful technique as it allows locking on a unique string across multiple threads and even AppDomains.

string.Intern returns the single reference to a string within .NET's intern pool. This ensures a single reference to a given string of the same value. Locking on a non-interned string is not enough as it will only lock around the exact string instance.

For example, the following code locks if 2 requests are made using the specific myLock instance.

string myLock = "myLockStr";
lock (myLock) {
   // do something
}

However, the following code using string.Intern will lock across all executing code where the same string value is used.

lock (string.Intern("myLockStr")) {
   // do something
}

It's possible to also attempt access to a lock without blocking by using the following code. This allows the library to kill unsuccessful refresh threads early.

if (Monitor.TryEnter(lockObj))
{
   try {
     // I have the lock

   } finally {
     Monitor.Exit(lockObj);
   }
} else {
   // I don't have the lock
}

Summary

SaintModeCache.Net provides a quick and simple solution for caching data within a .NET application, and can be used to complement other techniques. As the trend for rich immersive experiences increases, so do the performance challenges with managing near real-time data.

To read more about the library, follow these links:

• Github Source
• Nuget Binaries
• Demo Site