TypeCache

• Download demo project - 6.6 KB
• Download source files - 13.3 KB

Introduction

There are situations when the result of an output in an I/O operation (file read, database query, etc.) is known not to change from time to time, but to retrieve it, you are going to pay with a little (sometimes a huge) amount of time. What if you could do this I/O operation at once and store this data for a while, so the next time you needed that value it was ready right away? Since this value does not change frequently, but it does change, the stored value must be discarded and then a new I/O operation will be needed.

Well, this is a (distorted) description of a technique known as cache.

What this proposed .NET class does is to cache values of a specific data type. It is called TypeCache because it uses static fields (and its memory places) for each stored type.

Why a Type Cache?

My motivation was to create a cache system for Value Objects (VO), that kind of object that has only properties and fields and, in my case, it is filled with data from a database. One of this type of data that was requested a lot was the user account information (from my own user management system). Depending on the process, it was necessary to get the same user permission information four times in less than a second (or less). I know that user permission does not change frequently, so it is not necessary to go to the database every time. I had to do something a little bit smarter. All I needed was to save that user permission on a cache system and invalidate it every 10 seconds, which would save me lots of queries on my database.

After a few days of work and tests, I started using a code that looks a lot like this:

public UserPermission GetUserPermission (string username, string permission)
{
    UserPermition usrPermission  = 
        TypeCache<UserPermission>.Get(username + permition);
    if (userPermition == null)
    {
        usrPermission = GetUserPermissionFromDB(username, permission); 
        TypeCache<UserPermission>.Set(username + permission, valstr);
    }
    
    return usrPermission;
}

Background

Before we start, it is important to understand two concepts used by this class. Even if you just want to use the class and don't care how it works internally, it is important to check these concepts because it will help you to avoid wrong usage of the cache.

The first concept is "static fields" or "static member variable". These are class fields of which values can be read by any instance of the type.

For example:

class TestStatic
{
    private static int _count = 0;
    public void Add() { _count++; }
    public int GetCount() { return _count; }
}
    
class Program
{
    static void Main(string[] args)
    {
        TestStatic t1 = new TestStatic(); 
        TestStatic t2 = new TestStatic();
            
    Console.WriteLine("t1 Count:{0}  - t2 Count:{1}", 
                    t1.GetCount(), t2.GetCount());
        t1.Add();

        Console.WriteLine("t1 Count:{0}  - t2 Count:{1}", 
                    t1.GetCount(), t2.GetCount());
        t2.Add();

        Console.WriteLine("t1 Count:{0}  - t2 Count:{1}", 
                    t1.GetCount(), t2.GetCount());
        }
    }

This returns:

t1 Count:0  - t2 Count:0
t1 Count:1  - t2 Count:1
t1 Count:2  - t2 Count:2

As you can see, both instances of TestStatic share the value of _count variable and both instances can manipulate it.

This is just a quick view of static fields. For more information go here and here. For a VERY deep look inside, go here.

The other concept is generics. You probably know about generics already, so I will not explaing the basics of generics here (if know nothing about generics, go here). I am going to explain the behaviour when you mix generics and static fields.

What if in our last example the class TestStatic was a generic class? Would this change the value of _count? Let's start with a simple example and make t1 and t2 use the same type (string).

class TestStatic<T>
{
    private static int _count = 0;
    public void Add() { _count++; }
    public int GetCount() { return _count; }
}


class Program
{
    static void Main(string[] args)
    {
        TestStatic<string> t1 = new TestStatic<string>();
        TestStatic<string> t2 = new TestStatic<string>();

        Console.WriteLine("t1 Count:{0}  - t2 Count:{1}", 
                    t1.GetCount(), t2.GetCount());
        t1.Add();

        Console.WriteLine("t1 Count:{0}  - t2 Count:{1}", 
                    t1.GetCount(), t2.GetCount());
        t2.Add();

        Console.WriteLine("t1 Count:{0}  - t2 Count:{1}", 
                    t1.GetCount(), t2.GetCount());
    }
}

This returns:

t1 Count:0  - t2 Count:0
t1 Count:1  - t2 Count:1
t1 Count:2  - t2 Count:2

It has the same values and behaviour of the first example since t1 and t2 are the same "type".

But let's change the t2 type to TestStatic<int> and see what happens.

t1 Count:0  - t2 Count:0
t1 Count:1  - t2 Count:0
t1 Count:1  - t2 Count:1

So, as you can verify, the static values are accessible only for the same T from the generic class. In addition, it does not matter if it is a reference type (string and Hashtable) or value type (int and double), you are going to end up with a single value of _count for each type of T.

What about the cache?

What static fields and generics have to do with cache? Well, the idea is: what if you could store a value of type T for a while using a unique storage for each type? All you need is a type and a key.

Basically, you store a value that you can find using a unique key. This key/value will be discarded after a predetermined period of time.

To get and set a value for a specific key:

string valstr = TypeCache<string>.Get("key.str1");
if (valstr == null)
{
    valstr = GetStringFromExpensiveIO(); // Or any other method
    TypeCache<string>.Set("key.str1", valstr);
}

Normally, these lines of code above are all you need to know how to use the TypeCache. First you try to retrieve the value from the cache. In case it is not there, do the hard work and store it back on the cache.

I am using string type for this example only. In general, it is not a good idea to use these general types (int, string, etc.) with TypeCache. It will not be a problem as long as you have a very good control over your keys for that type. As a general rule, more common the type, bigger is the key.

For value types of T, the method get will not return null, so you must compare it with the value that the C# keyword default(T) returns. For numeric values (int, double, etc), it is 0 (zero).

To change period of time that a entry on the cache is a valid item, you can change the property ExpirationInteval.

TypeCache<double>.ExpirationInteval = 2000;  // Two seconds

To change the interval that cache cleaner service will run, change the property CacheCleanInteval.

TypeCache<Hashtable>.CacheCleanInteval = 10000;   // Ten seconds

Different from other cache systems, it does not use delegates or events to reload the cache value when it expires. The intention is to keep the class usage as simple as possible. Not that delegate is used only for rocket scientists, but I think that the code above is easier to read and understand. The other reason is that it does not use a precise expiration engine. It means that a clean up service will clean the cache from time to time and it will not notify or trigger an event or something like that. Do not worry! You will never get an expired value. It just means that internally the reference for that value will not be discarded as soon it expires.

Deep inside

The values must leave...

As a cache system, it must clean its memory from time to time. How to do it? On the first version of the class, it was using one thread for each type of T. It was very accurate, since each thread had a clean up timer different for each type. But if you are going to use this cache class with many types, it is not hard to imagine you having memory and resource problems.

The solution was to use singleton class (TypeCacheCleaner) that holds a collection of delegates (one for each type of T) and creates a unique thread that executes this delegates. Each delegate is a call to a cache cleaner for each type on the cache. This thread still respects the CacheCleanInteval property of each T, but its timer runs as fast as the lowest CacheCleanInteval it can find.

Thread-safe

To keep the class usage simple, I did the class thread-safe (Well, I tried at least. We never know if we are doing it right when it is about to multithread). So, any attempt to read or write to the internal cache collection is surrounded with locks. On the first version I was using ReaderWriterLock from System.Threading namespace, but after a quick research I found out that this class is not recommended anymore (even the CLR team have problems with multithread and it already has a replacement for it on .NET 3.5 (ReaderWriterLockSlim). Because this class is intended to run on .NET 2.0, it uses the old and good lock (System.Threading.Monitor). My tests showed that ReaderWriterLock is faster then the traditional lock, but I preferred to stay with reliability over performance.

Conclusion

The TypeCache class can be useful to solve some I/O problems. After a quick tuning (changing ExpirationInteval and CacheCleanInteval), all you need is start using the class through Get and Set methods.

I would be pleased to hear from you and get any feedback.

History

• 2007-06-15: Version 1.00