Index
- Introduction
- Managed IOCP with Lock-Free Queue
- Lock-Free Object Pool
- Using Lock-Free Object Pool in .NET applications
- Putting it all together - Design Overview of ManagedIOCP
- The ManagedIOCP Thread Pool
- Inside the ManagedIOCP Thread Pool
- Extensible Task Framework for the ManagedIOCP Thread Pool
- Sonic.Net source and demo applications
- Points of interest
- History
- Software usage
1. Introduction
Managed I/O Completion Ports (IOCP) is part of a .NET class library named Sonic.Net, which I first released on CodeProject in May 2005. Sonic.Net is a free open source class library that can be used in building highly scalable server side .NET applications. This part-2 of Managed IOCP builds on top of my first Managed IOCP article. So it is a pre-requisite to read the first part of Managed IOCP before you read this part two of Managed IOCP. My first article on Managed IOCP is titled "Managed I/O Completion Ports (IOCP)". I request the readers to read the discussion threads in my first article, as they have a wealth of clarifications and information regarding Managed IOCP and its usage. Especially, the discussion with 'Craig Neuwirt' has brought out a critical issue with my original Managed IOCP implementation and helped me rectify it and make ManagedIOCP better (thanks Craig :)).
2. Managed IOCP with Lock-Free Queue
Before getting into Lock-Free stuff, the Managed IOCP source and demo applications contained in the downloadable Zip of this article are compiled using my Lock-Free Queue class. Removing the conditional compiler constant LOCK_FREE_QUEUE from Sonic.Net project properties will enable the Sonic.Net assembly to be compiled with the .NET synchronized Queue class.
Coming back to our Lock-Free stuff, I used a synchronized System.Collection.Queue class as the internal object queue for Managed IOCP in the Sonic.Net v1.0 class library. This provides a scope for lock contention between threads that are dispatching objects to Managed IOCP, and also between threads that are retrieving objects from Managed IOCP. I had to use a synchronized queue because the pop and push operations of a queue are not atomic. During a push operation, a thread has to first set the next node of the queue's tail node to point to the new node that holds the object to be enqueued. Next, the thread has to point the tail to the new node as the new node will now be the tail of the queue. These two operations cannot be performed atomically (with a single CPU instruction). This means that when multiple-threads are pushing objects (enqueuing) to the same instance of a queue, there are chances that some of the enqueue operations will be unsuccessful, and more dangerously, without the thread performing the push operation ever coming to know about it. Here is the scenario...
- Thread-A started the enqueue (push) operation.
- Thread-A created new node that holds the new object to be enqueued.
- Thread-A assigned the new node address to the current tail's next node.
- Thread-B started the enqueue (push) operation.
- Thread-B created the new node that holds the new object to be enqueued.
- Thread-B assigned the new node address to the current tail's next node.
Here comes the disaster. Thread-A thought that it has pointed the current tail's next node to its new node. But before it could point the tail to the new node (to make the new node the new tail), Thread-B has pointed the current tail's next node to its own new node. Oops!!! Thread-A's new node future is now hanging in mid air. The disaster is not yet over. Check out the following sequence of operations in continuation with the above mentioned operations of Thread-A and Thread-B.
- Thread-A pointed the current tail to its new node (hoping to make its new node the new tail).
- Thread-A exited out of the enqueue (push) operation.
- Thread-B pointed the current tail to its new node, and in-fact, made its new node the new tail.
After step-9 in the above sequence of events, the new node that is actually enqueued by Thread-A is lost. It is lost because neither the tail's next node nor the tail is pointing to it. It has become an orphaned object. In the C++ world, it would have led to serious memory leaks in the application as there is no way to reach the new node and its contained object, so no one will be able to free it. Fortunately, in the .NET world, the CLR's Garbage Collector will come to our rescue, and will eventually cleanup the orphaned node and its contained object later at some point of time. But the more serious problem here is not with the orphaned object but with the lost enqueue operation without the knowledge of the thread (Thread-A) that is performing the operation. Thread-A would think it has enqueued the object successfully, and would not report any error. This leads to missing objects, which could mean loss of data in the application. Loss of data, that too without notice, is a serious problem for any application.
It should be clear from the above discussion that the following two logically related operations should be synchronized using some locking mechanism, so that only one thread in the process could execute them at any given point of time.
- Thread-X assigned the new node address to the current tail's next node.
- Thread-X pointed the current tail to its new node and made its new node the new tail.
Synchronizing access to common objects using locking may not always be the right approach. It will definitely stop the loss of data and corruption, but when your application is running in a multi-processor environment, it may reduce the performance of your application. This is because in a multi-processor environment, you will experience true parallelism in thread execution. So, when two threads are running in parallel and trying to enqueue (push) objects onto the same queue, then one of the threads has to wait until the other has come out of the locked code region. Since this enqueue operation is a core function that is used very frequently in Managed IOCP based applications, the threads _may_ experience lock contention, and the kernel activity might go up significantly as the OS has to keep switching the thread from running to suspended mode and vice-versa. It is desirable to suspend the thread when there are no objects queued onto the Managed IOCP. But when objects are present, it would be good if each thread is active for the maximum time possible in its given CPU cycle time.
We (Windows developers) have been using lock based synchronization successfully in Windows environment till now. So we can continue using it. But there are other more promising alternative techniques to lock based synchronization, especially when designing highly scalable server side applications. These are called Lock-Free algorithms. These techniques will allow multiple threads to safely use common objects without corrupting the object state or causing any loss of data. These techniques have been under heavy research in the recent past, and are making their way slowly into mainstream application development. The reason of their slower adoption is, it is difficult to prove the correctness of these techniques while implementing on certain data-structures like hashtables. But fortunately, for the Managed IOCP queue that is used heavily by multiple-threads, there are matured and well tested lock-free algorithms.
2.1. Lock-Free Queue
The Lock-Free queue that I implemented to use in Managed IOCP is built on one of the matured algorithms for designing the Lock-Free queue data structure. As we discussed earlier, we need two physical operations to perform a single enqueue (push) operation on a queue. So what we need to check is that, while we read the then next node of the current tail and assign our new node to it, no other thread should have modified the next node of the current tail. So, the read and exchange of the next node of the current tail should either succeed or fail atomically. The synchronization primitive provided by .NET (supported by hardware and Windows OS), CAS, is a perfect fit for this situation. I discussed about CAS in detail in section 4.2 of my first article on Managed IOCP (read section 1 for details on links to my first article on Managed IOCP).
Also, the loss of data that I discussed in this section is solved using a trailing tail technique. In this technique, after step-3 in above discussed sequence, the current tail's next node will be pointing to the new node created by Thread-A. So when Thread-B starts its enqueue operation, it can check if the current tail's next node is null. If it is not, it means that some other thread has changed the current tail's next node, but it has not yet modified the tail itself. So Thread-B would advance the _trailing_ current tail to point to the tail's own next node as Thread-A would anyway do it when it is given a CPU time slice by the OS. Now, Thread-B could restart its enqueue operation. When Thread-B successfully changes the current tail's next node, it can break out of this next node checking loop and point the tail to its new node, provided some other thread has not already advanced the tail during the next node checking step in the enqueue operation.
The code below demonstrates this trailing tail technique used to achieve lock-free enqueue operation:
public void Enqueue(object data)
{
Node tempTail = null;
Node tempTailNext = null;
Node newNode = _nodePool.GetObject() as Node;
newNode.Data = data;
do
{
tempTail = _tail as Node;
tempTailNext = tempTail.NextNode as Node;
if (tempTail == _tail)
{
if (tempTailNext == null)
{
if (Interlocked.CompareExchange(ref tempTail.NextNode,
newNode,tempTailNext) == tempTailNext)
break;
}
else
{
Interlocked.CompareExchange(ref _tail,tempTailNext,tempTail);
}
}
} while (true);
Interlocked.CompareExchange(ref _tail,newNode,tempTail);
Interlocked.Increment(ref _count);
}
One interesting point to note in the above code is that I used CAS (Interlocked.CompareExchange) on object types. This is a beautiful feature supported by .NET. This form of CAS will compare the object reference value pointed to by the first parameter, with that of the comparand, which is the third parameter, and will make the first variable point to the object specified in the second parameter. The downside of the current Interlocked.CompareExchange on object types is that you cannot use variables of your own reference types with this .NET API. So, I had to use the object data type to define my data members in the Node class and for the head and tail object references in the ManagedIOCP class. As you can observe in the above code for the Enqueue method, this leads to type casting of variables from the object to the Node type.
You can check out the lock-free Dequeue operation in the source code. After reading the above discussion, it would be easy to understand the lock-free Dequeue (pop) operation. I have also provided code comments to help the reader understand the logic behind the code.
2.2. Test results of Managed IOCP with Lock-Free Queue
I have run the same WinForms based demo application using Managed IOCP compiled with the Lock-Free queue implementation on a Pentium IV 3.0 GHz HT system and a Pentium III 700 MHz single processor system. The results are slightly better compared to the .NET synchronized Queue. I noticed good reduction in lock contention in the console demo application provided with this article, when using Managed IOCP with the Lock-Free queue (using the Performance Monitor application) when compared to using Managed IOCP with the .NET synchronized Queue class. I did not see noticeable benefits in terms of speed. This may be because of the fact that my demo applications are just used as a testing bed for verifying the feature correctness of Managed IOCP and other Sonic.Net classes, and stress testing them to identify any hidden multi-threading bugs. I'm positive that when used in real application scenarios with good processing in the threads, Managed IOCP with Lock-Free queue should perform better.
I believe that despite all the optimizations I discussed regarding the lock-free queue, the performance gained by using it varies based on its usage environment. So I request developers using Managed IOCP to test their application using both the .NET synchronized Queue and my custom implemented Lock-Free queue, to get a feel of the performance of both the queue classes.
3. Lock-Free Object Pool
Object Pooling is a technique that allows us to re-use existing objects again and again with different states. For example, in the Lock-Free queue data structure, I have a Node object that represents a node in the queue. But the only usage of the Node object is to hold the objects that are pushed onto the queue. After an object is popped out of the queue, the Node object holding it has no active references and will be garbage collected by the CLR at some point of time.
It would be efficient if I can re-use the Node object for other push operations, after its contained object is popped out of the queue. This way, I can reduce a lot of new object allocations, thus reducing the GC activity within the application. This will also increase the overall performance of the application as there would be less number of objects to garbage collect and the CLR will not hinder the execution of the application frequently for garbage collection.
But to maintain the list of freed-up Node objects, we need a data structure that can maintain a queue, which does not require new Node allocations. This special queue would act like a linked list, where we can insert new links at the top and remove links from the bottom (FIFO). This queue is special because when an object, in our case Node, is pushed onto it, it does not create a new Node to hold it. It assumes the objects pushed onto it contain a link member that can be used to link up the next Node in the queue. With this assumption, it can use the objects pushed onto it as Nodes by themselves. For this purpose, I created a type called "PoolableObject" that has a data member that points to an object of the same type (PoolableObject). So if I push an object of "PoolableObject" onto our Object Pool queue, it will just point our new object's link data member to the current top element of the queue, thus making our new object the top of the queue. When you pop out the object from this queue, it will just return the bottom object of the queue, and will set the link data member of the previous element of the bottom element to null, thus making it the bottom most element ready to be popped out.
The code below shows the definition of the "PoolableObject" type. Any type whose objects need to be pooled can derive from this type.
public class PoolableObject
{
public PoolableObject()
{
Initialize();
}
public virtual void Initialize()
{
LinkedObject = null;
}
public virtual void UnInitialize()
{
LinkedObject = null;
}
internal object LinkedObject;
}
The above mentioned PoolableObject type is used by a class called ObjectPool that provides us with a mechanism to store objects in a pool. This class provides us with methods to add new objects to the pool and retrieve existing objects from the pool. This ObjectPool class is implemented using the special FIFO Lock-Free queue that we talked about in the beginning of this section. This queue does not allocate Nodes to hold the poolable objects, rather it uses the object queued onto it as a Node by itself. For this reason, we said that we need a type that provides us with the ability to link itself to an object of the same type. Here comes the "PoolableObject" type defined in the above code snippet.
The above definition of PooledObject is simple. It has a default no-argument constructor for easy creation of poolable objects by a factory class named PoolableObjectFactory (I'll explain this in a moment). It has a virtual Initialize method that can be overridden by the derived classes to perform any initialization. This method is called by ObjectPool class when a new poolable object is created by the poolable object factory and when the ObjectPool class is returning an object from its object pool queue. It has a virtual UnInitialize method that can be overridden by the derived classes to perform any un-initialization. This method is called by ObjectPool when a poolable object is being added to its object pool queue.
As I mentioned above, I'm using a factory class, PoolableObjectFactory, to create new objects that are derived from the PoolableObject type. I need this because initially when a new ObjectPool is created, there will not be any objects in its pool. So when an application asks it for an object from its pool, it silently creates a new object and will return it to the caller. But for the ObjectPool to create a new poolable object, it does not know about the type of the object that is derived from the PoolableObject. This is application specific, and only the application that created a new type from the PoolableObject type knows what type of objects it will pool using the ObjectPool class. So, I provided an abstract factory type called PoolableObjectFactory, which can be implemented by developers to create and return the objects of their poolable type. As mentioned above in this section, the poolable types that the developers create should be derived from the abstract poolable type "PoolableObject".
The code below shows the definition of the PoolableObjectFactory type:
public abstract class PoolableObjectFactory
{
public abstract PoolableObject CreatePoolableObject();
}
4. Using Lock-Free Object Pool in .NET applications
The above section (section 3) discussed most of the details about PoolableObject and PoolableObjectFactory. In this section, I'll show you a practical implementation of the PoolableObject and PoolableObjectFactory types. This section will help you build your own poolable objects in your .NET applications that can be managed and pooled for you by the ObjectPool class.
As I mentioned in my previous section (section 3), the Node type used by the Lock-Free queue is a poolable object. It is derived from the PoolableObject type. It also shows a simple implementation of the Initialize method of the PoolableObject type by derived classes. The code below shows the definition of the Node type.
class Node : PoolableObject
{
public Node()
{
Init(null);
}
public Node(object data)
{
Init(data);
}
public override void Initialize()
{
Init(null);
}
private void Init(object data)
{
Data = data;
NextNode = null;
}
public object Data;
public object NextNode;
}
In the above definition of the Node class, you can see the overridden method Initialize of the PoolableObject class in bold. I did not override the PoolabelObject::UnInitialize method as the Node class need not do any un-initialization work. Instead, the default one provided by the PoolableObject class is sufficient. Remember that the derived class implementation of UnInitialize should call the base class (PoolableObject) UnInitialize method, as it does an important job of setting its Link data member to null. This is important for the proper functioning of the ObjectPool class.
Once we define a new poolable type, we need to provide an implementation of the abstract PoolableObjectFactory that will be used by the ObjectPool class to create new poolable objects when required. The code below shows an implementation of the PoolableObjectFactory class that I used to create new instances of the poolable Node type.
class NodePoolFactory : PoolableObjectFactory
{
public override PoolableObject CreatePoolableObject()
{
return new Node();
}
}
Now we have a type (Node) that is poolable using the ObjectPool class, and a type (NodePoolFactory) that can be used by ObjectPool to create new instances of our poolable Node type when required. Now, to use poolable Node objects, it is as simple as creating an instance of the ObjectPool class and providing it a reference to an instance of the NodePoolFactory class. Below is a code snippet showing the creation of a ObjectPool instance, taken from the Lock-Free queue class.
private ObjectPool _nodePool =
new ObjectPool(new NodePoolFactory());
Once the ObjectPool is instantiated, we can get an object of our poolable object type (Node) from the pool by using the GetObject() instance method of the ObjectPool class. The code below shows how to use the ObjectPool class to get the objects from the pool.
Node newNode = _nodePool.GetObject() as Node;
When we are done with a poolable object, we should give it back to the pool so that it can be re-used later. We can add an object back to the pool by calling the AddToPool() instance method on the ObjectPool class. For instance, in the Lock-Free queue class, once an object is popped out of the Queue, the Node holding that object can be re-used to hold any new object to be enqueued onto the Queue. So, just before leaving the Dequeue operation, we add the Node object to the Node object pool maintained by the Queue class.
_nodePool.AddToPool(tempHead);
5. Putting it all together - Design Overview of ManagedIOCP
The above diagram shows the design of ManagedIOCP and its relation to the Lock-Free ObjectPool and the Lock-Free Queue.
6. The ManagedIOCP Thread Pool
Thread Pools have been an integral part of applications with a good amount of asynchronous and parallel computing requirements. Generally, server side applications use Thread Pool for a consistent and easy to use programming model for executing tasks in parallel and asynchronously. With Managed IOCP as the core technology, I built a Thread Pool that not only provides basic thread management but few other important features as listed below:
- Maximum allowed threads in the Thread Pool (different from concurrency limit).
- Concurrency limit on active threads in the Thread Pool.
- Extensible Task framework for defining application tasks that can be executed by the Thread Pool.
Before getting into the inside implementation of the ManagedIOCP Thread Pool, I'll describe its usage in a .NET application. Firstly, ManagedIOCP executes objects that implement an interface named ITask. The definition of the ITask interface is shown below:
public interface ITask
{
void Execute(ThreadPool tp);
bool Active {get;set;}
void Done();
}
As shown from the above definition, the Execute method is where the logic for executing the task should be written. Once you have a type implementing the ITask interface with logic in its Execute method, using the ManagedIOCP Thread Pool is as simple as creating an instance of it and dispatching the ITask objects onto it. When a ITask object is chosen by the Thread Pool for execution, it will call the Execute method on the object. The code below shows a dummy implementation of the ITask interface and how to use it with the ManagedIOCP Thread Pool.
public class MyTask : ITask
{
#region ITask Members
public void Execute(Sonic.Net.ThreadPool tp)
{
MyTask objTsk = new MyTask();
tp.Dispatch(objTsk);
}
public void Done()
{
}
public bool Active
{
get
{
return _active;
}
set
{
_active = value;
}
}
#endregion
private bool _active = true;
}
The above code shows a type MyTask that implements the ITask interface. If you observe the code, the Execute method has a ThreadPool object as parameter, so that the object being executed by the Thread Pool has access to the ThreadPool object itself for any further dispatching of objects onto the Thread Pool that is executing the current task object.
Also, the ITask interface has two other important members. The Done method is called by the Thread Pool, when the Execute method on a ITask object is completed. This gives a chance to the ITask object to perform any clean-up or pool itself for re-use (this is a powerful concept, and I'll discuss this shortly in the 'Task Framework' section). The Active property indicates the Thread Pool whether to execute this ITask object (whether to call the Execute method) or not. So if an application, after dispatching an ITask object to the Thread Pool, for some reason decides to cancel the task execution, it can set the task object's Active property to false, thus canceling the task execution, provided the task object has not already been executed by the Thread Pool.
The code below shows how to create an instance of the Thread Pool in the first place, and dispatch a MyTask object to it for asynchronous and parallel execution:
ThreadPool tp = new ThreadPool(10,5);
ITask objTask = new MyTask();
tp.Dispatch(objTask);
7. Inside the ManagedIOCP Thread Pool
The ManagedIOCP Thread Pool is implemented as a simple wrapper around the core ManagedIOCP class. When an instance of a ManagedIOCP Thread Pool is created, it internally creates an instance of ManagedIOCP, creates all the maximum number of threads, and will register those threads with the ManagedIOCP instance. When a thread of the Thread Pool retrieves an object from the ManagedIOCP instance of the Thread Pool, it will cast the object to a ITask object and will call the Execute method on it.
If you observe the constructor of the ThreadPool class, it has a second form of constructor that takes in a delegate named ThreadPoolThreadExceptionHandler. When a handler is provided for this parameter, if a Thread Pool thread encounters any exceptions while executing the ITask object, it will call this delegate and will continue processing other objects. In case the handler throws any exception, the exception is ignored. If no handler is provided for this delegate, then the thread will ignore the exception and will still continue processing other objects.
7.1. Managing Burst and Idle situations in ManagedIOCP ThreadPool
Creating all the maximum threads at once for each Thread Pool instance will not be an overhead on the system. Because, the number of active threads in a ManagedIOCP Thread Pool is controlled by the concurrency limit of the Thread Pool, which is specified during the instantiation of a Thread Pool instance and which can also be set at runtime. There could be situations where the active number of threads retrieve ITask objects from the ManagedIOCP instance of the Thread Pool, and while processing them, might go into wait mode (not-running). This could happen if the Execute method of the ITask object is calling into a Web-Service synchronously, etc. When this happens, if there are any pending ITask objects in its queue, the ManagedIOCP instance of the Thread Pool will wake-up other sleeping threads for processing those objects. While these extra threads are processing the ITask object, the earlier threads that went into sleeping mode while executing their ITask objects could come out of sleeping mode and start running. This will create a state in the Thread Pool where more than the allowed concurrent number of threads will be running at a given point of time.
This above discussed situation can be eliminated by having the max. number of threads in the Thread Pool equivalent to the number of allowed concurrent threads. But this may reduce the scalability of the application. This is because if all running threads are waiting on external resource/triggers/events like web service calls, though the application is idle, it will not be able to service any pending requests.
Having the max. number of threads greater than the allowed concurrent threads in the Thread Pool is always desired to scale the application under loads and utilize the system resources as much as possible and as long as possible. In order to balance out the burst situations and the idle situations, ManagedIOCP used by ThreadPool has built in support for suspending un-wanted IOCPHandles that are registered with it. When a IOCPHandle is coming into wait state, if the number of current active threads are greater than or equal to the number of allowed concurrent threads, then the IOCPHandle is queued onto a suspended queue. This way though the number of max. threads in the ThreadPool is greater than the allowed concurrent threads, the threads that are waiting for processing the requests would be closer to the allowed concurrent threads. This would not stop the actual active threads being greater than the allowed concurrent threads, but would keep the difference at minimal levels. When a new object is dispatched to ManagedIOCP, if the current active thread count is less than the allowed concurrent thread count _and_ if the registered IOCPHandle count is greater than or equal to the allowed concurrent thread count, the ManagedIOCP will try to get a suspended thread (IOCPHandle). If it finds one, the thread is chosen for handling the dispatch by setting its IOCPHandle's wait event. This situation may occur if there are a few objects to be dispatched than the allowed concurrent threads _or_ some of the active threads went into waiting mode. In either case, waking up any unsuspended thread may not be an overhead, and by all means should be able to handle the idle situation discussed in this section.
This way Dynamic ManagedIOCP should be able to handle both burst and idle situations that are common in IOCP based ThreadPool designs. Dynamic ManagedIOCP is not enabled by default in the Sonic.Net library. The code related to Dynamic ManagedIOCP is inside the conditional compilation constant DYNAMIC_IOCP. The Dynamic ManagedIOCP can be enabled by specifying the conditional compilation constant named DYNAMIC_IOCP in the Sonic.Net class library project properties.
8. Extensible Task Framework for the ManagedIOCP Thread Pool
Task Framework provides an extensible framework for creating tasks that are to be executed by the ManagedIOCP Thread Pool. It provides abstract base classes with implementation for the Active property and the Done method of the ITask interface. These abstract base classes provide different varieties of tasks, like, waitable task, context bound task and waitable context bound task. Also, each abstract task class is derived from the PoolableObject type, thus providing task pooling. Each abstract task type has an associated abstract factory class to create instances of the corresponding task type. These abstract task factory types maintain a pool of task objects.
All the abstract task classes in the Task Framework are derived from a single abstract base class named Task. This abstract base class implements the Active property and the Done method of the ITask interface and is the one that implements the PoolableObject abstract class. Other abstract task classes derive from this class and provide their own capabilities like waiting on task completion, context binding, etc. All the abstract factory classes for creating different classes of task objects are derived from a single abstract base class named TaskFactory. This abstract base class provides task object pooling. This abstract base class is in-turn derived from PoolableObjectFactory, whose abstract methods have to be implemented by applications that wish to use the Task Framework.
The diagram below shows the ManagedIOCP ThreadPool Task framework:
The diagram below shows the ManagedIOCP ThreadPool Task Factory framework:
8.1. Creating and using Generic Task
GenericTask abstract class provides a basic implementation of the ITask interface for a task to be executed by the Thread Pool. GenericTask abstract class is derived from the Task abstract class, so that it provides features like canceling the task execution by setting the Active property value to 'false'. GenericTask is an abstract class because it does not implement the Execute method of the ITask interface. It is upto the application using the GenericTask to derive from it and implement the Execute method as required. The code below shows a class that is derived from GenericTask and implements the Execute method:
public class MyGenericTask : GenericTask
{
public override void Execute(ThreadPool tp)
{
}
}
Once we have the application specific generic task class, we can create an instance of it and dispatch it to the ThreadPool.
MyGenericTask gt = new MyGenericTask();
tp.Dispatch(gt);
We can use TaskFactory and its derived abstract classes to create/acquire instances of the GenericTask class. The advantage is that these abstract factory classes provide object pooling of task objects. The code below shows a class that is derived from GenerictaskFactory and implements the GetObject method. This factory creates/acquires instances of an application specific GenericTask class.
class MyGenericTaskFactory : GenericTaskFactory
{
public override PoolableObject CreatePoolableObject()
{
return new MyGenericTask();
}
}
Once we have the application specific GenericTask class and associated GenericTaskFactory class, we can create/acquire instances of the application specific GenericTask class and dispatch them to the ThreadPool.
MyGenericTaskFactory gtf = new MyGenericTaskFactory();
MyGenericTask gt = gtf.NewGenericTask(null, null);
tp.Dispatch(gt);
The first null parameter passed to the NewGenericTask method of the GenericTaskFactory class is the ID given to the task. This can be set to a valid non-object for uniquely identifying the task. The second null parameter is any application related object that needs to be associated with the task. This can be used to pass additional information associated with the task, which can be used during its execution.
8.2. Creating and using Waitable Task
The WaitableTask abstract class provides a task on which the application can wait for a task to be executed by the Thread Pool, after dispatching the task to the Thread Pool. The creation and usage of WaitableTask class is same as that of GenericTask. WaitableTask does not have its own factory class, as it is an extension of GenericTask and provides a waitable mechanism to wait on the completion of the underlying GenericTask. Wait on the WaitableGenericTask supports time-out in milliseconds. If time-out occurs during wait operation, the Wait method on the WaitableGenericTask class will return 'false'. The code below shows how one can wait on the WaitableGenericTask:
MyGenericTaskFactory gtf = new MyGenericTaskFactory();
MyWaitableGenericTask gt = gtf.NewGenericTask(null, null);
tp.Dispatch(gt);
bool bTimeOut = gt.Wait(-1);
8.3. Creating and using ContextBound Task
The ContextBoundGenericTask abstract class provides a task whose execution is serialized with other tasks within the same context. Context provides a logical locking/unlocking mechanism, which can be used by tasks executing under a context. When a task locks the associated context during execution, other tasks trying to lock the context will be suspended until the task that locked the context unlocks it. The ManagedIOCP Task Framework has an interface named IContext that represents a context. The Task Framework also has a default implementation of the IContext interface named Context. The Context class provides locking and unlocking semantics using the Monitor synchronization object.
The creation and usage of ContextBoundGenericTask class is same as that of the GenericTask. A separate factory class named ContextBoundGenericTaskFactory is provided with the Task Framework. Applications have to derive from the ContextBoundGenericTaskFactory class and implement its CreatePoolableObject method to create/acquire instances of classes derived from the ContextBoundGenericTask class.
One additional step in creating the ContextBoundGenericTask derived class is that, the code implemented in the Execute method of the derived class should lock and unlock the context object available in the base ContextBoundGenericClass as a property named Context.
The code below shows how to create and use an application specific ContextBoundGenericTask class:
public class MyContextBoundGenericTask : ContextBoundGenericTask
{
public override void Execute(ThreadPool tp)
{
Context.Lock();
Context.UnLock();
}
}
public class MyContextBoundGenericTaskFactory :
ContextBoundGenericTaskFactory
{
public override PoolableObject CreatePoolableObject()
{
return new MyContextBoundGenericTask();
}
}
MyContextBoundGenericTaskFactory ctxGTF =
new MyContextBoundGenericTaskFactory();
object ctxId =
ContextIdGenerator.GetInstance().GetNextContextId();
Context ctx = new Context(ctxId);
MyContextBoundGenericTask ctxGT =
ctxGTF.NewContextBoundGenericTask(null, null,ctx);
tp.Dispatch(ctxGT);
9. Sonic.Net source and demo applications
Below are the details of files included in the article's ZIP file:
- Sonic.Net (Folder) - I named this class library as Sonic.Net (Sonic stands for speed). The namespace is also specified as
Sonic.Net. All the classes that I described in this article are defined within this namespace. The folder hierarchy is described below: Sonic.Net
|
--> Assemblies
|
--> Solution Files
|
--> Sonic.Net
|
--> Sonic.Net Console Demo
|
--> Sonic.Net Demo Application
The Assemblies folder contains the Sonic.Net.dll (contains ObjectPool, Queue, ManagedIOCP, IOCPHandle and ThreadPool classes), Sonic.Net Demo Application.exe (demo application showing the usage of ManagedIOCP and IOCPHandle classes) and Sonic.Net Console Demo.exe (console demo application showing the usage of the ThreadPool and ObjectPool classes).
The Solution Files folder contains the VS.NET 2003 solution file for the Sonic.Net assembly project, Sonic.Net demo application WinForms project, and Sonic.Net console demo project.
The Sonic.Net folder contains the Sonic.Net assembly source code.
The Sonic.Net Console Demo folder contains the Sonic.Net console demo application source code. This demo uses a file that will be read by the ThreadPool threads. Please change the file path to a valid one on your system. The code below shows the portion in code to change. This code is in the ManagedIOCPConsoleDemo.cs file.
public static void ReadData()
{
StreamReader sr =
File.OpenText(@"C:\aditya\downloads\lgslides.pdf");
string st = sr.ReadToEnd();
st = null;
sr.Close();
Thread.Sleep(100);
}
The Sonic.Net Demo Application folder contains the Sonic.Net demo application source code.
- Win32IOCPDemo (Folder) - This folder contains the WinForms based demo application for demonstrating the Win32 IOCP usage using PInvoke. When compiled, the Win32IOCPDemo.exe will be created in the Win32IOCPDemo\bin\debug or Win32IOCPDemo\bin\Release folder based on the current build configuration you selected. Default build configuration is set to Release mode.
10. Points of interest
To summarize, we now have a Sonic.Net library that provides Lock-Free data structures like Queue and ObjectPool, asynchronous and parallel programming infrastructure classes like ManagedIOCP, ThreadPool and a Task Framework. Apart from these classes, there is a small utilities class named StopWatch that comes with the Sonic.Net assembly. The StopWatch class can be used to measure the elapsed time easily in a convenient manner. Check it out in case you are interested. Apart from the test applications I have provided with this class library, I believe that the real test for this type of class library is a good real-world server side application. I request users of this class library to provide any feedback/suggestions to fix bugs and improve it.
I'm working on a version of Sonic.Net for .NET 2.0. I'm moving Managed IOCP to a _Generic_ class with the data to be queued, defined as a template parameter. In this context, I had to use my lock-free queue for queuing objects onto Managed IOCP, as .NET 2.0 does not yet support synchronized Generic collections. I'm creating a lock-free Generic (templated) queue to be used in generic Managed IOCP. As soon as I complete, I will update this article with new code and share my experience on using .NET 2.0 Generics. Also the most exciting part of .NET 2.0 is that it supports Generic (templated) Interlocked.CompareExchange. This means the data members in our Node class and head and tail node object references in our Managed IOCP class can be of Node type rather than object type. This would be efficient and would save some type casting overhead during runtime.
11. History
Date: Apr 17, 2006
Fixed an issue related to usage of Interlocked.CompareExchange. Thanks to Smith Cameron (LexisNexis organization) for pointing out this issue.
Date: Aug 15, 2005
Sonic.Net v1.1 - Lock-Free Queue, ObjectPool, ManagedIOCP with revamped (and enhanced) thread choosing algorithm for executing dispatched objects, ManagedIOCP based ThreadPool, and an extensible Task Framework for defining tasks to be executed by the ManagedIOCP ThreadPool.
Date: May 09, 2005
I also fixed a small bug in the Windows demo application (that existed in version 1.0). This bug can allow two threads in the _demo_ application to use the same Label object to display their count. This is fixed in this version (1.1). The bug is fixed in the ManagedIOCPTestForm::StartCmd_Click(...) method.
Date: May 04, 2005
Sonic.Net v1.0 (Class library hosting ManagedIOCP and IOCPHandle class implementations with .NET synchronized Queue for holding data objects in Managed IOCP).
12. Software Usage
This software is provided "as is" with no expressed or implied warranty. I accept no liability for any type of damage or loss that this software may cause.
