Solved: “The number of WaitHandles must be less than or equal to 64″

I came across this solution in the following blog post: http://www.switchonthecode.com/tutorials/csharp-tutorial-using-the-threadpool. The solution though was hidden pretty far down into the comments, so let me try here to bring you a bit more directly to the solution.

So your initial problem was, “The number of WaitHandles must be less than or equal to 64?. Where does it come from? Well, as the error messages says, you are limited to 64 WaitHandles. More specifically, you can’t create more than 64 ManualResetEvent objects. So in order to work our way around that problem, we have to stop relying on one ManualResetEvent object per thread to decide if all threads are completed or not.

The solution is to introduce a member variable that keep track of the number of threads yet to finish. When that variable reaches zero, we signal on one specific ManualResetEvent object that all threads are done. In order for this to work, you need to make the decrement of the pending threads to work as an atomic operation. The way to do so is by using Interlocked.Decrement, which ensures atomic decrement.

An example of how this pattern can be implemented:

using System;
using System.Threading;

namespace ThreadPoolTest
{
    class Program
    {
        private static int _numerOfThreadsNotYetCompleted = 100;
        private static ManualResetEvent _doneEvent = new ManualResetEvent(false);

        private static void Main(string[] args)
        {
            for (int threadNumber = 0; threadNumber < 100; threadNumber++)
                ThreadPool.QueueUserWorkItem(new WaitCallback(DoWork), 
					(object)threadNumber);

            _doneEvent.WaitOne();

            Console.WriteLine("All done.");
        }

        private static void DoWork(object o)
        {
            try
            {
                Console.WriteLine("Thread number: {0}", (int)o);
            }
            finally
            {
                if (Interlocked.Decrement(ref _numerOfThreadsNotYetCompleted) == 0)
                    _doneEvent.Set();
            }
        }
    }
}

Using this, you can queue as many threads as you like, and still be able to wait for all of them to finish.