How to Create WCF Timeout Operation Behavior

• Download source (RAR) - 156.4 KB
• Download source (ZIP) - 166.5 KB

Introduction

WCF acts up when it is under heavy requests, the weird behavior is shown as high CP utilization accompanied with very high memory consumption and delayed responses that continuously increase till another problem happens (OutOfMemoryException, TimeoutException, etc.)

If the client is a web application, the web page will wait for some time (110 milliseconds in .NET 4.0) and then times out, keeping the service running and eating up server resources.

Whilst there are timeout configurations for the WCF client (such as OpenTimeout, CloseTimeout, SendTimeout, ReceiveTimeout), there is no configuration for the server time-out.

One way to do that is to create a behavior that can be added to the WCF operation and that controls the time they consume to complete their jobs by interrupting them if they exceed the given time.

The Solution

The solution would be if we can intercept the call and check how long it takes to complete, and cancel it if it takes more than specified configured time.

What I thought of is having IOperationBehavior to do that for me utilizing the IOperationInvoker, to take control over the invocation.

As for the asynchronous calls, I would not alter them; let them continue in the same original way.
My changes are to the synchronous calls, each call I am starting a new task to execute it, while I watch the elapsed time, if it exceeds a certain time, it will get interrupted and end the whole operation.

Using the Code

The most important part of the code is how to make a task time out after a certain point of time.

For me, this was implemented by the following:

public class TimeoutOperationInvoker : IOperationInvoker
   {
       IOperationInvoker originalInvoker;
 
       public TimeoutOperationInvoker(IOperationInvoker invoker)
       { originalInvoker = invoker; }
       public object[] AllocateInputs()
       {
           return originalInvoker.AllocateInputs();
       }
 
       public object Invoke(object instance, object[] inputs, out object[] outputs)
       {
           var waitTolkenSource = new CancellationTokenSource();
           var tolkenSource = new CancellationTokenSource();
           object[] objectOutputParameter = outputs = null;
           Func<object> x = delegate() { return originalInvoker.Invoke(instance, inputs, out  objectOutputParameter); };
           var WcfTimeoutInMilliseconds = (int)(WcfTestService.Properties.Settings.Default.WcfTimeoutInSeconds * 1000);
           var mainTask = Task.Factory.StartNew(() =>
            {
               tolkenSource.CancelAfter(WcfTimeoutInMilliseconds);
 
                var response = x.Invoke();
                waitTolkenSource.Cancel();
                tolkenSource.Token.ThrowIfCancellationRequested();
                return response;
 
            }, tolkenSource.Token);
 
           var waitTask = Task.Factory.StartNew(() =>
           {
               var cancelled = waitTolkenSource.Token.WaitHandle.WaitOne(WcfTimeoutInMilliseconds);
               if (mainTask.Status == TaskStatus.Running)
               {
                   tolkenSource.Cancel();
               }
               return string.Empty;
           }, waitTolkenSource.Token);
 
 
           waitTolkenSource.Token.WaitHandle.WaitOne(WcfTimeoutInMilliseconds );
           Task.WaitAny(mainTask,waitTask);//
           if (mainTask.Status.HasFlag(TaskStatus.RanToCompletion))
           {
               var result = mainTask.Result;
               outputs = objectOutputParameter;
               return result;
           }
           else
           {
               tolkenSource.Token.ThrowIfCancellationRequested();
               throw mainTask.Exception;
           } 
       }
 
       public IAsyncResult InvokeBegin(object instance, object[] inputs, AsyncCallback callback, object state)
       {
           return this.originalInvoker.InvokeBegin(instance, inputs, callback, state);
       }
 
       public object InvokeEnd(object instance, out object[] outputs, IAsyncResult result)
       {
           return this.originalInvoker.InvokeEnd(instance, out outputs, result);
       }
 
       public bool IsSynchronous
       {
           get { return this.originalInvoker.IsSynchronous; }
       }
   }

Now, let's see the actual invoker that does nothing but overriding the synchronous requests, because asynchronous ones won't suffer from the time out issue.

public class WcfTimeoutOperationBehavior : Attribute, IOperationBehavior
    {
        public void AddBindingParameters(OperationDescription operationDescription, 
        System.ServiceModel.Channels.BindingParameterCollection bindingParameters)
        {
 
        }
 
        public void ApplyClientBehavior(OperationDescription operationDescription, 
        System.ServiceModel.Dispatcher.ClientOperation clientOperation)
        {
 
        }
 
        public void ApplyDispatchBehavior(OperationDescription operationDescription, 
        System.ServiceModel.Dispatcher.DispatchOperation dispatchOperation)
        {       
            dispatchOperation.Invoker = new TimeoutOperationInvoker(dispatchOperation.Invoker); 
        }
 
        public void Validate(OperationDescription operationDescription)
        {
 
        }

The service helper is very important on the client side, the purpose of it is to make sure that the client (proxy) won't suffer because the service hangs and stops responding when the channel is faulted. The service helper would be like this (also check the references at the end of this tip).

    public static void Using<TService>(Action<TService> action)
        where TService : ICommunicationObject, IDisposable, new()
    {
        var service = new TService();
        bool success = false;
        try
        {
            action(service);
            if (service.State != CommunicationState.Faulted)
            {
                service.Close();
                success = true;
            }
        }
        finally
        {
            if (!success)
            {
                service.Abort();
            }
        }
    }
}

The self hosted service and the client can be like this. You can run your code to test it.

static void Main(string[] args)
       {
           using (var host = new ServiceHost(typeof(WcfTestService.Service1)))
           {
               host.Open();
 
               int input = 0;
               var sw = new System.Diagnostics.Stopwatch();
               while (true)
               {
                   string response = null;
                   sw.Start();
                   Console.WriteLine("Input is :{0}", input);
                   try
                   { 
                       ServiceHelper.Using<wcfclient.client.service1client>(client =>
                           {
                               response = client.GetData(input);//
                               Console.WriteLine("Response {0}", response);
                           });
                   }
                   catch (FaultException<exceptiondetail> ex)
                   {
                       var tmp = Console.ForegroundColor;
                       Console.ForegroundColor = ConsoleColor.Red;
                       if (null == ex.Detail.InnerException) Console.WriteLine(ex.Message);
                       else Console.WriteLine(ex.Detail.InnerException.Message);
                       Console.ForegroundColor = tmp;
                   }
                   finally
                   {
                       input += 100;
                       Console.WriteLine("Response took {0} milliseconds", sw.ElapsedMilliseconds);
                       sw.Reset();
                   }
 
                   if (Console.ReadKey().Key == ConsoleKey.Escape)
                       break;
               }
               Console.WriteLine("Application is terminating, Please wait...");
               host.Close();
           }
       }

Finally the configurations:

<system.servicemodel>
    <services>
      <service name="WcfTestService.Service1">
        <host>
          <baseAddresses>
            <add baseaddress="http://localhost/WcfTestService">
          </baseAddresses>
        </add></host>
        <endpoint contract="WcfTestService.IService1" binding="basicHttpBinding" />
      </service>
    </services>
    <behaviors>
      <servicebehaviors>
        <behavior>
          <servicemetadata httpsgetenabled="true" httpgetenabled="true">
          <servicedebug includeexceptiondetailinfaults="true">
        </servicedebug></servicemetadata></behavior>
      </servicebehaviors>
    </behaviors>

Concerns and Alternative

The sample shows the basic idea of aborting the call when it exceeds a certain period of time; it will need some enhancements and many tests to see how it works under heavy loads and different requests in real production scenarios.

One of my major concerns was that concept depends on TPL and it will create 2 more threads of executions added to the main calling thread, and that happens for each and every call, which is a major concern in terms of impacting the server’s memory and also which questions the overhead that this solution would bring along.

It is a common WCF problem that faulted channel will hand and stop to respond. That is where the solution provided by Damien McGivern (see the reference below) comes in the scene as the sought solution for this problem.

A much simpler way to avoid that overhead is to embark the OperationContext to manage the state of the service or the operation to be more specific and check the time it is taking multiple times in its invocation, this solution works great without any impact or overheads that threads would add, but it will not work if you can’t break in the call to multiple lines of code in which you can check the timeout.

Unfortunately, we can’t apply operation behaviors in the configuration file, unlike service and endpoint ones. Those can be controlled from the configuration by extending BehaviorExtensionElement, the reason is that there are explicit configurations for all the services and all the endpoints of each but not for any of their operations where you can apply extensions.

The Result

Here you can see the result. The client will send requests, each request will be delayed 100 milliseconds and the timeout configuration is 1 second. Once the response takes 1000 milliseconds or more, it will immediately abort and return without any delay, saving the server resources.

References

• WCF CommunicationObjectFaultedException " cannot be used for communication because it is in the Faulted state" MessageSecurityException "An error occurred when verifying security for the message"
• WCF and service-side Timeouts