Safe Method Caller -- A Defensive Programming Technique

• Download source code - 2.75 KB

Introduction

This article is part of a series of articles covering the topic of "Defensive Programming". If anyone has any good stories or techniques that they've developed regarding defensive programming, I'd love to hear them.

There are numerous times when we write code that communicates with a service or technology that, by nature of the connection or the robustness of the technology, can fail. For example, any call to a database is subject to failure--the connection might fail, the task we are trying to perform may generate an error, or there may simply be a bug in the database that ends up throwing an exception or worse, never return. Another example--the user may inadvertently make a request to a service that ends up taking far too long to run. These are the kinds of scenarios that this article helps the programmer deal with, by wrapping the call so that the application has a chance to gracefully recover from call exceptions, timeouts, and errors.

The Safe Method Caller

I've named this a safe method caller because the service calls the desired method in a "safe" manner. A call can fail in the following ways:

• An exception is thrown
• The call never returns
• The call returns with an error (rather than exception)

There are, of course, other kinds of failures--wrong return value, incorrect actions taken, and so forth. This code is not intended to address those failure cases. The safe method caller protects the application from the three cases listed above.

Implementation

The heart of the service is the ManualCall method. This method can be used to make a single "protected" call using the method caller service, though in typical applications, I end up describing an execution workflow which I provide in the SafeMethodCaller constructor. For the purposes of this discussion though, let's look at the ManualCall method:

Input Parameters

public bool ManualCall(
    MethodDlgt fnc, 
    int msExpectedResponse, 
    CriticalFailureDlgt criticalFailureFnc, 
    FailureDlgt failureFnc, 
    TimeoutFailureDlgt timeoutFnc
    )

First off is the method signature itself. It accepts:

• a delegate for the method (the callee) you wish called by the service
• a timeout value in milliseconds
• a method that can be called if the callee throws an exception
• a method that can be called if the callee returns a failure state
• a method that can be called if the callee times out

The signature for the callee is:

public delegate bool MethodDlgt();

This signature can be a bit constraining--it returns a bool to indicate success (true) or failure (false), and doesn't take any parameters. It is therefore typical that your application will provide a method that interfaces between this service and the actual method to be called. The interface (not meant as a C# interface, but the abstract concept of interfacing between two things) method will provide the appropriate parameters the technology requires, and will return a bool based on possibly a more complicated error code returned by the technology. The interface method does not need to catch any exceptions--that's what this service does.

The exception, failure, and timeout handler delegates have the following signatures, respectively:

public delegate void CriticalFailureDlgt(Exception e);
public delegate void FailureDlgt();
public delegate bool TimeoutFailureDlgt();

To process exceptions thrown by the callee, you would provide a method that takes an Exception object from which you can manage the exception. A failure method takes no arguments, and a timeout failure returns true if you want the service to try the call again.

And last, the ManualCall method returns true if the call succeeded, false otherwise (exception, timeout, or error results).

Making the Call

Internally, the service uses a separate thread to initiate the call. This lets the service monitor when the call returns, and to timeout if the call hasn't returned within the specified time. However, first the service has to ensure that the thread even started:

string strFnc = GetMethodInfo(fnc);
// Assume success.
bool success = true;
// Create a method call package.
MethodPackage mp = new MethodPackage(fnc);
int n = threadStack.Count;
// Call our wrapper.
IAsyncResult ares = InternalCall.BeginInvoke(mp, null, null);
bool threadStartTimeout = false;
// Wait for function to even begin execution.
WaitForThreadStartup(n, strFnc, out success, out threadStartTimeout);

The callee is packaged up and the BeginInvoke method is used to make the call on a separate thread. WaitForThreadStartup waits until the new thread places itself on the thread stack:

/// <summary>
/// Wraps a try-catch block around the call so we can trap exceptions and 
/// call the critical failure callback.
/// </summary>
internal void InternalCallMethod(MethodPackage mp)
{
  lock (threadStack) 
  {
    threadStack.Push(Thread.CurrentThread);
  }

  try
  {
    bool ret = mp.Fnc();
    mp.Result = ret;
  }
  catch (Exception e)
  {
    mp.Exception = e;
  }
}

Waiting for the Call to Complete

Once the thread making the call starts, the service waits until the call returns or a timeout occurs:

// Get the async thread.
if (!threadStartTimeout)
{
  Thread asyncThread = threadStack.Pop();

  // The async callback starts now.
  DateTime then = DateTime.Now;

  // While the async method is running...
  while (!ares.IsCompleted)
  ...

Often, I'm debugging code that has been called by this service, and I don't want the execution to fail because I took longer stepping through the code than the timeout interval. A neat little trick is to test whether the code is being run in the debugger and reset the time:

if (Debugger.IsAttached)
{
  // Do not use timeouts when debugging.
  then = DateTime.Now;
}

When the Call Times Out

When the call times out, the timeout handler has the opportunity to tell the service to retry the call; otherwise, the service attempts to terminate the thread:

if (ts.TotalMilliseconds >= msExpectedResponse)
{
  Debug.WriteLine("!async:Timeout on " + GetMethodInfo(fnc));
  // Call the optional timeout callback.
  bool retry = Timeout(timeoutFnc);
  Debug.WriteLine("!async:Timeout return = " + retry.ToString());

  if (retry)
  {
    then = DateTime.Now;
  }
  else
  {
    // Call failed.
    try
    {
      // kill the thread that timed out.
      Debug.WriteLine("!async:Thread terminating for " + GetMethodInfo(fnc));
      asyncThread.Abort();
      Debug.WriteLine("!async:Thread terminated.");
    }
    catch (Exception e)
    {
      Debug.WriteLine("!async:" + e.Message);
    }

    success = false;
    break;
  }
}

Callee Returns Failure

If the callee (as mentioned above, you will usually need to implement a wrapper that tests the actual method's return code and returns true/false to indicate success/failure) returns with a failure, the ResultFailure handler (which you supply) is called:

if (!mp.Result)
{
  Debug.WriteLine("!async:Result false on " + GetMethodInfo(fnc));
  // If the async callback returned failure,
  // call the optional failure callback.
  ResultFailure(failureFnc);
  // Call failed.
  success = false;
}

Callee Threw an Exception

If the callee throws an exception, your supplied exception handler is called:

if (mp.HasCriticalError)
{
  Debug.WriteLine("!async:Critical error on " + GetMethodInfo(fnc));
  Debug.WriteLine("!async:" + mp.Exception.Message);
  Debug.WriteLine("!async:" + mp.Exception.StackTrace);
  // Call the optional critical failure callback.
  CriticalFailure(criticalFailureFnc, mp.Exception);
  // Call failed.
  success = false;
}

Usage

As usual, the best way to describe the usage is with unit tests. These unit tests use the following setup:

[TestFixture]
public class SafeMethodCallTests
{
  protected SafeMethodCaller caller;
  protected bool timeout;
  protected bool failure;
  protected bool critical;

  [TestFixtureSetUp]
  public void TestFixtureSetup()
  {
    caller = new SafeMethodCaller();
  }
  ...

A Successful Call Test

This test makes a call to a method which simply returns true, indicating success:

[Test]
public void WithinTimeTest()
{
  timeout = false;
  failure = false;
  critical = false;
  caller.ManualCall(SuccessMethod, 20, Critical, Failure, Timeout);
  Assertion.Assert(!timeout, "Did not expect timeout.");
  Assertion.Assert(!critical, "Did not expect critical callback.");
  Assertion.Assert(!failure, "Did not expect failure callback.");
}

This test (and most of the others) illustrate calling a single method with the service.

Timeout Test

This test verifies that the service handles a method that takes longer than the specified 20ms timeout:

[Test]
public void ExceedsTimeTest()
{
  timeout = false;
  failure = false;
  critical = false;
  caller.ManualCall(TakesTooLongMethod, 20, Critical, Failure, Timeout);
  Assertion.Assert(timeout, "Expected timeout.");
  Assertion.Assert(!critical, "Did not expect critical callback.");
  Assertion.Assert(!failure, "Did not expect failure callback.");
}

The callee:

protected bool TakesTooLongMethod()
{
  System.Threading.Thread.Sleep(50);

  return true;
}

takes 30ms too long to return, resulting in the timeout method being called:

protected bool Timeout()
{
  timeout = true;

  return false;
}

which, by returning false, indicates that the service should not retry the call.

Call Failure Test

This test simulates the callee returning an error code, which the wrapper method signals to the service by returning false:

[Test]
public void FailureTest()
{
  timeout = false;
  failure = false;
  critical = false;
  caller.ManualCall(FailureMethod, 20, Critical, Failure, Timeout);
  Assertion.Assert(!timeout, "Did not expected timeout.");
  Assertion.Assert(!critical, "Did not expect critical callback.");
  Assertion.Assert(failure, "Expected failure callback.");
}

protected bool FailureMethod()
{
  return false;
}

Call Exception Test

This test verifies that an exception is handled by the service:

[Test]
public void ExceptionTest()
{
  timeout = false;
  failure = false;
  critical = false;
  caller.ManualCall(ExceptionMethod, 20, Critical, Failure, Timeout);
  Assertion.Assert(!timeout, "Did not expected timeout.");
  Assertion.Assert(critical, "Expected critical callback.");
  Assertion.Assert(!failure, "Did not expect failure callback.");
}

protected bool ExceptionMethod()
{
  throw new ApplicationException("This is an async exception.");
}

Execution Sequence Test

One of the common things I do is to execute a sequence of calls to one or several technologies. This test verifies execution sequencing, and introduces the AddCalls and ExecuteAll methods:

[Test]
public void ExecuteAllTest()
{
  caller.ClearCalls();
  caller.AddCalls(new CallInfo[]
  {
    new CallInfo(Call1, 10, null, null, null),
    new CallInfo(Call2, 10, null, null, null),
  });

  bool ret = caller.ExecuteAll();
  Assertion.Assert(ret, "Expected ExecuteAll to return true.");
}

The service calls each method in sequence. Any call that fails aborts the sequence.

Execution Sequence Failure Test

This test verifies that the service returns a false when running an execution sequence:

[Test]
public void ExecuteAllFailTest()
{
  caller.ClearCalls();
  caller.AddCalls(new CallInfo[]
  {
    new CallInfo(Call1, 10, null, null, null),
    new CallInfo(FailureMethod, 10, null, null, null),
    new CallInfo(Call2, 10, null, null, null),
  });

  bool ret = caller.ExecuteAll();
  Assertion.Assert(!ret, "Expected ExecuteAll to return false.");
}

About the Download

The download includes only the file SafeMethodCaller.cs.

Conclusion

I have used this service in conjunction with Tiered Error Recovery, to easily manage the complex error recovery and to ensure that the application keeps running even in the face of failing and buggy third party technologies.