Bi-directional HTTP Connection

• Download source - 19.6 Kb

Abstract

Implementing a peer to peer application requires either end to be set up as both a client and a server. The simplest approach would be to let the parties connect to each other at their respective end-points. But this approach does not usually work if the parties are separated by a fire-wall where the network administrator is usually reluctant to permit outgoing connections. So, maintaining and reusing an established connection for bi-directional traffic is the only option.

The essential concept

To accomplish this, we must distinguish between an incoming request and an incoming response, explained like this. In one case, we expect the other party to send us a request to which we simply return a response, and in the other case it is we who shall send a request and wait for the other party to send us a response. We shall make this distinction with the help of out-of-band information that we can pass along, employing the HTTP protocol.

So, we shall communicate peer to peer using HTTP messages. Once the listening socket accepts a connection, we will assume and completely receive an HTTP message which may then be easily identified and processed as either a request or a response. Here is some code to illustrate the basic idea:

// create and start the listener
TcpListener listener = new TcpListener(7070);
listener.Start();

// wait and accept a client connection
Socket socket = listener.AcceptSocket();

// create an HTTP message object and receive the HTTP message
HttpMessage msg = new HttpMessage();
msg.Receive(_socket);

// process the HTTP message as either a request or a response
if(msg.IsResponse)
  ProcessResponse(msg);
else
  ProcessRequest(msg);  

The HttpMessage object simply reads from the socket stream the essential parts of the HTTP protocol, the first line, the list of optional headers and the message body. A distinction between request and response is made by inspecting the first token in the first line. In the case of a response, the first line must start with "HTTP". That is what msg.IsResponse ascertains.

Our keenest interest is about the method ProcessResponse(msg). How do we co-relate a response to a request we have previously made? Let us examine the method of sending a request.

void SendRequest(Socket socket, HttpMessage msg)
{
  // create a correlation id and assign it to the message
  msg.CorrelationID = Guid.NewGuid().ToString();

  // send the message
  msg.Send(socket);

  // retain the message for future reference in a hash table
  _requests[msg.CorrelationID] = msg;
}

The central idea is to create and attach a unique identifier to the outgoing HTTP message. We expect that this identifier also be present in the returning HTTP response message. Here is an illustration of the essential HTTP protocol exchange.

// outgoing request
GET / HTTP/1.1
Correlation-ID: 0B83745D-2AAB-4bce-8AC9-B8A590F07768

// incoming response
HTTP/1.1 200 Ok
Correlation-ID: 0B83745D-2AAB-4bce-8AC9-B8A590F07768

We can now turn our attention to the method ProcessResponse(msg). Here is the model:

void ProcessResponse(HttpMessage response)
{
  // use the correlation id to match the response to the request
  HttpMessage request = _requests[response.CorrelationID];
  
  // use the request and the response for further processing
}

A bi-directional communication manager

The central idea of the bi-directional communication has now been described. Let us proceed to develop a module that we can practically deploy. What we want is a class that can manage the details of a bi-directional communication and that we may deploy like so:

// create and start the listener
TcpListener listener = new TcpListener(7070);
listener.Start();

while(true)
{
  // wait for and accept a client connection
  Socket socket = listener.AcceptSocket();
  
  // create the connection object that will
  // manage the bi-directional communication
  Connection conn = new Connection(socket);
  
  // spawn a worker thread for this connection
  // and loop back to accept another client
  new Thread( new ThreadStart(conn.ThreadProc) ).Start();
}

The class Connection is responsible for managing the bi-directional communication. We pass to it the accepted socket and rely upon the connection object to use that same socket for sending and receiving HTTP messages. In order to wait for and receive additional connections, we spawn a worker thread to manage the established connection. Here is the connection's thread procedure:

// the connection object's thread procedure
void ThreadProc()
{
  while(Continue())
  {
    // create and receive the HTTP message
    HttpMessage msg = new HttpMessage();
    msg.Receive(_socket);
    
    // process the message as either a response or request
    if(msg.IsResponse)
      ProcessResponse(msg);
    else
      ProcessRequest(msg);
  }
}

The code inside the thread procedure should be familiar. Let us re-examine the method of sending a request. We would like to send a request and synchronously wait for the response like so:

HttpMessage request = new HttpMessage();
request.Verb = "GET";
request.RequestUri = "/";
request.Version = "HTTP/1.1";

HttpMessage response = conn.SendMessage(request);

This implies that the method SendMessage(request) must wait until the response has been received. We need a way to signal the arrival of the response. The best approach to this problem is to implement the complementary asynch methods, BeginSendMessage and EndSendMessage.

public IAsyncResult BeginSendMessage(HttpMessage request)
{
  // create a correlation id and assign it to the message
  request.CorrelationID = Guid.NewGuid().ToString();

  // send the request on the assigned socket
  request.Send(_socket);
  
  // create the waitable object
  IAsyncResult async = new HttpAsyncResult();
  
  // map the correlation id to the waitable object
  _requests[request.CorrelationID] = async;
  return async;
}

public HttpMessage EndSendMessage(IAsyncResult async)
{
  // wait until the signal is set, e.g the response has been received
  if(!async.IsCompleted)
    async.AsyncWaitHandle.WaitOne();

  // get the matching response
  HttpMessage response = (HttpMessage)_requests[async];
  
  // clear the requests table
  _requests.Remove(async);
  
  // return the response
  return response;
}

Before discussing this code, let us show how the synchronous version is implemented. It is very simple.

public HttpResponse SendRequest(HttpRequest request)
{
  IAsyncResult async = BeginRequest(request);
  return EndRequest(async); 
}

Let's discuss the asynchronous version. We are mapping the correlation ID of the outgoing message to a waitable object that implements the IAsyncResult interface. Obviously, the waitable object will need to be set, the moment the response to the outgoing message arrives. That must happen inside the ProcessResponse method. Here is its implementation:

void ProcessResponse(HttpMessage response)
{
  // use the correlation id to get the waitable object
  HttpAsyncResult async = (HttpAsyncResult)_requests[response.CorrelationID];
  
  // remove the correlation id from the requests table
  _requests.Remove(response.CorrelationID);
  
  // map the waitable object to the response
  _requests[async] = response;
  
  // set the signal so that the response becomes availabe
  async.Set();
}

You need to closely compare the methods EndSendMessage and ProcessResponse. It is understood that once we have send a request, we must wait until the response has arrived.

Now, let's turn our attention to the case where we need to process an incoming request, as per ProcessRequest. Our Connection here is firstly about managing a bi-directional communication. So, it only makes sense to delegate the processing of the HTTP request to some external agent. We can best accomplish it by defining an appropriate delegate: public delegate HttpMessage ProcessRequestDelegate(HttpMessage request);. So, here is the simple implementation of the PrecessRequest method.

// delegate member
public ProcessRequestDelegate DelegateRequest;

// private method
void ProcessRequest(HttpMessage request)
{
  // let an external agent process the request
  HttpMessage response = DelegateRequest(request);
  
  // Take the response and set the correlation id
  response.CorrelationID = request.CorrelationID;
  
  // now send the response on the assigned socket
  response.Send(_socket);
}

We just want to be sure that request processing is accomplished in parallel. We cannot afford to wait as we seek to be ready for any incoming HTTP message. Let us therefore refine the ProcessRequest method.

// temporary queue for storing the request
Queue _queue = Queue.Synchronized( new Queue() );

void ProcessRequest(HttpMessage request)
{
  // store request temporarily in a queue
  _queue.Enqueue(request);
  new Thread( new ThreadStart(this.ProcessRequestThreadProc) ).Start();
}

// delegate member
public ProcessRequestDelegate DelegateRequest;

// private method
void ProcessRequestThreadProc()
{
  // get request from temporary queue
  HttpMessage request = (HttpMessage)_queue.Dequeue();
  
  // let an external agent process the request
  HttpMessage response = DelegateRequest(request);
  
  // Take the response and set the correlation id
  response.CorrelationID = request.CorrelationID;
  
  // now send the response on the assigned socket
  response.Send(_socket);
}

Using the source code

You may download the source and try it out. The zip file contains a Visual Studio 2003 solution. The connection manager rests in a separate assembly, 'Connection'. In addition, there is a 'client' and a 'server' project for you to try.