Contents
- Introduction
- Probe Internals
- Concept and Design
- Interface contract
- What is published
- Publisher
- Checkpoints
- Plug-in
- Installation
- Usage
- Subscribers
- SoapProbeSubscriber2
- WebService Analyzer Studio
- Analyzing First Sample
- Test environment
- Conclusion
Consuming a Web Service in distributed .NET applications is fully transparent using the web service client proxy. When the client invokes a method on the proxy, the SOAP infrastructure packs all information into the message SoapClientMessage and passes it through the communication network to the web server, where its ASP.NET dispatcher forwards the message to the proper worker process. This worker process (aspnet_wp.exe) unpacks the message to obtain the information for invoking a method on the web service. The result of the SOAP call goes back to the caller in the same manner using the message SoapServerMessage.
The SOAP Messages travel through the SOAP infrastructure located on both ends - client and sever. Using the SOAP extension feature makes it easy to plug-in a custom process for each processing stage and publish its state.
Subscribing to the WebService Probes, we can obtain a knowledge base of the business workflows, which can be used for their post-processing tasks such as monitoring, analyzing, modeling, simulation, tuning, error catch tracer (like an aircraft black box) etc.
This article describes how to design and implement the loosely coupled SoapMessage Publisher (called as the WebService Probe) using the SOAP extensions and COM+ Event System notifications. Also I will give you a lite tool - WebService Analyzer Studio to see how to subscribe and use the Probe notifications and business workflows between the web services and their consumers. For instance, you can catch the consumer call - HttpRequest and then simulate it many times from the Analyzer. It is a useful SOAP tool for any stage of the development and production environment.
Let's look in-depth at the WebService Probe. I am assuming that you have a knowledge of the SOAP and Web Service infrastructure. Also, have a look at the similarl concept of the Probe for remoting, described in my previous article [1].
The concept of loosely coupled message notifications is based on using the COM+ Event Publisher/Subscriber design pattern, where the event call is fired in the proper place of the SOAP Extension. The following picture shows this:
The SOAP ends exchange information using the SoapClientMessage and SoapServerMessage respectively in the Request/Response design pattern. ASP.NET has an extensibility mechanism for calling XML web services using SOAP built-in. This feature is based on overriding the SoapExtension methods to intercept the call at specific stages in the message processing on either the client or the server side.
The message can be intercepted in the following processing stages:
BeforeSerialize AfterSerialize BeforeDeserialize AfterDeserialize
The Probe uses the AfterSerialize and AfterDeserialize stages to publish the SOAP request messsage and SOAP response message at the ends. The publisher fires the events all the time, without any knowledge of the subscriber in the loosely coupled manner. To stop the publishing it is necessary to unregister the Probe from the web service.
Note that the SoapExtension object life is short; only during the Before and After message processing stages at the same end. The SoapExtension object is initiated each time when the SOAP Message has been created. According to the above picture, there are four different instances of the SoapExtension, actually each end only creates two of them. This is absolutely correct when we are handling a web service - see more details at the [2].
The Probe uses the internal SOAP header _SoapProbeTicket as a passport to travel through all the processing stages at the both ends. This is a logical header per each call (sample session) and it holds the message ID (GUID).
The Probe only fires a local event call (that's the limitation of the COM+ Event Service, where an event class and their subscribers have to be on the same machine). Using a persistent subscriber such as SoapProbeSubscriber (included in this solution), the event call can be forwarded to the Enterprise subscriber using .NET Remoting.
The contract between the Publisher and Subscriber is represented by the SoapEventClass, which is an event meta class derived from the ISoapProbe interface.
[Guid("927B319E-BF79-429b-8CA0-118FBF9724CD")]
public interface ISoapProbe
{
[OneWay]
void WriteLogMessage(
DateTime timestamp,
string strSource,
string strDest,
string strAction,
string strMsgType,
string strMsgId,
object objMsgBody);
}
[Guid("C515BB2F-664B-4afa-A40F-CD15D119C7E8")]
[EventClass]
[Transaction(TransactionOption.NotSupported)]
[ObjectPooling(Enabled=true, MinPoolSize=25, MaxPoolSize=100)]
[EventTrackingEnabled]
public class SoapEventClass : ServicedComponent, ISoapProbe
{
public void WriteLogMessage(
DateTime timestamp,
string strSource,
string strDest,
string strAction,
string strMsgType,
string strMsgId,
object objMsgBody
)
{
throw new Exception("This class can be called only by the COM+ Event System");
}
The contract has only one method passing the SOAP Message contents and additional info about the source to the subscriber(s). This method is [OneWay] attributed, which is important for the remoting subscriber.
The Probe publishes a SAOP Extension state using the interface contract via the WriteLogMessage method with the following arguments, where:
-
timestamp is the checkpoint DateTime
-
strSource is the identifier of the stage, action, error, etc.
-
strDest is the url address of the web service
-
strAction is the web method
-
strMsgType is the type of the event call. There are the following types generated by the Probe:
public class SoapMessageType
{
public const string ClientReq = "0-SoapClientRequest";
public const string ServerReq = "1-SoapServerRequest";
public const string ServerRsp = "2-SoapServerResponse";
public const string ClientRsp = "3-SoapClientResponse";
public const string ClientReqErr = "0-SoapClientRequest_Err";
public const string ServerReqErr = "1-SoapServerRequest_Err";
public const string ServerRspErr = "2-SoapServerResponse_Err";
public const string ClientRspErr = "3-SoapClientResponse_Err";
}
-
strMsgId is the GUID of the session call. Each call has own unique guid generated by the probe and using for all messages related with this call. It's travels in the SoapProbeTicket header.
-
objMsgBody represents the log message body. It's a dictionary of the HttpHeaders and SOAP envelopes.
The following code snippet shows how objMsgBody is constructed:
private void Write(string source, string msgType, string url, string action)
{
string[] headers = null;
HttpContext hc = HttpContext.Current;
if(hc != null)
{
NameValueCollection nvc = HttpContext.Current.Request.Headers;
headers = new string[nvc.Count];
for(int ii=0; ii<nvc.Count; ii++)
{
headers[ii] = nvc.GetKey(ii) + "=" + nvc[ii];
}
}
newStream.Position = 0;
StreamReader sr = new StreamReader(newStream);
string strSoap = sr.ReadToEnd();
newStream.Position = 0;
Hashtable htSoapMsg = new Hashtable();
htSoapMsg[SoapMessageBody.HttpHeaders] = headers;
htSoapMsg[SoapMessageBody.SoapStream] = strSoap;
delegatePublisher delegator = new delegatePublisher(Publisher);
delegator.BeginInvoke(DateTime.Now, source, url, action, msgType,
m_ProbeTicket.ticketId, htSoapMsg, null, null);
}
The Probe Publisher is simply and straightforward code. Its responsibility is to initiate the Event class and invoke the contract method on that class. The following code snippet shows this:
private void Publisher(
DateTime timestamp,
string strSource,
string strDest,
string strAction,
string strMsgType,
string strMsgId,
object objMsgBody
)
{
try
{
using(SoapEventClass sec = new SoapEventClass())
{
sec.WriteLogMessage(timestamp, strSource, strDest, strAction, strMsgType,
strMsgId, objMsgBody);
}
}
catch(Exception ex)
{
Trace.WriteLine(string.Format("{0}[{1}]:Publisher catch = {2}",
this.GetType().FullName, this.GetHashCode(),
ex.Message));
}
}
Note that the Publisher runs in the background process using the delegate design pattern - see the previous code snippet implementation. This solution allows us to isolate the publisher from the subscribers. Because the publishers (worker threads) are running in the thread pool, there is no guarantee of their order in which they have been generated. It means that the Response message can be received before its Request one. So, the contents of the log message such as timestamp, id, and type are efficient to identify their order.
Basically, there are four checkpoints suitable to publish their state marked as 0, 1, 2 and 3. The checkpoint calls the Publisher to make an event call to the Event System in the fire-forget manner.
The Probe fires the following events;
- 0 - SoapClientRequest
- 1 - SoapServerRequest
- 2 - SoapServerResponse
- 3 - SoapClientResponse
in the AfterSerialize and AfterDeserialize stages of the message processing. The following code snippet shows the checkpoints in the SoapExtension class:
public override void ProcessMessage(SoapMessage message)
{
switch (message.Stage)
{
case SoapMessageStage.BeforeSerialize:
SoapTicket(message);
break;
case SoapMessageStage.AfterSerialize:
WriteOutput(message);
break;
case SoapMessageStage.BeforeDeserialize:
Copy(oldStream, newStream);
newStream.Position = 0;
break;
case SoapMessageStage.AfterDeserialize:
SoapTicket(message);
WriteInput(message);
break;
default:
throw new Exception("invalid stage");
}
}
The Probe can be plugged into the SOAP infrastructure performing in one of the following ways:
-
programmatically using the SoapProbeAttribute for the specified web method. In this case the client and server sides need a reference to the SoapMessageProbe assembly. This is a tight design pattern and hard coded solution. Note that this solution is suitable when only a specified web method on the server is going to be analyzed.
-
administratively using the config file and <soapExtensionTypes> tag in the <webServices> config section.
<configuration>
<system.web>
<webServices>
<soapExtensionTypes>
<add type="RKiss.SoapMessageProbe.SoapProbe, SoapMessageProbe,
Version=1.0.1002.16362,Culture=neutral,
PublicKeyToken=719b2bb9abf58c2d" priority="0" group="0" />
</soapExtensionTypes>
</webServices>
</system.web>
</configuration>
There are two possibilities using the <soppExtensionTypes> in the config file. The first place is to modify the machine.config file, which it will automatically plugged the Probes for all web services on the box. The other option is to modify only an application config file such as the .exe.config or web.config files. I am recommending to use this option, it's controllable and loosely coupled with the application. See the Analyzer solution, where the Probes can be registered/unregistered to/from the specified web service on the fly.
In the both cases the SOAP infrastructure requires the type of the SOAP extension and access to its assembly (the best way is to install the Probe's assemblies into the GAC). The Probe doesn't have any config properties to be initialized during the c'tor time. It's a stateless object following the web service rules.
Note that the changes made in the .exe.config file will take effect after restarting its application. In the other applications such as ASP.NET where a web.config file is used, it's not necessary to manually restart the application - the web server will automatically refresh it.
The WebServiceProbe solution consists of the following projects:
- SoapMessageProbe.dll: this is the assembly of the SOAP extension
- SoapProbeEventClass.dll: this is an assembly of the event meta class and interface contract
- SoapProbeSubscriber2.dllthis is an assembly of the persistent subscriber to distribute the event calls remotely
Additionally there are also projects using the SOAP Probe:
- SoapProbeSubscriber.dll - Analyzer subscriber
- WebServiceAnalyzerStudio.exe - GUI to explore the SOAP Messages published by the Probes
Installation steps:
-
Launch the InstallSoapProbe.bat file to perform installation of the assemblies into the GAC and registry of the Enterprise Services in the COM+ Catalog.
-
Create administratively a persistent subscription for the SoapProbeSubscriber using the COM+ Explorer and Subscription Wizard. See the following picture:
At this time, the WebService Probe is installed on your machine and ready to use it. But ..., you will see no response from the Probe until you plug the Probe into the application and its subscriber is launched (locally or remotely).
The first condition has been described earlier, now we'll look at the second one - Subscriber - more closely.
The WebService Probe usage is dependant on its Subscriber. The Probe without using any subscriber is a passive publisher to fire an event for "null". Let's look at for them:
There can be many different subscribers designed and implemented for the WebService Probes. From the simplest one, (like writing the messages to the file system) to more complicated business control loops based on the strobed business knowledge base. Some solutions can be leaded to a more sophisticated model than the main application.
Basically, the Subscribers are divided into the following two groups:
- passive, where received information is used for post-processing such as monitoring, analyzing, modelling, simulation and other off-line tools.
- active, where received information is used to control business model - its workflow such as tuning, balancing and etc. - on-line solutions.
I included one simple Subscriber to distribute the event call on the Enterprise Network using .NET Remoting. Here are some details:
This is a poolable ServicedComponent derived from the ISoapProbe interface. The object is initialized by the constructor string which represents a URL address of the remoting server object. Based on this string the remoting channel is registered either programmatically or administratively using the config file. Note that this component is activated as a server, so its host process is dllhost.exe and that's the reason why the config file is located in the %WINDIR%\system32 folder.
The Event method received the call from the Probe publisher and then forwarding it to the remoting infrastructure. It looks like a small bridge between the event system and remoting. Here is its implementation:
#region ISoapProbe
[OneWay]
public void WriteLogMessage(
DateTime timestamp,
string strSource,
string strDest,
string strAction,
string strMsgType,
string strMsgId,
object objMsgBody
)
{
try
{
DelegateWriteLogMessage sn = new DelegateWriteLogMessage(onNotify);
sn.BeginInvoke(timestamp, strSource, strDest, strAction, strMsgType,
strMsgId,objMsgBody, null, null);
}
catch(Exception ex)
{
string msg = string.Format("Subscriber2.WriteLogMessage failed. Error={0}",
ex.Message);
Trace.WriteLine(msg);
}
}
private void onNotify(
DateTime timestamp,
string strSource,
string strDest,
string strAction,
string strMsgType,
string strMsgId,
object objMsgBody
)
{
try
{
ISoapProbe robj = (ISoapProbe)Activator.GetObject(typeof(ISoapProbe),
strTargetUrl);
robj.WriteLogMessage(timestamp, strSource, strDest, strAction,
strMsgType, strMsgId, objMsgBody);
}
catch(Exception ex)
{
string msg = string.Format("Subscriber2.onNotify failed. Error={0}",
ex.Message);
Trace.WriteLine(msg);
}
}
#endregion
As you can see, the call spawns the worker thread to invoke the remoting method. Note that the remoting method is an [Oneway] attributed, so there is no waiting time for its response.
As a target object any remoting object derived from ISoapProbe using the standard (tcp/http) or custom channels such as MSMQ, WebService, etc. can be used: see my articles [3] and [4].
How powerful is the SOAP details published by the Probe is shown in my included solution called WebService Analyzer Studio? Here are more details:
The WebService Analyzer Studio is the GUI Subscriber to the WebService Probe(s) to explore the SOAP messages between the web services and their consumers. It has been designed and implemented (using the "as it is" methodology) to demonstrate power of the published knowledge base by SOAP Probe. It can be used for developing, testing and tracing the web service issues during the product lifecycle. Note that the Analyzer is a fully isolated, loosely coupled design pattern from your web services and their consumers.
The major features of the WebService Analyzer Studio:
-
plug-in/unplug the Probes to/from the applications and web services on the fly
-
storing the incoming samples from the Soap Probes in the TreeView layout
-
exploring the samples (source, headers, soap, etc.)
-
simulate the captured Request sample
-
timestamp measurement
-
soap message control flow
-
storing samples in the circulated manner
-
filtering samples on Error
-
forwarding incoming samples to the remoting object
-
subscribing to the Event System locally and remotely using the loosely coupled design pattern
-
customizing the Analyzer behaviour on the fly
-
store/load the customized option to/from the xml file
-
it's free
The concept of the WebService Analyzer Studio is based on exploring the incoming event calls into the TreeView layout. Basically, there are two kinds of panels with the splitter. The left panel is dedicated to the probes, simulation of the SOAP Request calls and customizing. The second one, on the right side is for storing the incoming samples: event calls into the TreeView layout for the post-processing examination.
Probes Tab
The following screen shoot shows the list of the applications where the SOAP Probe has been registered or unregistered. The Analyzer will take care to modified or created config file for this application. The original config file (if it exists) is renamed for backup purposes. The TreeView nodes are persisted into the analyzer xml file and loaded during the start up process.
Simulation Tab
The incoming Request samples (client or server) can be selected for their simulation. This selection will make a send a copy of the sample to the Simulator Tab:
Clicking on the Run selection of the pop-menu of the HttpRequest node causes the POST Request to be sent to the web service. Its SOAP Envelope is a copy of the original one. Based on this Request call, the simulator will receive HttpResponse - see the above screen shoot. This is a great feature of the Analyzer, which allows to simulate a client's request in the post-processing manner. For instance, we can simulate the Request calls which cause the exception on the web method to debug the business rules, validation, etc. in the web service.
Customize Tab
Some Analyzer properties can be customized using the following PropertyGrid control.
You can configure storing the samples into the TreeView control such as number of samples, circulation mode, filtering on error or keeping the original formatted SOAP Envelop. For instance: the Analyzer can be hooked to the web service and store the SOAP error messages only in circulated storage with a specified number of samples.
Note that this feature is opened to enhance the Analyzer option. For instance, the triggering option can wait for a specified sample and then start or stop the storing samples in the storage.
Samples panel
This right-hand panel is dedicated to storing the incoming samples into the TreeView layout based on the general or the action node (web method node) option. All nodes are read-only. The TreeView layout is organized like is shown in the following screen shoot:
The samples can be expanded to show their contents including name, schema, value, etc. The first SOAP message initiates the sample session identified by the unique ID (GUID).
Some interesting pop-menu features:
-
F2 Option - allows to modify properties
-
F7 Simulate - request to simulate this message
-
F9 Mark - mark the sample timestamp as a reference point for the time measurement between the samples. This feature allows to see timing control flow between any samples in milliseconds. It's valuable information when a performance issue is arising.
The Analyzer can be disconnected from the incoming samples by pressing the Master Switch on the status bar. The green icon indicates that the analyzer is connected to the Probe (either via the Event System or Remoting).
That's all for the WebService Analyzer Studio and I hope you will enjoy it. Now it's the time to make a first sample analyzing.
I built the TestSample solution to demonstrate capability of the WebService Analyzer Studio. You can fond it in the separate folder \Test:
- TestWebService - ASP.NET Web Service
- TestWebForm - ASP.NET Web Application (client)
- TestWinForm - Windows Application (client)
The Probe and Analyzer have been implemented using .NET SP2 and tested on Windows 2000 Advanced Server and Windows 2000 Professional. It should be work without any problem also on XP machines.
Before using this test, the ASP.NET projects have to be deployed and installed on your web server. The TestSample solution is very simple like the "Hello world". The client invokes the web methods Test or Echo with many different types of the arguments included DataSet.
First of all, it's necessary to have a functional test environment for TestSample solution without using the SOAP Probe and Analyzer. Be sure that their application config files have no plug-in of the SoapMessageProbe type in the webServices tag. Then try to make a quick test pressing the Test or Echo button on the Windows and Web Clients. After that, we can step-in to the using the SOAP Probe, (I am assuming that the SoapMessageProbe has been already installed - see Instalation).
There can be more scenarios to analyze the Web Service consuming. Depending from the Probe locations there can be the following major scenarios:
I. SOAP Probe on the client side
This is a simple scenario analyzing the web service consumer side - the Windows Application. The SOAP Probe is plugged-in to the SOAP extension of the web service client proxy. Here are the test steps:
-
Be sure that the applications such as SoapProbeEventClass and SoapProbeSubscriber are in the COM+ Catalog.
-
Launch the WebServiceAnalyzerStudio program
-
Be sure that the Analyzer is subscribed to the Event System (check the tooltip icon on the status bar)
-
Select the Probes Tab
-
Modify your Windows application path (full path name of the assembly, no extension)
-
Click on Register on the pop-menu
-
If the Registration succeeded, the Probe has been plugged-in to TestWindowsForm.exe.config.
-
In this moment, the test environment is ready to make a test
-
Press the Test or Echo Button on the client program
-
You should see two received samples by the Analyzer (Samples panel).
-
Now, the samples can be explored.
-
Select the node: 0-SoapClientRequest
-
Click Simulate on the pop-menu
-
Go to the Simulation panel and select the node: HttpRequest
-
Click the Run on the pop-menu
-
You should see the HttpResponse on the Simulation panel. Note that the Simulation using the POST/GET requests communication to the web service, so there are no samples published by the probe.
-
Repeat step 9 to capture more samples by the Analyzer.
II. SOAP Probe on the Web Server side
This scenario requires access to the Web Server and a decision where the Analyzer has to be located. There are the following choices:
-
Web Server - the solution is using the local Event System
-
Local Network - the solution is using the persistent subscription. Be sure it's enabled and the constructor string is addressed to your remote machine)
-
Web Server and Network - the solution is using the local Event System and persistent subscription
Let's select the first choice. The test steps are:
-
Be sure that the applications such as SoapProbeEventClass and SoapProbeSubscriber are in the COM+ Catalog.
-
Launch the WebServiceAnalyzerStudio program
-
Be sure that the Analyzer subscribed to the Event System (check the tooltip icon on the status bar)
-
Select the Probes Tab
-
Modify your web service and web application path (full path name of the assembly, no extension)
-
Click Register on the pop-menu for each of them Application nodes
-
If the Registration succeeded, the Probe has been plugged-in to their web.config files
-
In this moment, the test environment is ready to make a test
-
Press the Test or Echo Button on the client program
-
You should see, four received samples by the Analyzer (Samples panel).
-
Now, the samples can be explored.
-
Select the node: 0-SoapClientRequest or 1-SoapServerRequest
-
Click Simulate on the pop-menu
-
Go to the Simulation panel and select the node: HttpRequest
-
Click Run on the pop-menu
-
You should see the HttpResponse on the Simulation panel and captured samples by the Analyzer.
-
Repeat the step 9 to capture more samples by the Analyzer.
When the above test is working, you can close the Analyzer program and launch it on the Local Network using the .NET Remoting feature. This case will require you to enable the persistent subscription SoapProbeSubscriber and set it up properly such that the URL address of the remote Analyzer located in the Construct string is correct.
Major .NET technologies and techniques used in this solution
-
XML Web Services
-
SOAP and SOAP extensions
-
COM+ Event System
-
Remoting
-
Windows Form
-
TreeView Controls
-
XmlDocument
-
HttpRequest
-
Multi-threading techniques
-
Updating GUI in the background process
-
Loosely coupled design patterns
Using LCE WebService Probes in the Distributed Application Framework to strobe the business workflow gives you an additional post-processing Knowledge Base which can be used for many things. This is an incremental development pyramid based on developing the suitable subscribers. In this article, I presented one of them - WebService Analyzer Studio - which is a small tool to examine the contents of the SOAP Messages.
References:
[1] http://www.codeproject.com/useritems/RemotingProbe.asp
[2] //MS.NETFrameworkSDK/cpguidenf/html/cpconalteringsoapmessageusingsoapextensions.htm
[3] http://www.codeproject.com/csharp/msmqchannel.asp
[4] http://www.codeproject.com/cs/webservices/remotingoverinternet.asp
