Click here to Skip to main content
15,887,214 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hi,
I have a problem with a WCF Call back crashing after the callback works fine about 20 times without any issues. The call back crashes with the error
<pre>The message could not be transferred within the allotted timeout of 00:00:20. There was no space available in the reliable channel's transfer window. The time allotted to this operation may have been a portion of a longer timeout.


I am going to start describing my code below. Please review.

I have a Contract as shown below.
  [ServiceContract(SessionMode = SessionMode.Required, CallbackContract = typeof(ICallBack1))]
    public interface IService1
    {
         [OperationContract(IsOneWay = true)]
         void StartSlot( int slotNumber);
      

    }

    [ServiceKnownType(typeof(Gen5.Model.STBTestModel))]
    public  interface ICallBack1
    {
         [OperationContract(IsOneWay = true )]
         void TestUpdateEvent(Gen5.Model.STBTestModel stb1);      
         [OperationContract(IsOneWay = true)]
         void UpdateLogBasic(int slot,string text,string testType);
         [OperationContract(IsOneWay = true)]
         void UpdateLogDetailed(int slot,string text,string testType);
         [OperationContract(IsOneWay = true)]
         void UpdateLogBasicandDetailed(int slot, string text, string testType);

     }

I create a proxy client using the code below. Please note the parameters for the proxy client. I also have the server defined further way below in code.

<pre> public class Proxy1
    {
          private EndpointAddress _endPointAddress;
          private NetTcpBinding _tcpBinding ;
          InstanceContext _mContext;
          System.ServiceModel.DuplexChannelFactory<IService1> proxy;
          IService1 proxyfunction;

          public Proxy1(InstanceContext context, string serviceName, string destIPAddresss, int portNumber,TimeSpan sendTimeOut,TimeSpan recieveTimeOut, TimeSpan openTimeOut, TimeSpan closeTimeOut)
          {
              _mContext = context;              
              _tcpBinding = new NetTcpBinding(SecurityMode.None, true);
              _tcpBinding.Security.Message.ClientCredentialType = MessageCredentialType.None;
              _tcpBinding.ReliableSession.Ordered = true;
              _tcpBinding.ReliableSession.Enabled = true;
              _tcpBinding.ReliableSession.InactivityTimeout = new TimeSpan(24, 0, 0);
              _tcpBinding.Security.Mode = SecurityMode.None;
              _tcpBinding.OpenTimeout = openTimeOut;
              _tcpBinding.CloseTimeout = closeTimeOut;
              _tcpBinding.TransactionFlow = false;
              _tcpBinding.TransferMode = TransferMode.Buffered;
              _tcpBinding.SendTimeout = sendTimeOut;
              _tcpBinding.ReceiveTimeout = recieveTimeOut;           
              _tcpBinding.MaxConnections = 1000;
              _tcpBinding.ListenBacklog = 1000;
              _tcpBinding.MaxBufferSize = 1000000;//1MB
              _tcpBinding.MaxReceivedMessageSize = 1000000;//1MB
              _endPointAddress = new EndpointAddress("net.tcp://" +/*"10.248.1.3"*/destIPAddresss + ":" +portNumber.ToString() + "/" +"Service"/*serviceName.ToString()*/);
              proxy = new DuplexChannelFactory<IService1>(_mContext, _tcpBinding);
              proxyfunction = proxy.CreateChannel(_endPointAddress);
          }



The proxy object is initialized in another class 'Controller' which is the CallBackClass also as shown below. This 'Controller' class implements the 'Gen5.WCF.ICallBack1'. The timeout settings as shown below are (the constructor is shown above ). send-20seconds, receive-24 hours, opentimeout=4 seconds, closetimeout=4 seconds. The client is connecting to one service the server and is calling back to the same client back. So it is a one to one setup.

[CallbackBehavior(ConcurrencyMode = ConcurrencyMode.Multiple, UseSynchronizationContext = false)]
   public class Controller : Gen5.WCF.ICallBack1, Gen5.View.IController
   {

void CreateProxy(int slotNumber)
        {
            string ipTestPC = Shared.Functions.GetTestPCIP(slotNumber);
            proxy1 = new Gen5.WCF.Proxy1(new InstanceContext(this), "Service", ipTestPC, 4000, new TimeSpan(0, 0, 20),new TimeSpan(24, 0, 0), new TimeSpan(0, 0, 4), new TimeSpan(0, 0, 4));
        }



This is the connection code on the client side to connect to the server. The I call Start function. This code executes on the same CallBack interface class Controller.

proxy1[i].Connect();
proxy1[i].Start();


On the ServerSide, the Connect creates the call back context in TestPCController which implements the 'IService1' and the Start function calls a thread which starts a USER Interface Test. This is non blocking since its a thread. Please note all the timeouts and settings for the server in the code below but I set it up the same.

public partial class TestPCController : Gen5.WCF.IService1, ITestPCController
    {
        #region WCF
        ServiceHost host;
        InstanceContext context;
        NetTcpBinding binding1;
        ServiceThrottlingBehavior throttle;
        public delegate void Gen5TestPCLogUpdateEvent(object sender, int slot, string log, string testTitle, Shared.enumLogType logType);
        public static Gen5TestPCLogUpdateEvent TestPCLogUpdateEvent;
        static IGen5TestPCView viewTestPC;
        static STBTestModel stb;        
        static Devices devicesLocalTestPC;
        public TestPCController()
        {
        }
        public TestPCController(IGen5TestPCView viewGen5TestPC1)
        {
            viewTestPC = viewGen5TestPC1;
        }
        public void StartService(TimeSpan sendTimeOut, TimeSpan recieveTimeOut, TimeSpan openTimeOut, TimeSpan closeTimeOut)
        {

            string uri1 = "net.tcp://localhost" + ":4000/" + "Service";
            host = new ServiceHost(typeof(TestPCController), new Uri(uri1));
            context = new InstanceContext(this);
            binding1 = new NetTcpBinding(SecurityMode.None, true);
            binding1.Security.Message.ClientCredentialType = MessageCredentialType.None;
            binding1.ReliableSession.Ordered = true;
            binding1.ReliableSession.Enabled = true;
            binding1.ReliableSession.InactivityTimeout = new TimeSpan(24, 0, 0);
            binding1.SendTimeout = sendTimeOut;
            binding1.ReceiveTimeout = recieveTimeOut;
            binding1.OpenTimeout = openTimeOut;
            binding1.CloseTimeout = closeTimeOut;
            binding1.MaxBufferSize = 1000000;//1MB
            binding1.MaxReceivedMessageSize = 1000000;//1MB
            binding1.MaxConnections = 1000;
            binding1.ListenBacklog = 1000;
            host.AddServiceEndpoint(typeof(Gen5.WCF.IService1), binding1, "");


            throttle = host.Description.Behaviors.Find<ServiceThrottlingBehavior>();
            host.Description.Behaviors.Find<ServiceDebugBehavior>().IncludeExceptionDetailInFaults = true;
            if (throttle == null)
            {
                throttle = new ServiceThrottlingBehavior();
                throttle.MaxConcurrentCalls = 1000;
                throttle.MaxConcurrentSessions = 1000;
                throttle.MaxConcurrentInstances = 1000;
                host.Description.Behaviors.Add(throttle);
            }
            host.Open();
            Console.WriteLine("Started Gen5Service at " + uri1 + " successfully");


            STBTestModel.testUpdateEvent += STB_TestUpdateEvent;
            STBTestModel.logEvent += STB_LogEvent;
            STBTestModel.logActionEvent += STBTestModel_logActionEvent;

        }


public void Connect()
       {
           iServiceCallback = OperationContext.Current.GetCallbackChannel<Gen5.WCF.ICallBack1>();

       }

       public void StartSlot(int slotNumber)
       {

       }


About 4 different functions are called using callback. They call back functions get called at least 50 times in a row and then crashes. I know its not UI blocking on the client because the callback functions dont execute any code- its blank temporarily- it still crashes. Something in my code is wrong that causes the callback to crash at some point. Rentrant etc. I have no idea why the call back is crashing. I know that the callback is only suscribed to once- is there a limit on how many times server can call back using. The only weird thing I noticed is that the code hangs exactly at the same line everytime. Since there is no code in the callback code in the client, I know its WCF related. Something is causing the call back to timeout and it is a fixed event.

Please share your thoughts and don't hesitate to ask any questions please. I tried to be as clear as I could.

What I have tried:

I tried all different combinations like Concurrent.Muliple, Rentrant etc.

I tried doing a trace but the trace only shows the Service.Model throwing exception and crashing.

I removed all code from the client call back execution to make sure it was not a UI Block on the client side.

I added the following code for binding on both server and client

binding1.ReliableSession.Ordered = true;
          binding1.ReliableSession.Enabled = true;
          binding1.ReliableSession.InactivityTimeout = new TimeSpan(24, 0, 0);
          binding1.TransactionFlow = false;
          binding1.TransferMode = TransferMode.Buffered;


I put synchronization context = false
Posted
Updated 18-May-17 15:31pm
v5

1 solution

WOOOOO. Solved the issue finally after 2 days of troubleshooting.

I did the following to fix the issues. However I am not exactly sure which step fixed the issue. I would need to dig in more to find that but wanted to post this before I forget.

1) My client CallBack class constructor did not have a default constructor.
as noted here, this can cause issues.
WCF – Improve performance with greater concurrency | theburningmonk.com[^]

2) I increased the _tcpBinding.MaxConnections = 10000 and
_tcpBinding.ListenBacklog = 10000; just in case.
I increased the following also:
_tcpBinding.MaxBufferSize = 100000000;//100MB
             _tcpBinding.MaxReceivedMessageSize = 10000000;//100MB


3) I implemented the synchronization below

[^]

4) I increased the threadpool on both client and server
Why Are WCF Responses Slow and SetMinThreads Does Not Work? – Wenlong Dong's Blog[^]

5) I had already installed the servertrace on the sever after putting the below configurations in app.config of server nad client. This is just a troubleshooting tool and now I can confirm that my wcf calls and callbacks only take 4ms average(This is a local communication server client). I processed over 500 callbacks without any issue. I am going to put it in a loop for a day and see what happens.

<configuration>
  <system.diagnostics>
    <sources>
      <source name="System.ServiceModel"
              switchValue="Information, ActivityTracing"
              propagateActivity="true">
        <listeners>
          <add name="traceListener"
              type="System.Diagnostics.XmlWriterTraceListener"
              initializeData= "c:\wcflog\traces.svclog" />
        </listeners>
      </source>
    </sources>
  </system.diagnostics>
  <system.net>
    <defaultProxy>
      <proxy usesystemdefault="False"/>
    </defaultProxy>
  </system.net>
 
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>
</configuration>


If I have any updates I will post but whew.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900