Implementing an Object Request Broker in C++

Introduction

This is a discussion of how I implemented an ORB as part of a large (million'ish lines) code base that I have worked on for some time. I'll give some explanation of how an ORB works below, but basically it is a very nice way of doing client/server applications, providing the glue between the two sides, and extending the type safety of the language to the client's calls. In a large application with a lot of client/server interfaces, it can make a massive difference in the effort required.

Background

The term Object Request Broker goes back a long way, and really sort of fails to capture the glory. Some of you may be familiar with simpler technologies like some remote procedure call systems, some of which may just be a wrapper around a straight socket (or other) connection. An ORB is a much more complex beast. It effectively creates a 'party line' between any client and server, and all of the threads in that client will share that connection.

The calling client threads don't directly talk to the server. Instead, they make a call, which is packaged up for them, put on a queue, and the thread is put to sleep. A thread that manages the connection to that server will spool over commands to the server. The server side has a handler thread on that side which pulls out incoming commands and queues them, and in this case a farm of worker threads grab commands, find the target handler on the server, make the call, get the results back, package that up, and then the process happens in reverse.

When the results get back to the client side, it is put into the calling thread's results and the thread is awakened. As far has he knows, it all just happened synchronously. All of this happens quite automagically.

At the heart of it is an 'IDL' or Interface Description Language. It describes an interface, the methods thereof, their parameters, return values, etc... This is done in a language neutral format, in XML in my case. An IDL compiler reads this file and spits out two classes. One is a 'client proxy' and the other is a 'server stub'. We'll see how those are implemented below.

The client proxy is used as is, i.e., it's a concrete class. It implements the methods defined in the IDL, so it has a fully type-safe, language specific interface. You make calls to it as though you were making local calls, and the above described magic occurs.

On the server side, the stub class is an abstract base class, in which the methods of the interface are defined as virtuals. You implement your own derivative of it, override those virtuals to provide the required functionality, and return the results as appropriate. Here again, you really don't know anything about the underlying mechanisms. This call could have come to you from some local caller. The server creates instances of his derived handler class(es) and installs those on the ORB. That makes them available to be called from clients.

Of course the calling mechanism is more than just opening a socket since, as discussed above, it's not just a direct connection. In most cases, an intermediate server is used as a 'help available' type of board. Servers advertise their available interfaces there, and clients go there to find out how to connect to those interfaces. This is very flexible since those interfaces can move around over time and the clients don't really care. This type of server is usually called something like a 'name server', and it provides a service analogous to DNS on the internet, though much more full featured because it's not just a target address but a full, typed interface, that is being looked up.

The name server, BTW, is talked to through an ORB interface. It's common for ORBs to provide a set of standard servers that themselves implement ORB interfaces. In my case, I have a name server, a log server, and configuration server.

Demo Code Overview

The structure of this demo is as follows:

• A shared library that contains 'patient record' class that both sides need to know about, and some types that that class uses
• A server that has a small 'database' of patient records, which really is just dummy data loaded into a list for demo purposes. The server implements and exposes an ORB interface that allows the client to find a record by name or find all records for people of a particular marital status.
• A client that demonstrates connecting to the server and making those calls.

As mentioned, it might be worth looking at the previous article as well. One of the things that the IDL compiler does, besides interfaces, is to create very strongly typed enumerations. You will see in this code that I'm streaming enumerations, formatting them, translating them to/from text, etc..., things obviously not supported by C++ itself.

Also note that the code is stripped down heavily for brevity. Comments and deleted methods and such have been removed.

So let's start with the IDL for this interface, which should be fairly obvious from the above description.

<CIDIDL:Interface>

    <CIDIDL:ClassIntf   CIDIDL:Name="VideoDemo"
                        CIDIDL:InterfaceId="E1CCC9CBCFA1FDAF-29A8F7CB04763265">

        <CIDIDL:Constants>
            <CIDIDL:Constant CIDIDL:Name="strBinding"
                             CIDIDL:Type="TString"
                             CIDIDL:Value="/VideoDemo/IntfImpl"/>
        </CIDIDL:Constants>

        <CIDIDL:Methods>

            <!-- Query a record by name -->
            <CIDIDL:Method CIDIDL:Name="bQueryByName">
                <CIDIDL:RetType>
                    <CIDIDL:TBoolean/>
                </CIDIDL:RetType>
                <CIDIDL:Param CIDIDL:Name="strToFind" CIDIDL:Dir="In">
                    <CIDIDL:TString/>
                </CIDIDL:Param>
                <CIDIDL:Param CIDIDL:Name="recFound" CIDIDL:Dir="Out">
                    <CIDIDL:Object CIDIDL:Type="TDemoRecord"/>
                </CIDIDL:Param>

            </CIDIDL:Method>

            <!-- Query all with a specific status -->
            <CIDIDL:Method CIDIDL:Name="bQueryAllStatus">
                <CIDIDL:RetType>
                    <CIDIDL:TBoolean/>
                </CIDIDL:RetType>
                <CIDIDL:Param CIDIDL:Name="eToFind" CIDIDL:Dir="In">
                    <CIDIDL:Enumerated CIDIDL:Type="tVideoDemoSh::EStatus"/>
                </CIDIDL:Param>
                <CIDIDL:Param CIDIDL:Name="colFound" CIDIDL:Dir="Out">
                    <CIDIDL:TVector CIDIDL:ElemType="TDemoRecord"/>
                </CIDIDL:Param>

            </CIDIDL:Method>

        </CIDIDL:Methods>

    </CIDIDL:ClassIntf>

</CIDIDL:Interface>

You can see that it defines two methods. Via XML, it indicates the names, the return types and the parameters. For the first, there's a string input (name to find) and a record output (filled in if found.) For the second, there is a martial status type to find, and a vector of records to hold all the matches.

The Server Side

When the IDL runs on the server side, and is told to spit out the server version of this, it creates something like this (with irrelevant details removed.)

class  TVideoDemoServerBase : public TOrbServerBase
{
    public :
        static const TString strInterfaceId;
        static const TString strBinding;
        ~TVideoDemoServerBase();
        virtual tCIDLib::TBoolean bQueryByName
        (
            const TString& strToFind
            , TDemoRecord& recFound
        ) = 0;

        virtual tCIDLib::TBoolean bQueryAllStatus
        (
            const tVideoDemoSh::EStatus eToFind
            , TVector<TDemoRecord>& colFound
        ) = 0;

    protected :
        TVideoDemoServerBase();
        TVideoDemoServerBase(const TOrbObjId& ooidThis);
        tCIDLib::TVoid Dispatch
        (
            const  TString&      strMethodName
            ,      TOrbCmd&      orbcToDispatch
        )   override;
};

So it has created an abstract base class, which derives from the ORB's server side base class. It provides pure virtual methods for the two methods of the interface. Note the Dispatch() call. When an incoming call comes in from a client, that is called with the name of the method and the bundled up call information (parameters basically.)

If we look at Dispatch, it is really quite simple. The reason for this simplicity is that I have my own entire, consistent system. I have a well defined binary streaming scheme for objects. That means that any class that implements the 'streamable' mixin interface can be binarily streamed, and that's what's used to pack and unpack the parameters and returned information.

tCIDLib::TVoid
TVideoDemoServerBase::Dispatch(const TString& strMethodName, TOrbCmd& orbcToDispatch)
{
    if (strMethodName == L"bQueryByName")
    {
        TString strToFind;
        orbcToDispatch.strmIn() >> strToFind;
        TDemoRecord recFound;
        tCIDLib::TBoolean retVal = bQueryByName(strToFind, recFound);
        orbcToDispatch.strmOut().Reset();
        orbcToDispatch.strmOut() << retVal;
        orbcToDispatch.strmOut() << recFound;
    }
     else if (strMethodName == L"bQueryAllStatus")
    {
        tVideoDemoSh::EStatus eToFind;
        orbcToDispatch.strmIn() >> eToFind;
        TVector<TDemoRecord> colFound;
        tCIDLib::TBoolean retVal = bQueryAllStatus(eToFind, colFound);
        orbcToDispatch.strmOut().Reset();
        orbcToDispatch.strmOut() << retVal;
        orbcToDispatch.strmOut() << colFound;
    }
     else
    {
         TParent::Dispatch(strMethodName, orbcToDispatch);
    }
}

The bundled up command info class has a input stream that the generated code uses to stream out incoming parameters. It then makes the call and gets the return value and output, which it in turn streams back into the command info class for return via an output stream. So it takes very little code to implement this 'glue' bit.

My server program now has to implement that base class, again leaving out extraneous details, and ignoring the header file which is trivial. It's just the usual overriding of virtual methods. It creates a vector of demo record objects to act as the faux database.

TVector<TDemoRecord> m_colPatients;

The implementation of my derived class looks like this:

tCIDLib::TBoolean
TVideoDemoIntfImpl::bQueryByName(const TString& strToFind
                                , TDemoRecord& recFound)
{
    tCIDLib::TBoolean bRet = kCIDLib::False;
    m_colPatients.ForEach
    (
        [&bRet, &recFound, &strToFind](const TDemoRecord& recCur)
        {
            if (bRet = strToFind.bCompareI(recCur.m_strFullName))
                recFound = recCur;
            return !bRet;
        }
    );
    return bRet;
}

tCIDLib::TBoolean
TVideoDemoIntfImpl::bQueryAllStatus(const   tVideoDemoSh::EStatus   eToFind
                                    ,       TVector<TDemoRecord>&   colFound)
{
    colFound.RemoveAll();
    m_colPatients.ForEach
    (
        [eToFind, &colFound](const TDemoRecord& recCur)
        {
            if (recCur.m_eStatus == eToFind)
                colFound.objAdd(recCur);
            return kCIDLib::True;
        }
    );
    return !colFound.bIsEmpty();
}

tCIDLib::TVoid TVideoDemoIntfImpl::Initialize()
{
    // Load up some dummy records
    m_colPatients.objAdd(TDemoRecord(L"Angie", L"Night", tVideoDemoSh::EStatus::Married, 34, 1));
    m_colPatients.objAdd(TDemoRecord(L"Tom", L"Jones", tVideoDemoSh::EStatus::Single, 28, 2));
    m_colPatients.objAdd(TDemoRecord(L"Nancy", L"Sinatra", tVideoDemoSh::EStatus::Single, 24, 3));
    m_colPatients.objAdd(TDemoRecord(L"Robin", L"Ford", tVideoDemoSh::EStatus::Widowed, 29, 4));
    m_colPatients.objAdd(TDemoRecord(L"Reggie", L"Singleton", tVideoDemoSh::EStatus::Divorced, 54, 5));
    m_colPatients.objAdd(TDemoRecord(L"Jack", L"Anthony", tVideoDemoSh::EStatus::Married, 56, 6));
    m_colPatients.objAdd(TDemoRecord(L"Scott", L"Lassiter", tVideoDemoSh::EStatus::Married, 54, 7));
    m_colPatients.objAdd(TDemoRecord(L"Jane", L"Doe", tVideoDemoSh::EStatus::Single, 16, 8));
}

So I'm just implementing the two virtual methods. Both methods just iterate the vector of records looking for matches, and returning those records if found. The ORB does init/term callbacks to objects before they are registered and before they are cleaned up. I'm using the init call to put in some dummy records to use for testing.

The server has to set himself up to make this snazzy new interface available, which is pretty simple. Here is the code (from the main Cpp file of the server):

facCIDOrb().InitClient();
facCIDOrb().InitServer(9999, 1);

TVideoDemoIntfImpl* porbsImpl = new TVideoDemoIntfImpl();
facCIDOrb().RegisterObject(porbsImpl);

facCIDOrbUC().RegRebindObj
(
    porbsImpl->ooidThis()
    , TVideoDemoServerBase::strBinding
    , L"Video Demo Server Interface"
);
facCIDOrbUC().StartRebinder();

It initializes the client and server sides of the ORB (it needs the client because it also needs to talk to the name server, to which it is a client.) It tells the server side to listen on port 9999. It then creates an instance of our implementation of the server side stub class, which it registers with the ORB. It then also registers it with the ORB's rebinder, and starts the rebinder.

The reason for the rebinder is that the name server may go down for some reason. If that happens, we need to ensure that any interfaces we have registered get re-registered as soon as is reasonable after the name server is available again. At this point, the interface is available to clients. If I use a little administrative command line tool to dump the contents of the name server, I get this:

Scope=Root  '$Root$'
{
   Scope=CIDLib  'CIDLib Scope'
   {
      Scope=CIDLogSrv  'CIDLib Log Server Scope'
      {
         Binding=CoreAdmin  'Log Server Core Admin Object'
         Binding=LogObj  'CIDLib Log Server Logger Object'
      }
      Scope=CIDNameSrv  'CIDLib Name Server Scope'
      {
         Binding=CoreAdmin  'CIDLib Name Server Core Admin Object'
      }
   }
   Scope=VideoDemo  ''
   {
      Binding=IntfImpl  'Video Demo Server Interface'
   }
}

The name server provides a hierarchical structure in which 'bindings' can be placed. A binding is a server side ORB interface that has been bound to a name in the name server. In our case, it is in /Root/VideoDemo/IntfImpl. The root bit is implied. If you go back up to the original IDL file, it defines a constant for the binding /VideoDemo/IntfImpl which can be used by the clients who want to connect to this bound interface.

The Client Side

As mentioned, the IDL spits out a client side object that is a concrete class and which acts as a proxy for the remote interface on the server. I won't bother with the code for that. As you might expect, it defines the methods of the interface and it does the mirror image packaging/unpackage of the inputs and outputs as what we saw in the server example above. They look almost the same, just backwards.

One different thing it does have to provide is the mechanism to link it to the server. It has a constructor like this:

TVideoDemoClientProxy(const TOrbObjId& ooidSrc, const TString& strNSBinding)

So we provide a binding name, as mentioned above, and an 'object id'. The latter contains the contact information to connect to the server side interface, and is gotten from the name server. So, on the client side, we have this simple setup to get prepared to communicate:

facCIDOrb().InitClient();
TOrbObjId ooidSrv;
{
    tCIDOrbUC::TNSrvProxy orbcNS = facCIDOrbUC().orbcNameSrvProxy();
    tCIDLib::TCard8 c8Cookie;
    orbcNS->bGetObject(TVideoDemoClientProxy::strBinding, ooidSrv, c8Cookie);
}
TVideoDemoClientProxy orbcTest(ooidSrv, TVideoDemoClientProxy::strBinding);

We have to initialize the client side ORB. We then use that to get a client proxy for the name server. We then ask it for the object id for our binding path. If that works (and I've removed error checking here), then we have ooidSrv set up correctly.

We can then just construct an instance of the proxy, passing it the object id. If this doesn't throw, then it worked and we are connected. We can then make a call to it like this:

TVector<TDemoRecord> colFound;
if (orbcTest.bQueryAllStatus(tVideoDemoSh::EStatus::Married, colFound))
{
    TStreamIndentJan janIndent(&conOut, 8);
    conOut  << L"\nFound " << colFound.c4ElemCount() << L" records\n"
            << L"------------------------\n";
    colFound.ForEach
    (
        [&](const TDemoRecord& recCur)
        {
            conOut << recCur << kCIDLib::NewLn; return kCIDLib::True;
        }
    );
}

So we create a vector of records and call bQueryAllStatus(), passing married as the status. If it returns true, then we have records in the list and we can output them, which in this case outputs:

Found 3 records
------------------------
Name: Angie Night, Status: Married, Age: 34, Record: 1
Name: Jack Anthony, Status: Married, Age: 56, Record: 6
Name: Scott Lassiter, Status: Married, Age: 54, Record: 7

Summary

That was obviously a lot of work just to get that little bit of information. But, of course, the real benefit comes when this is a complex client/server interface with lots of methods and parameters. The fact that all of the packaging and unpackaging and transport is handled for you, with compile time checking of parameters, that makes a huge difference when the going gets complex. And when there are tens of such complex client/server interfaces, all the more so.

In my case, the fact that nothing special is required to pass objects (and enumerations) as parameters and return values makes it that much more powerful. An ORB that must work with code that is not part of such a tightly integrated might possibly place more burden on you to provide special handling for classes you want to use in these ORB interfaces.

That may ultimately mean you end up creating special classes, pulling data out of the real classes, putting them into the ORB'ified class, and sending that, and the values have to be pulled back out on the other side, etc. That would be a substantial lowering of the convenience bar relative to a tightly integrated system like this. It also may mean tying those classes to that ORB product, whereas here we just use the same binary streaming interface that is used to stream to files, memory buffers, etc. So the classes passed know nothing of the ORB at all.

In a subsequent article, I may get more into the guts of the ORB itself. But that would not have been very comprehensible without first seeing how it works from the outside.