Dealing with DCOM Hardening - Part 1

Introduction

This article is written from a background both as a software developer and system administrator. When dealing with this class of problem, it is important to have a decent understanding of the situation. A lot has been written during the last couple of years but I figured that if you find yourself suddenly dealing with this in devops, a quick rundown would be beneficial.

In this first short article, I'll explain the issue and the non-invasive ways you can use to approach the problem. In follow-up articles, I will cover the invasive ways when none of the approaches in this article work, and vendor support is not an option for various reasons. Those approaches will require coding. I've thought about putting it all in 1 big article, but the problem is there is more than just 1 independent article I could write. I don't want to lump everything together. And at the same time, I don't want to repeat this information every time.

So part 1 is the rundown with off-the-shelf fixes.

Background

COM is an object model technology. In essence, a client application can use objects in an object oriented fashion, and those objects can either execute code in the client application itself, or in a server application running on the same computer. If the code is running outside the client (or the code is running in a different threading setup), the object is proxied. This means that a method call is serialized and sent to the server. There, it is unpackaged, processed, and the result is sent back the same way. The transport uses only Windows message queues (EDIT: I mean the win32 messaging infrastructure, not MSMQ which is a different technology), which are always under control of Windows itself in a secure layer.

DCOM is a remoting technology built on RPC. It extends COM to work beween

For the sake of completeness, I should point out that The D in DCOM means distributed. COM stands for an 'component object model', referring to dealing with objects on the same machine. It becomes DCOM when the components are on different machines. Over time, the distinction disappeared and you'll find 'DCOM' used as a general term. This makes sense because any component can be used locally or remotely, and the client doesn't always know or care about that. From a technical point of view, it would be more correct to talk about COM and DCOM separately.

The whole issue of DCOM hardening comes into play only if the client and server are running on different machines. If they are on the same machine.

However, if they run on different machines, the network comes into play. These days, any protocol uses security by default. The idea that a client would send unsecured, readable, anonymous method calls over an internal network or even the internet, is completely bananas. But you have to remember that DCOM was 'hip' in the early 90s. As Raymond Chen mentioned in one of his blog posts: whenever you look at DCOM and think about fixes to the obvious problems, you need to keep in mind that it had to work on a 80386 with 4 megabytes of memory. And the internet was not yet the malware infected hell-scape it is today.

It was a simpler life, and while authentication was implemented, most of the available options don't make sense. There are 6 available authentication levels which can be used in the communication:

RPC_C_AUTHN_LEVEL_DEFAULT means the system default settings configured using DCOMCnfg will be used. Applications using this are also the easiest to fix without touching the application itself.

RPC_C_AUTHN_LEVEL_NONE was only used when having to work with DCOM in anonymous environments, such as over the internet, or in a workgroup environment. Looking back with today's eyes, this makes as much sense as asbestos ceiling tiles.

RPC_C_AUTHN_LEVEL_CONNECT used to be the default, both in DCOMCnfg and applications where the programmer even bothered to implement a call to CoInitializeSecurity. Many servers came with the recommendation to explicitly use 'Connect' level security. This makes sense because from a user / admin perspective, the software implements access control and the software will work (or not) depending on the user account. Sure, the actual transport was still unsecure, without any of the overhead of packet privacy or malware countermeasures. But at the application level, access management was fully functional.

RPC_C_AUTHN_LEVEL_CALL and RPC_C_AUTHN_LEVEL_PKT don't make any sense in my opinion. They don't really change the overall security implications. There's still no added privacy compared to RPC_C_AUTHN_LEVEL_CONNECT. There's still no way to avoid tampering with the packet data. And at the same time, there's also no added security compared to RPC_C_AUTHN_LEVEL_CONNECT because any attacker who can compromise authentication once, can do so again. In all my years as a software developer I've never encountered these 2 authentication levels.

RPC_C_AUTHN_LEVEL_PKT_INTEGRITY is the first secure level that makes sense, because every packet is tamperproof. There is still no privacy, but at the very least a client and server can trust their interaction. RPC_C_AUTHN_LEVEL_PKT_PRIVACY is equally secure, with the added bonus of ensuring privacy between the client and the server.

The DCOM hardening means that only RPC_C_AUTHN_LEVEL_PKT_INTEGRITY and RPC_C_AUTHN_LEVEL_PKT_PRIVACY are accepted. Requests for lower settings will fail.

On a final note, I should point out that the concept of 'server' and 'client' is rather fluid when dealing with DCOM. A client application may want to activate a DCOM object in a server application, but in many cases, the server application will as a result try to activate a DCOM object in the client application for handling callbacks. From a DCOM perspective, there can be a client - server relationship in both directions.

Server Hardening

On the server side, hardening means rejecting connections that are not at the minimum level. This is the timeline that was in play:

During the phase when this could be enabled or disabled, af registry change could be used to enable or disable hardening via the key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Ole\AppCompat with a DWORD value named "RequireIntegrityActivationAuthenticationLevel". 0 means disabled. 1 means enabled. If this value is not defined, it will default to enabled.

When servers reject connection, this is visible in the system eventlog with EventID 10036, 10037 or 10038.

This part of the change is relatively simple because from a the server level perspective, everything just becomes more strict. The servers themselves don't really have to care because they don't have to deal with this. They either get activated or they don't. And when they do, it's just business as usual since the remoting is handled by Windows. The real challenge is for the client.

Client Level Mitigation

There are several possible approaches possible.

The Vendor

In a business environment, the first response should be to contact the software vendor and get them to provide support. But this may not always be possible. Plenty of vendors have gone out of business while their applications go on. Especially in the business world, applications can have lifecycles measured in decades. In that case, you'll still need to cope.

Changing the Defaults

If an application uses DCOM without specifying a security level, either directly through CoInitializeSecurity or indirectly through CoCreateInstanceEx, the system will attempt to use the authentication level which was specified for that particular DCOM object. If there was no authentication level specified for the specific object, the system default which was configured in DCOMCnfg will be used.

However, this assumes that the client application itself doesn't initialize security anywhere. If it does, then this will override any of those configured values. In this case, your only 2 choices are to contact the vendor to request an update to the application, or you rely on auto-elevation which is explained in the next section.

Client Side Auto-Elevation

This is by far the most convenient approach, and brilliant in its simplicity. Just like Microsoft implemented a change to reject server level activation requests that are too low, they implemented a client level change in Windows which simply elevates the requested authentication level to the minimum supported level for all non-anonymous clients if it is too low.

The 'non anonymity' is important here because DCOM used to allow anonymous remoting. If a client request is raised to a secure authentication level, this cannot be done transparently because no form of authentication can take place. Anonymous DCOM is now a thing of the past and will not work.

If you use anonymous DCOM and want to rely on auto elevation, your only option is to remove the anonymity, by joining the client and server to the same domain and / or configuring them to use known user accounts.

Other than that, this fix is foolproof if you are dealing with clients running on Windows 8.1 or Server 2012 R2 or higher. Just install a recent cumulative update (2023-02 or higher) and you're set.

Client Side Auto-Elevation for Legacy Systems

Windows 7 and Windows 2008R2 have a security rollup available. However, there is a snag. Not only are those systems end of life already, they are also past extended support. This means that Microsoft doesn't fix anything unless there is a really, really, REALLY good case to support an OS that is 15 years old. And it expects you to have purchased extended support licenses to make it worth their while to do so. Without support license installed, these updates will not install.

If you are in this situation and want to buy such a license, you're out of luck because the various articles I've found suggest that Microsoft stopped selling them in January 2023. And indeed, I've discovered that no matter what I approve in WSUS, the clients report in daily but say they don't any updates that ran past the end of life cycle.

I should add that I did not expect the following approach to work. I would expect that a hotfix only installs on systems that are eligible. If a hotfix does install, I am willing to proceed on the basis that I am not doing anything wrong.

The actual updates themselves can still be downloaded manually from the catalog, if you know which ones you need. In my case, I needed 4. And they need to be installed in that particular order.

1. windows6.1-kb4474419-v3-x64_b5614c6cea5cb4e198717789633dca16308ef79c
2. windows6.1-kb5006749-x64_4934673a816c4c0e5d6c380e0a61428da8aab4ac
3. windows6.1-kb5028264-x64_1142dafe4d79f24f014a975a6012d2bdfea9197d
4. windows6.1-kb5023769-x64_3e40559476ec20b845d0031d316321e99539e140

The first update adds SHA-2 signing support to Windows, which is required because recent updates are no longer signed using older signing technologies. If this was not yet already installed, a reboot is necessary before continuing with the next ones. Updates 2 and 3 are required to update the servicing stack, which is the software responsible for managing Windows updates. For some reason, I had to install both. Update 4 is the rollup that contains the auto-elevation hotfix.

This is where the support licensing comes back to haunt us. Manually installing the msu file works. However, as soon as the server restarts, the cumulative hotfix is rolled back and only 1, older, hotfix of that cumulative rollup is visible in Control Panel\Programs and feature\installed updates. The rest is all rolled back. I suspect this is because there is no extended support license. However, I still don't know why it would then install in the first place. But I figure that if I can install it without taking special measures, that is expected.

You won't find kb5023769 in the list of installed updates. The bizarre thing is: for a brief few minutes after the reboot, my application was working. I discovered this accidentally by just happening to look at the diagnostics window while waiting for the reboot to finalize. This led me to believe that the update did install properly, and only after finalizing everything did the windows installer figure 'Uh, hold one, I don't think I was supposed to have done this, let's roll back before anyone finds out...'. This in turn led me to think: suppose we could prevent that from happening...

After some experimenting, I've found the following. The way to work around that is:

1. Install the 4^th update in the list.
2. When a dialog appears, asking to reboot, just close the dialog without rebooting.
3. Open Control panel, Control Panel\Programs and feature\installed updates and verify that kb5023769 is installed. This may take a couple of seconds.
4. Open Administrative tools -> Services
5. Disable the ‘Windows Modules Installer service’
6. Disable the ‘Windows Update Service’
7. Reboot the computer.

The installed update will not be removed anymore because TrustedInstaller.exe can no longer start and manage hotfixes.

Note the following things:

• Disabling TrustedInstaller needs to be permanent. If TrustedInstaller.exe can start, it will remove the hotfix again. If there is a group policy to enable the windows Modules Installer Service, there needs to be an overriding GPO to make sure it stays disabled.
• While TrustedInstaller.exe is disabled, it is not possible to use the control panel to view installed updates.
• This tactic can be used with exactly 1 hotfix. kb5023769 (or another hotfix) requires a reboot. No more hotfixes can be installed until the reboot. But TrustedInstaller needs to be disabled at next boot to prevent it from removing kb5023769. This means that no more updates can ever be installed. Since we are dealing with a 15 year old OS that is already past extended support and actively being phased out in production, this should not be a problem but it’s a point that needs mentioning.
• If you need this level of trickery to get things going, you should really take it as a sign from the heavens that you need to get a move on, migrating your business application to a modern platform.

Points of Interest

It's been a while since I wrote an article here. I don't write for the sake of content volume. I write when I come across something that tickles my curiosity and makes me want to dive in to figure out the details. DCOM spelunking is an interesting cross-over between technology and archaeology. This article only scratches the surface, but outlines the problem and the solution.

To be honest, I went back and forth on whether to include the workaround because I am not sure this is intended behavior. On the other hand, it's a good assumption that anyone in that situation is already working on a migration and only dealing with this problem because a) they are not in a position to stop the rollout of server hardening and b) not in charge of support decisions. And I figured if you're the one stuck between hammer and anvil, with a downed production system and the board breathing down your neck, you deserve all the help you can get.

I am interested though. If the above information made a difference for anyone, I'd appreciate hearing from you and how it applied to your situation.

In the next article(s), I will go into possible approaches where vendor support is not an option, the hotfixes are not a solution for whatever reason (the main one would be dealing with legacy systems) and you need to resort to more brutal measures.

History

• 19^th October, 2023: First version