Hooking a DirectX/COM Interface

• Download source files - 26.82 KB
• Download demo project - 37.07 KB

Introduction

After all the helpful articles I read here, I am glad that I can contribute a subject which has not yet been covered.

This article features a description on how to hook a DirectX/COM interface. I used the DirectInput interface as an example of how to hook an interface function.

For the basic Windows hook, I refer to an article by Wade Brainerd, which describes the API hooking process.

Task

To intercept a method of a COM interface requires an extended approach compared to hooking an API call. If the desired DLL is examined, only the create interface function is actually exported by the DLL. So, how can you hook your desired function?

A COM interface is basically a list of virtual function pointers, which are linked together. You merely have to follow the links and modify every node till you finally reach the pointer of the function, which you would like to replace.

Step 1

As you can see, only the create interface COM functions are visible, so you have to start your hooking chain at the DirectInputCreate function which returns a COM interface.

Here, you have to inject your DLL into the import address table (IAT) of the calling program.

Step 2

If the calling program invokes a DirectInputCreate, your function is called, you receive a pointer to a pointer of a pointer to a virtual function table, which is the interface of direct input:

DECLARE_INTERFACE_(IDirectInputW, IUnknown)
{
    /*** IUnknown methods ***/
    STDMETHOD(QueryInterface)(THIS_ REFIID riid, LPVOID * ppvObj) PURE;
    STDMETHOD_(ULONG,AddRef)(THIS) PURE;
    STDMETHOD_(ULONG,Release)(THIS) PURE;

    /*** IDirectInputW methods ***/
    STDMETHOD(CreateDevice)(THIS_ REFGUID,LPDIRECTINPUTDEVICEW *,LPUNKNOWN) PURE;
    STDMETHOD(EnumDevices)(THIS_ DWORD,LPDIENUMDEVICESCALLBACKW,LPVOID,DWORD) PURE;
    STDMETHOD(GetDeviceStatus)(THIS_ REFGUID) PURE;
    STDMETHOD(RunControlPanel)(THIS_ HWND,DWORD) PURE;
    STDMETHOD(Initialize)(THIS_ HINSTANCE,DWORD) PURE;
};

Step 3

Now you can create your device with CreateDevice. You will again receive an address to a different virtual function pointer table, which represents the Device.

Pick the method you would like to replace and change the virtual function pointer table in the desired place to inject your function.

Step 4

Do the actual data manipulation.

Implementation

Step 1

To hook yourself into an API function, you can simply use the Windows API call SetWindowsHookEx. Here you create a system hook to monitor starting processes and match it to your desired program. After you identified your program, you have to compare the import module names with the DLL you wish to replace. Since this hook is written for direct input, the entry we are looking for is DINPUT8.DLL. To find this entry you have to loop through the descriptors till you find your DLL.

// Iterate through each import descriptor, and redirect if appropriate
  while ( pImportDesc->FirstThunk )
  {
    PSTR pszImportModuleName = MakePtr( PSTR, hModEXE, pImportDesc->Name);

    if ( lstrcmpi( pszImportModuleName, Hook->Name ) == 0 )
    {
      sprintf(dbBuffer,"Dll Found in module  %s replace it\n", Hook->Name );
      WriteToLog(dbBuffer);
      RedirectIAT( Hook, pImportDesc, (PVOID)hModEXE );
    }

    pImportDesc++;  // Advance to next import descriptor
  }

After you found your entry, you have to remove the write protection from the IAT with...

VirtualQuery( pIAT, &mbi, sizeof(mbi) );

... to be able to write into the memory. After the memory is open, you have to find your entry by iterating through the IAT.

while ( pIteratingIAT->u1.Function )
{
  void* HookFn = 0;  // Set to either the SFunctionHook or pStubs.

  if ( !IMAGE_SNAP_BY_ORDINAL( pINT->u1.Ordinal ) )  // import by name
  {
    PIMAGE_IMPORT_BY_NAME pImportName = MakePtr( PIMAGE_IMPORT_BY_NAME,
        pBaseLoadAddr, pINT->u1.AddressOfData );

    // Iterate through the hook functions, searching for this import.
    SFunctionHook* FHook = DLLHook->Functions;
    while ( FHook->Name )
    {
      if ( lstrcmpi( FHook->Name, (char*)pImportName->Name ) == 0 )
      {
        sprintf(dbBuffer,"Hooked function: %s\n",(char*)pImportName->Name );
        WriteToLog(dbBuffer);
        // Save the old function in the SFunctionHook structure and get the new one.
        FHook->OrigFn = (unsigned long*)pIteratingIAT->u1.Function;
        HookFn = FHook->HookFn;
        break;
       }

        FHook++;
      }
    }

Now, you can replace it with your own one.

// Replace the IAT function pointer if we have a hook.
if ( HookFn )
{
  // Cheez-o hack to see if what we're importing is code or data.
  // If it's code, we shouldn't be able to write to it
  if ( IsBadWritePtr( (PVOID)pIteratingIAT->u1.Function, 1 ) )
  {
    pIteratingIAT->u1.Function = (DWORD)HookFn;
  }
  else if ( osvi.dwPlatformId == VER_PLATFORM_WIN32_WINDOWS )
  {
    // Special hack for Win9X, which builds stubs for imported
    // functions in system DLLs (Loaded above 2GB).  These stubs are
    // writeable, so we have to explicitly check for this case
    if ( pIteratingIAT->u1.Function > (DWORD)0x80000000 )
        pIteratingIAT->u1.Function = (DWORD)HookFn;
  }
}

The only thing remaining is to restore the memory attributes, as nothing ever happened.

VirtualProtect( pIAT, sizeof(PVOID) * cFuncs, flOldProtect, &flDontCare);

Step 2

Inside of the CreateInterface method, we start hooking into the COM interface by injecting our own CreateDevice function pointer into the Virtual function table (Vtbl), which is returned in the ppvOut pointer of the original call.

DirectInput8Create_Type OldFn =
   (DirectInput8Create_Type)D3DHook.Functions[D3DFN_DirectInput8Create].OrigFn;
  HRESULT hr = OldFn( hinst, dwVersion, riidltf, ppvOut, punkOuter );

Resolve the pointer until you get the pointer to the Vtbl of the interface.

With this address, you have to again remove the memory protection before you can inject your function into the table and save the old function pointer for later use.

Inject your function pointer into the offset of the CreateDevice function pointer inside of the interface Vtbl and restore the memory protection.

As you can see, the CreateDevice is the fourth method of the DirectInput interface, which means the offset inside of the Vtbl is 0x0C (pointer(DWORD)*index 3).

typedef struct IDirectInput *LPDIRECTINPUT;

#if !defined(__cplusplus) || defined(CINTERFACE)
#define IDirectInput_QueryInterface(p,a,b) (p)->lpVtbl->QueryInterface(p,a,b)
#define IDirectInput_AddRef(p) (p)->lpVtbl->AddRef(p)
#define IDirectInput_Release(p) (p)->lpVtbl->Release(p)
#define IDirectInput_CreateDevice(p,a,b,c) (p)->lpVtbl->CreateDevice(p,a,b,c)
#define IDirectInput_EnumDevices(p,a,b,c,d) (p)->lpVtbl->EnumDevices(p,a,b,c,d)
#define IDirectInput_GetDeviceStatus(p,a) (p)->lpVtbl->GetDeviceStatus(p,a)
#define IDirectInput_RunControlPanel(p,a,b) (p)->lpVtbl->RunControlPanel(p,a,b)
#define IDirectInput_Initialize(p,a,b) (p)->lpVtbl->Initialize(p,a,b)
#else
#define IDirectInput_QueryInterface(p,a,b) (p)->QueryInterface(a,b)
#define IDirectInput_AddRef(p) (p)->AddRef()
#define IDirectInput_Release(p) (p)->Release()
#define IDirectInput_CreateDevice(p,a,b,c) (p)->CreateDevice(a,b,c)
#define IDirectInput_EnumDevices(p,a,b,c,d) (p)->EnumDevices(a,b,c,d)
#define IDirectInput_GetDeviceStatus(p,a) (p)->GetDeviceStatus(a)
#define IDirectInput_RunControlPanel(p,a,b) (p)->RunControlPanel(a,b)
#define IDirectInput_Initialize(p,a,b) (p)->Initialize(a,b)
#endif

After we know where to inject it, we can start thinking about the implementation. When you look at the declaration of CreateDevice in the dinput.h, it does not match up with the declaration you see in the DirectX Help.

HRESULT CreateDevice(

    REFGUID rguid,
    LPDIRECTINPUTDEVICE *lplpDirectInputDevice,
    LPUNKNOWN pUnkOuter
);

As you can see in the definition inside the dinput.h, you have to add a fourth parameter, which is the interface pointer. This ends up in the following function declaration:

HRESULT __stdcall   PASCAL MyCreateDevice(LPVOID *ppvOut,REFGUID rguid,
    LPDIRECTINPUTDEVICE *lplpDirectInputDevice,
    LPUNKNOWN pUnkOuter

This is important. You have to make sure, to use the __stdcall calling convention in your declaration. Refer to the MSDN.

The __stdcall calling convention is used to call Win32 API functions. The callee cleans the stack. __cdecl is the default calling convention for C and C++ programs. The stack has to be cleaned up by the caller, which is not the case with our function.

When you look at the disassembly of this call, you can see the stack pointer verification function _RTC_CheckEsp is called after the call to the interface function.

if (lpdi->CreateDevice(GUID_SysKeyboard, &lpdikey, NULL)!=DI_OK)
00401365  mov         esi,esp
00401367  push        0
00401369  push        offset lpdikey (4552C8h)
0040136E  push        offset _GUID_SysKeyboard (44643Ch)
00401373  mov         eax,dword ptr [lpdi (4552C4h)]
00401378  mov         ecx,dword ptr [eax]
0040137A  mov         edx,dword ptr [lpdi (4552C4h)]
00401380  push        edx
00401381  mov         eax,dword ptr [ecx+0Ch]
00401384  call        eax
00401386  cmp         esi,esp
00401388  call        _RTC_CheckEsp (4026A0h)
0040138D  test        eax,eax
0040138F  je          Game_Init+78h (401398h)
   return(0);
00401391  xor         eax,eax
00401393  jmp         Game_Init+107h (401427h)

// set cooperation level
if (lpdikey->SetCooperativeLevel(main_window_handle,

If you forget to declare your function with __stdcall your function will process fine, but you will fail the esp pointer test of this function which sets the eax and test it after the function call.

Step 3

When you now create the device, the call is re-routed to your CreateDevice function.

After you call the original function, you receive a new pointer in lplpDirectInputDevice, which will direct you to the Vtbl of the device.

HRESULT hr = OldCreateDev(ppvOut,rguid,lplpDirectInputDevice,pUnkOuter);

In my sample, I replaced the GetDeviceState, since I can add additional input in the return data of the calling function. To get to the offset, you have to look at the definition inside the DInput.dll. You see that the GetDeviceState function is the tenth method inside of the device, which leads to an offset of 0x24.

After you know the offset, we can proceed with the instructions in step 2 to remove the protection, store the old pointer, inject your own function, and restore the memory protection.

Step 4

When the GetDeviceState function is called by the target program, the injected function is called and you can manipulate the data as you wish.

History

• 8th May, 2006: Article posted