Click here to Skip to main content
15,868,016 members
Articles / Desktop Programming / Win32
Article

Introduction Into Windows Anti-Debugging

Rate me:
Please Sign up or sign in to vote.
4.81/5 (39 votes)
18 Sep 2008CPOL3 min read 103K   116   16
A brief introduction into Windows anti-debugging techniques.

Introduction

In recent times, tools for use in reverse engineering have flourished. There are plenty of resource sites for those who are interested in the field, and the field is very much worth the time invested in it. I found that learning C++ while introducing myself to reverse engineering and assembly language really helped me to understand how code works, and improved my C/C++ coding and my ASM coding at the same time. However, reverse engineering also has a darker side. Crackers are individuals who use their knowledge of reverse engineering to reverse another programmer's code, often to decode how a serial is processed or to remove a protection from a trial. Naturally, a pioneer will want to protect their investment; this can be done with tools such as Themida, Execryptor, Armadillo, and even a protection system coded by a CodeProject resident Jim Charles named Eagle Protector. This article is meant to inform individuals of some anti-debugging techniques, and is not meant to be all-inclusive, nor does it explore some of the more complex routines that commercial protectors use.

Background

An individual reading this should have a solid understanding of ASM, how computers handle memory, the Win32 Debugging API, and at least some knowledge of Windows internals. This code most likely will not work on any *nix platform due to the fundamental differences of the Operating Systems. Any other knowledge in the field of reverse engineering is also a plus. One great thing about learning and implementing anti-debugging is that you also develop your reversing skills, which is a great plus to anyone interested in the field. Along with the other mentioned subjects, an interested reader should also be familiar with the tools used for binary application reversing such as OllyDBG, WinDBG, SoftICE, IDA Pro, and others. Here are some links to some information that is important for readers to be familiar with before reading the following text:

IsDebuggerPresent

This is probably the simplest method for debugger protection. This function is part of the Win32 Debugging API, and its reference page can be found here: MSDN.

C++
if(IsDebuggerPresent())
{
    MessageBox(NULL, TEXT("Please close your debugging application" + 
               " and restart the program"), 
               TEXT("Debugger Found!"), 0);
    ExitProcess(0);
}
// Normal Code Here....

IsDebuggerPresent Using the PEB

The IsDebuggerPresent function is actually a wrapper around this code. It directly access the PEB for the process and reads a byte value that signifies if the process is being debugged.

C++
char IsDbgPresent = 0;
__asm {
     mov eax, fs:[30h]
     mov al, [eax + 2h]
     mov IsDbgPresent, al
}

if(IsDbgPresent)
{
    MessageBox(NULL, TEXT("Please close your debugging " + 
               "application and restart the program"), 
               TEXT("Debugger Found!"), 0);
    ExitProcess(0);
}
// Normal Execution

CheckRemoteDebuggerPresent

This is another Win32 Debugging API function; it can be used to check if a remote process is being debugged. However, we can also use this as another method for checking if our own process is being debugged. For those of you who wonder how this function differs, CheckRemoteDebuggerPresent calls the NTDLL export NtQueryInformationProcess with the SYSTEM_INFORMATION_CLASS set to 7 (ProcessDebugPort). The MSDN reference for the function can be found here: MSDN.

BOOL IsDbgPresent = FALSE;
CheckRemoteDebuggerPresent(GetCurrentProcess(), &IsDbgPresent);
if(IsDbgPresent)
{
        MessageBox(NULL, TEXT("Please close your debugging" + 
                   " application and restart the program"), 
                   TEXT("Debugger Found!"), 0);
        ExitProcess(0);
}
// Normal Execution

NtQueryInformationProcess

Instead of calling CheckRemoteDebuggerPresent an individual could also make the call to NtQueryInformationProcess process theirself. MSDN does not encourage the use of any NtxXx function because of the possibility of its behavoir changing, therefore some consideration may want to be made about using the function before committing to using it. The MSDN reference for this function can be found here: MSDN.

// Function Pointer Typedef for NtQueryInformationProcess
typedef unsigned long (__stdcall *pfnNtQueryInformationProcess)(IN  HANDLE, 
        IN  unsigned int, OUT PVOID, IN ULONG, OUT PULONG);

 
// ProcessDebugPort
const int ProcessDbgPort = 7;
 
// We have to import the function
pfnNtQueryInformationProcess NtQueryInfoProcess = NULL;
 
// Other Vars
unsigned long Ret;
unsigned long IsRemotePresent = 0;
 
HMODULE hNtDll = LoadLibrary(TEXT("ntdll.dll"));
if(hNtDll == NULL)
{
    // Handle however.. chances of this failing
    // is essentially 0 however since
    // ntdll.dll is a vital system resource
}
 
NtQueryInfoProcess = (pfnNtQueryInformationProcess)
   GetProcAddress(hNtDll, "NtQueryInformationProcess");
if(NtQueryInfoProcess == NULL)
{
    // Handle however it fits your needs but as before,
    // if this is missing there are some SERIOUS issues with the OS
}
 
// Time to finally make the call
Ret = NtQueryInfoProcess(GetCurrentProcess(), ProcessDbgPort, 
      &IsRemotePresent, sizeof(unsigned long), NULL);
if(Ret == 0x00000000 && IsRemotePresent != 0)
{
    // Debugger is present
    MessageBox(NULL, TEXT("Please close your debugging " + 
               "application and restart the program"), 
               TEXT("Debugger Found!"), 0);
    ExitProcess(0);
}

NtGlobalFlag

NtGlobalFlag is a DWORD value inside the process PEB. This value contains many flags set by the Operating System that affects the way the process runs. When a process is being debugged, the flags FLG_HEAP_ENABLE_TAIL_CHECK (0x10), FLG_HEAP_ENABLE_FREE_CHECK(0x20), and FLG_HEAP_VALIDATE_PARAMETERS(0x40) are set for the process, and we can use this to our advantage to identify if our process is being debugged.

unsigned long NtGlobalFlags = 0;

__asm {

    mov eax, fs:[30h]
    mov eax, [eax + 68h]
    mov NtGlobalFlags, eax
}


if(NtGlobalFlags & 0x70)
// 0x70 =  FLG_HEAP_ENABLE_TAIL_CHECK |
//         FLG_HEAP_ENABLE_FREE_CHECK | 
//         FLG_HEAP_VALIDATE_PARAMETERS
{
    // Debugger is present
    MessageBox(NULL, TEXT("Please close your debugging " + 
               "application and restart the program"), 
               TEXT("Debugger Found!"), 0);
    ExitProcess(0);
}
// Normal execution

Points of Interest

There is a lot more to anti-debugging in Windows, several methods such as using CloseHandle or OutputDebugStringA are patched in Windows Vista, and therefore didn't make it into the article. I do plan to write other articles on other anti-debug techniques, many methods available are difficult to implement in inline assembler alone, and would need to be coded in another ASM flavor/compiler in order to be implemented more efficiently.

References

History

  • September 18, 2008 - Original revision.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Student
United States United States
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionMy vote of 3 Pin
ali_reza_zareian24-Aug-11 6:51
ali_reza_zareian24-Aug-11 6:51 
GeneralMy vote of 4 Pin
bacy00116-Apr-11 3:39
bacy00116-Apr-11 3:39 
GeneralMy vote of 5 Pin
gndnet27-Dec-10 3:15
gndnet27-Dec-10 3:15 
GeneralYour article Part 2 Pin
Elmue20-Sep-10 16:14
Elmue20-Sep-10 16:14 
Generalvery good article Pin
neoxquick@gmail.com20-Nov-08 3:13
neoxquick@gmail.com20-Nov-08 3:13 
GeneralAnti-debugging Reference Pin
j.newger24-Oct-08 13:12
j.newger24-Oct-08 13:12 
GeneralGood article ... but too easy to exploit Pin
galpha27-Sep-08 9:34
galpha27-Sep-08 9:34 
GeneralRe: Good article ... but too easy to exploit Pin
Aelthegrin27-Sep-08 12:07
Aelthegrin27-Sep-08 12:07 
GeneralRe: Good article ... but too easy to exploit Pin
galpha29-Sep-08 5:05
galpha29-Sep-08 5:05 
GeneralRe: Good article ... but too easy to exploit Pin
Joshua Tully29-Sep-08 17:09
Joshua Tully29-Sep-08 17:09 
GeneralGood Stuff.. Pin
enhzflep22-Sep-08 9:02
enhzflep22-Sep-08 9:02 
GeneralGood topic for discussion. Also about crackers... Pin
Shawn Poulson19-Sep-08 3:12
Shawn Poulson19-Sep-08 3:12 
GeneralRe: Good topic for discussion. Also about crackers... Pin
Joshua Tully19-Sep-08 3:41
Joshua Tully19-Sep-08 3:41 
GeneralNice article, undervalued!! Pin
jpmik19-Sep-08 0:52
jpmik19-Sep-08 0:52 
GeneralRe: Nice article, undervalued!! Pin
Joshua Tully19-Sep-08 3:44
Joshua Tully19-Sep-08 3:44 
GeneralRe: Nice article, undervalued!! Pin
aduvm6-Nov-13 19:49
professionaladuvm6-Nov-13 19:49 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.