This article will demonstrate how an application can detect if it is being run from inside a virtual machine software.
The code in this article will detect two well known machine virtualization software:
- Microsoft's Virtual PC (formally from Connectix).
- VMWare from VMWare.com
Other virtual machine software such as Bochs or Plex86 are not covered in this article.
It is best that the readers have a general idea about the Intel x86 assembly language to better understand how the code works, however I will do my best to explain the techniques in layman's terms.
Please note that whenever I use the term "Virtual Machine Software", this means I am referring to a software such as Virtual PC or VMWare. When the term "Virtual Machine" is used, this means the emulated machine, usually running an operating system.
A little about virtual machine software
Virtual machine software are software that emulate a given machine's architecture using software (code) instead of relying on hardware, thus allowing a code to be executed in that virtual machine as if it is being run from a real machine.
Till today, these software are far from being perfect, and emulating a given real machine still poses many challenges due to complexities involved when trying to emulate every component of a given machine.
Both Virtual PC and VMWare allow you to install "add-in"s to accelerate emulation, allow drag-n-drop from your real desktop to your virtual desktop, and allow file sharing between your real machine and the virtual machine.
In order to accomplish this task, a communication mechanism between the virtual machine software and the virtual machine itself must exist.
This sort of interfacing is called a "backdoor interfacing", since, using a special/undocumented mechanism, certain commands can be carried and interpreted in a different manner (by the virtual machine software) unlike having them interpreted by the real machine.
Next, I'll be covering how you can tell whether your software is being executed using a real machine or a virtual machine software (covering both Virtual PC and VMWare).
How to detect Virtual PC
As you may already know, every machine has a defined set of instructions commonly referred to as Instruction Set Architecture (ISA).
When an invalid instruction (that is not present in the ISA) is encountered, the machine raises an exception of the type "Invalid Opcode". The software can either handle the exception (using the usual
catch mechanism), let the operating system handle the exception, or crash the machine in worst cases.
Virtual PC uses a bunch of invalid instructions to allow the interfacing between the virtual machine and the Virtual PC software.
Here's what happens when Virtual PC's virtual machine wants to talk with Virtual PC:
- The program sets exception handlers (
- Set needed parameters before calling the VM software.
- Issue a special "Invalid Opcode" instruction.
- VM software will recognize this invalid opcode and act accordingly, causing no exception if VPC was present, and an exception if VPC isn't present.
- The program's "
catch" block will handle the exception and examine the returned parameters for the presence/absence of VM software.
In short, Virtual PC uses the "Invalid Opcode" mechanism as a backdoor.
The following code shows how to detect Virtual PC's presence:
DWORD __forceinline IsInsideVPC_exceptionFilter(LPEXCEPTION_POINTERS ep)
PCONTEXT ctx = ep->ContextRecord;
ctx->Ebx = -1; ctx->Eip += 4; return EXCEPTION_CONTINUE_EXECUTION;
bool rc = false;
_asm push ebx
_asm mov ebx, 0 _asm mov eax, 1
_asm __emit 0Fh
_asm __emit 3Fh
_asm __emit 07h
_asm __emit 0Bh
_asm test ebx, ebx
_asm setz [rc]
_asm pop ebx
More details on the code:
- Install exception handlers.
- Prepare input registers "
eax" and "
- Issue invalid instruction 0x0F 0x3F 0x07 0x0B. This invalid instruction is like a function designator, it tells Virtual PC what to do exactly. For other functionality, Virtual PC uses another invalid instruction.
- Inside the exception handler -> modify registers so to mark VPC's absence (
EBX is set to -1 if exception is triggered -> VPC is absent).
- Return from exception and resume execution (only if VPC was absent).
- Inspect returned registers accordingly.
How to detect VMWare
The Intel x86 provides two instructions to allow you to carry I/O operations, these instructions are the "
IN" and "
OUT" instructions. These two instructions are privileged instructions and cannot be used in a user-mode (while in protected mode) process unless the necessary privileges are enabled, so using them in normal cases will cause an exception of the type: "
VMWare uses the "
IN" instruction to read from a special port. This port does not effectively exist, however when VMWare is present, that port will be the interface between the virtual machine and VMWare.
Here's the code:
bool rc = true;
mov eax, 'VMXh'
mov ebx, 0
mov ecx, 10
mov edx, 'VX'
in eax, dx
cmp ebx, 'VMXh'
rc = false;
- The program sets exception handlers (in case VMWare isn't present, we just discard its presence).
- Set up into the
EAX register the magic number 0x564D5868 (or 'VMXh').
EBX register to any value but the magic number.
ECX register to the "function number" value. The value 10 means Get VMWare version, other codes means other functionality.
- Set into
DX the magic port number 0x5658 (or 'VX'), this special port number allows interfacing with VMWare when it is present.
- Read from that port into
- When VMWare is not present, an exception will occur and we discard VMWare's presence.
- Otherwise, the code flow continues.
EBX should now read the magic number value.
- If so, then VMWare is present.
Using the code
This article comes with a sample GUI written in C#/VB.NET and VC++, allowing you to detect the presence of either VMWare or Virtual PC.
The C++ code uses "DetectVM.cpp/.h" which exports the following functions:
The C#/VB.NET code uses the following read-only properties:
Hope you enjoyed and learned from reading this article and using the code. Thanks to CodeProject members for their continuous support and quality articles/code.
Special thanks goes to Mr. Ken Kato, for his work that enabled me to learn about VMWare's backdoor interface.
- 03/14/2005 - Initial version of article packed with only VC++ sample code.
- Updated the article's
- Rewritten both
IsInsideVMWare() using less ASM code.
- Added .NET samples (thanks to unruledboy for his .NET code contribution).
Elias (aka lallousx86, @0xeb) has always been interested in the making of things and their inner workings.
His computer interests include system programming, reverse engineering, writing libraries, tutorials and articles.
In his free time, and apart from researching, his favorite reading topics include: dreams, metaphysics, philosophy, psychology and any other human/mystical science.
Former employee of Microsoft and Hex-Rays (the creators of IDA Pro), was responsible about many debugger plugins, IDAPython project ownership and what not.
Elias currently works as an Anticheat engineer in Blizzard Entertainment.
Elias co-authored 2 books and authored one book:
- Practical Reverse Engineering
- The Antivirus Hacker's Handbook
- The Art of Batch Files Programming