It is available, but this is not just the include file.
Including a header file itself does not make libraries accessible. You also need to link them. With Visual Studio, you can set all appropriate options (in particular, telling the compiler and the linker to look for certain files in certain directories) just by using one of the available Win32-related project template. If you create a project from a template, everything will be set for you. Then you can look for linker and compiler options and find out where it what.
Make sure you include the header files correctly. For "windows.h" it should be:
#include <windows.h>
Please see:
http://en.wikipedia.org/wiki/Windows.h[
^].
—SA