I go through a similar exercise every time I had a new source code module to a project. It's a lot easier now since I have sorted out all of my headers so that they are essentially "self-contained." By that I mean if I need to use a class I can include its header file and it will include everything needed to compile a module with it. I think that is really the key - the headers themselves have to be sorted out properly so it becomes a simple matter to just include it for a class or function you need to use and move on.
In my opinion, a header file should not require other files to be included in addition to it. I think it should include everything needed to compile itself and do so in a well-behaved way. What that means to me is if it includes a file that uses
#pragma once
then it is well behaved. If it does not then I use its include guard to include it only if necessary, meaning only if not already included. I use Visual C++ and most of its headers use the pragma but not all do. Here's an example with a file that does not use the pragma :
#ifndef _SHLOBJ_H_
#include <ShlObj.h>
#endif