Don't try to work out a time difference by assuming that you can subtract hours from hours, minutes from minutes and so on.
Think about it: how do you work out 45 minutes ago? By subtracting 45 from the current minutes and winding teh hour back if necessary.
The easest way to do this is to convert both times into a number of seconds:
totalSeconds = ((hours * 60 + minutes) * 60) + seconds
Then subtract one from the other, and convert the result back to minutes, and then hours by divide and modulus by 60.