Wouldn't it be
fmod((1/(p*p*q)), lcm((p-1), (q-1)))
?
Also, you are getting a blank screen because your while loop never converges, so you are stuck in an infinite loop. I'm also not sure what the "modulo" value is based on the information you gave, so I can't really tell you if that part is right or wrong...