Start by using the debugger to check exactly what is happening: put a breakpoint at the start of each method and run your app.
You should almost immediately get a Paint event so remove or disable it, continue the app, and then put it back (making sure you don't invalidate the app form with VS).
Then hold the mouse button down ready to drag. Did that event happen? If so, follow the code through and make sure all your data looks right. Remove or disable the breakpoint temporarily.
Continue your app and do it again, this time hold and drag.
You should get the third method breakpoint firing. follwo teh code through, make sure it all looks right.
Continue your app.
Did you get another Paint event for your Panel?
If not, change
this.Invalidate();
to
panel1.Invalidate();
and try again.