How do I check if the window is open and shown on the screen and e.g. There's no lag?

Question

0.00/5 (No votes)

See more:

I want to make sure that the window is open and shown on the screen (with support to more than one monitor of course), because I'm ordering Windows to fake a mouse click on the form so Google Translate textbox in the browser gets selected (I've tried a lot to select it but it doesn't work as not always it gets selected), so I don't want the mouse click to click on something else, I have made a 3x3 square of colors, and I want to check on the screen if these 9 colors are really on the screen then fast order the fake mouse click, I don't get the right colors when I test with this code:

What I have tried:

private void FormMain_Paint(object sender, PaintEventArgs e)
 {
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50, 1, Color.Gray);
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50 + 1,1, Color.DarkGray);
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50 + 2, 1, Color.Brown);
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50, 2, Color.RosyBrown);
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50 + 1, 2, Color.DarkSlateGray);
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50 + 2, 2, Color.DarkGray);
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50, 3, Color.SlateGray);
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50 + 1, 3, Color.LightSlateGray);
     ClassSetGetPixel.SetPixel(this, pictureBoxSettings.Right + 50 + 2, 3, Color.LightGray);
 }

if (formMain.WindowState != FormWindowState.Minimized)
                {
                    if ((GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50, 1))) == Color.Gray) &&
                     (GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50 + 1, 1))) == Color.DarkGray) &&
                     (GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50 + 2, 1))) == Color.Brown) &&
                     (GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50, 2))) == Color.RosyBrown) &&
                     (GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50 + 1,2))) == Color.DarkSlateGray) &&
                     (GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50 + 2, 2))) == Color.DarkGray) &&
                     (GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50, 3))) == Color.SlateGray) &&
                     (GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50 + 1, 3))) == Color.LightSlateGray) &&
                     (GetPixel(formMain.PointToScreen(new Point(formMain.pictureBoxSettings.Right + 50 + 2, 3))) == Color.LightGray))
                    {
                        Program.DoMouseClick(formMain.Location.X + formMain.pictureBoxSettings.Right + 1, formMain.Location.Y + formMain.pictureBoxSettings.Top + 100);
                    }
                }

static public void SetPixel(Control control, int x, int y, Color color)
        {
            if (control != null)
            {
                IntPtr hDC = GetDC(control.Handle);
                int argb = color.ToArgb();
                int colorRef =
                    (int)((argb & 0x00FF0000) >> 16) |
                    (int)(argb & 0x0000FF00) |
                    (int)((argb & 0x000000FF) << 16);
                SetPixel(hDC, x, y, colorRef);
                ReleaseDC(control.Handle, hDC);
            }
        }

static Color GetPixel(Point p)
{
    using (var bitmap = new Bitmap(1, 1))
    {
        using (var graphics = Graphics.FromImage(bitmap))
        {

            graphics.CopyFromScreen(p, new Point(0, 0), new Size(1, 1));
        }
        return bitmap.GetPixel(0, 0);
    }
}

Posted 26-Mar-20 22:37pm

john1990_1

Updated 28-Mar-20 6:17am

#realJSOP

v4

Add a Solution

Comments

Richard MacCutchan 27-Mar-20 5:47am

What colours do you get? Are you sure that your pixel addresses are correct?

john1990_1 27-Mar-20 5:50am

I made a Color c=GetColor(); of the first address and it returned a random and unnamed color (as "Named color" was false), however when I do a mouse click on the first location it presses correctly on the location of the drawn pixels... maybe it's because of blurr from the bar of the form (window) and I should get the pixels a little down to run away from the blurr.

john1990_1 27-Mar-20 5:54am

Putting the pixels at location Y=20,21,22 didn't work.

Richard MacCutchan 27-Mar-20 6:00am

On of the problems with this may be that Windows controls the physical screen and often colours will be merged, so that what you paint into a form is actually modified when it shows on the screen.

john1990_1 27-Mar-20 6:01am

I need to be sure, so I give up if that's true, how can we know for sure whether this is the problem?

Richard MacCutchan 27-Mar-20 6:38am

You need to use the debugger to examine exactly what pixel addresses are being returned from the PointToScreen call, and what colour values at each point.

john1990_1 27-Mar-20 6:42am

When I invoke a mouse move to the location of PointToScreen it goes to the middle of my 3x3 square. Would you want to see the full VS source code? how can I share this?

Richard MacCutchan 27-Mar-20 8:31am

Sorry, no I don't want to see the full source as I don't have the time to study all your code.

john1990_1 27-Mar-20 8:32am

No problem, any guess as why this isn't working?

Richard MacCutchan 27-Mar-20 8:42am

In my experience guessing never works. The only way to find out is by using the debugger to see what happens inside the code.

john1990_1 27-Mar-20 9:50am

I think the problem is because I have 4 screens, how to tell GetPixel to look for the point in 4 screens coordinates. (e.g. 3 on the table and one above)

Richard MacCutchan 27-Mar-20 9:53am

See windows - WinApi - Get Monitor Index on which a process is running - Stack Overflow[^]

john1990_1 27-Mar-20 9:59am

What if the windows is on 2 or more monitors at the same time?

Richard MacCutchan 27-Mar-20 10:02am

Then you are going to have to try a few things and check the MSDN documentation.

john1990_1 27-Mar-20 12:02pm

I added my method of "GetPixel", check it out plz.

Richard MacCutchan 27-Mar-20 12:16pm

Sorry, but I am not sure what you are asking. As I said earlier you need to spend time with the debugger to check exactly what is happening in your code. It is not something that anyone else can do for you.

john1990_1 27-Mar-20 12:19pm

I debugged for a long time and reached a dead end, what if it's about Windows screens zoom?

john1990_1 27-Mar-20 12:21pm

It's about Windows desktop screens zoom being at 150%, it worked when I made the zoom 100%, how to fix that (make the program take zooming in consideration)?

Richard MacCutchan 27-Mar-20 12:43pm

You have probably reached the point where your original colours have been 'adjusted' by Windows in order to display the zoomed images. But I have no idea what you can do in such a case. You are probably going to have to rethink what you are trying to achieve.

john1990_1 27-Mar-20 12:47pm

The colors are the exact same, it's just that I can't get to their true location as one way is giving me the location at 100% zoom and one at 150% zoom, I tested it and I saw it gets the image not in the right location, but when I made the zoom at 100% in all screens, I got the true pic of the form (when I sent the coordinates of the form and size at new Size(200,200)).

john1990_1 27-Mar-20 12:49pm

I want to set focus input on the source textbox in the web browser control in the form when the form opens or F1 is pressed, I tried for a really long time but no way worked flawlessly except invoking a mouse click on an empty pixel on the form.

Richard MacCutchan 27-Mar-20 13:04pm

It is not something that I have done but there may be .NET classes that can find controls on another form.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

#realJSOP · Answer 1 · 2020-03-28T02:25:00

Solution 1

Visual Studio comes with a tool called "Spy++". Use that to find controls in a given window. To manipulate a control in a window in an external app, you're going to need to use System.Runtime.InteropServices.

You're gonna need to search for the tool because VS no longer puts a shortcut to it in the windows Start menu. I found it and added a custom item under the Tools menu in VS.

Posted 28-Mar-20 2:25am

#realJSOP

Comments

john1990_1 28-Mar-20 8:28am

Does this way make sure the window isn't in lag and is in fact shown on the screen with colors? I mean is this the same as if one called GetPixel on the display it detects the colors shown on the window and it isn't opening or in lag?

#realJSOP 28-Mar-20 10:03am

Has nothing to do with colors - just allows you to find windows and controls that are currently active in windows.Z However, once you find the desired control window, you can discover whatever window properties you might want. This is going to require some googling on your part. You can use Spy++ to inspect a given window to see if you can get a lead on what to look for. Caveat, there's no "fast way" to find out what to do, unless you happen to stumble across a fairly complete assembly that contains all of the necessary InteropServices calls.

john1990_1 28-Mar-20 10:07am

I want to invoke a mouse click on an empty point on the form to select input to Google Translate's source textbox, I want to make sure the form is shown, I want to checks colors on the display that are in the form, so the click doesn't go to liking a post on FB or something, how do I make sure the window is not in lag and its colors are actually on the display? does the method you're suggesting make sure the click doesn't go to another window if my program is in lag for being displayed on the display?

#realJSOP 28-Mar-20 10:32am

You can use InteropServices to send a window an event. I don't know (or care) what you want to do, but it can (and probably requires) be done with interopservices since it involves another window in an external app. Like I mentioned before, google is your friend.

BTW, a good "empty point" is the top/left pixel of the window's client area. There's almost never a control there. The color of the pixel at that point is inconsequential.

john1990_1 28-Mar-20 10:37am

You don't understand, I have a Windows Forms program, in this program there's a web browser control, in this web browser control Google Translate is sometimes navigated to, I want to focus on the "source" textbox in the web browser control, I tried many way and I didn't find a way that always does select the input on the source textbox, I fake a mouse click from my program on my program, on an empty place in my program (not on the browser control), I want this click from my program to my program, I want to make sure that my program is shown on the display and isn't lagging to open, because if my program thinks my program is shown on the display and it does a click thinking my program is there to catch the click, the click my be clicked on "Like" on FB if my window of my program wasn't shown on the display!

#realJSOP 28-Mar-20 11:03am

0) For the record, your problem descriptions suck.

1) You don't need to "simulate a mouse click". Simply call the control's Focus() method.

2) Find out what the id of the textbox is. You can do this by going to the google translate page in an external browser and viewing the source.

3) In your code, use HTMLAgilityPack (install it in your project with NuGet) to parse the page and find the control when it's browsed to in your app. From there, it's a simply matter of manipulating the DOM to do whatever it is you want to do.

john1990_1 28-Mar-20 11:07am

I did this a zillion times, I call wb.Focus(); and wb.Document.GetElementByID("source").InvokeMember("focus"); and many things like these. It gets selected only sometimes, and I realized this can't be done this way, if you can fix it like this then awesome! otherwise plz help me make sure the fake mouse click doesn't go to something else on that point in the display.

#realJSOP 28-Mar-20 12:02pm

I'm merely grasping at straws here, but try InvokeMember("click") or InvokeMember("mouseup").

Also, do it this way:

HTMLNode ctrl = wb.Document.GetElementByID("source"); 
if (ctrl != null)
{
    ctrl.InvokeMember("click");
}

That way you can put a breakpoint in there and see what's happening.

john1990_1 28-Mar-20 12:46pm

I will try to publish the program to my hard drive for testing, would you please tell me all methodnames in "InvokeMember" or things I can ask my program to do on the web browser so the source textbox gets selected? For example maybe "InvokeScript" does a better work to select the source textbox?

#realJSOP 28-Mar-20 12:59pm

F*ckin google it dude. Hint - they're javascript events.

Richard MacCutchan 28-Mar-20 13:07pm

Don't hold back John, say what you feel. :) :)

john1990_1 28-Mar-20 13:21pm

This is all I could come up with:
HtmlElement textArea1 = formMain.webBrowserGoogleTranslate.Document.GetElementById("source");
if (textArea1 != null)
{
formMain.webBrowserGoogleTranslate.Document.InvokeScript("'click'", new object[] { textArea1 });
}

#realJSOP 28-Mar-20 13:36pm

This guy can't possibly be from the US (as indicated in his profile). US programmers aren't generally this f*ckin helpless...

john1990_1 28-Mar-20 13:45pm

I know only C# from school and I'm a hobbyist, I study biology, would you please help me fix the risk in my program of making sure none of my friends who use my program and me end up with a random mouse click in Facebook?