How to programmatically (C# or Java) launch an app in Windows and invoke click in it's window?

Question

There is a simple application that works in Windows. It has very simple interface: squre window with buttons in fixed coordinates.

I need to write a program that makes use of this application: to launch it and to click one of buttons (let's say invoke a click at (150,200)).

Is there any way to do it in Java or .NET?

score 3 · Accepted Answer · answered Aug 01 '12 at 12:53

3

The Java based solution is to launch the app. in a Process and use the Robot to interact with it.

The best solution on this thread was by @HFoE but deleted by a moderator. For reference, it basically came down to..

If you want to control another Windows application, use a tool that was built specifically for this such as AutoIt V3.

Since "Don't do it" seems to be considered a valid answer when an alternative is supplied (by general opinion on Meta), I cannot understand why the answer was deleted.

answered Aug 01 '12 at 12:53

Andrew Thompson

168,117
40
217
433

Thanks I dream to try this too some day)) – Bohdan Aug 01 '12 at 13:25
Does this Robot thing let us to "capture" windows or play with particular windows but not OS itself? – Bohdan Aug 01 '12 at 13:26
The `java.awt.Robot` can click on buttons that are on-screen, but does not interact with the OS at a low level. – Andrew Thompson Aug 01 '12 at 13:28

score 2 · Answer 2 · answered Aug 01 '12 at 12:29

As Hovercraft Full Of Eels if you can - use autoit - it's much easier. If AutoIt is not an option then you will need to use winAPI functions in order to do it.

For example to call mouseclick at coordinates:

[DllImport("user32.dll")]
static extern bool SetCursorPos(int x, int y);

[DllImport("user32.dll")]
static extern bool GetCursorPos(ref Point lpPoint);

[DllImport("user32.dll")]
public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);

public void LeftMouseClick(int xpos, int ypos) //Make a click at specified coords and return mouse back
{
    Point retPoint = new Point();
    GetCursorPos(ref retPoint); // set retPoint as mouse current coords
    SetCursorPos(xpos, ypos); //set mouse cursor position
    mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0); 
    mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0); //click made
    SetCursorPos(retPoint.X, retPoint.Y); //return mouse position to coords
}

But be aware, that to make click inside a window it needs to be at front of you - you cannot click to a minimized app for example.

If you want to try - you can find all needed functions(how to run a programm, get needed window by hwnd and so on) at PInvoke

score 1 · Answer 3 · answered Aug 01 '12 at 12:16

For .Net you can pretty much use AutomationElement which I prefer. There's a bit of learning time, but it shouldn't take much. You can start your app with ProcessStartInfo.

If you have VS2010 Pro or Ultimate you can use the CodedUITests to generate a couple of button pushes.

As @Hovercraft Full Of Eels suggested - Autoit, Python could do the same

score 1 · Answer 4 · edited May 23 '17 at 12:18

Yes - in C#...

Use the Process class to start the process (there are plenty of resources on the web on how to do this.
Wait until the process has started (either just wait for a fixed amount of time which is probably going to be long enough, or you could try and do something fancy like IPC or monitoring for a window being created)
To simulate the click take a look at How to simulate Mouse Click in C#? which uses a P/Invoke call to the mouse_event function.

However note that there are several things that can go wrong with this

Someone might move the window, or place another window on top of that window in the time it takes to launch the application
On a slower PC it may take longer to load the application (this risk can be mitigated by doing things like monitoring open windows and waiting for the expected application window to appear)

score 1 · Answer 5 · edited May 23 '17 at 10:24

1

In .net you can Process.Start from System.Diagnostics to launch an application, you can even pass parameters, and to simulate mouse events you can use P/Invoke there is already an answer to that on SO here

edited May 23 '17 at 10:24

Community

1
1

answered Aug 01 '12 at 12:19

Vamsi

4,237
7
49
74

score 0 · Answer 6 · answered Aug 01 '12 at 13:24

Here is my working test app to play with clicking in windows. We just start some app and hope to click it in right place) It would be nice to have some solution for capturing windows this way =)

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;

namespace ConsoleApplication8
{
    class Program
    {
        static void Main(string[] args)
        {
            var startInfo = new ProcessStartInfo(@"C:\Users\Bodia\Documents\visual studio 2010\Projects\ConsoleApplication8\WindowsFormsApplication1\bin\Debug\WindowsFormsApplication1.exe");
            startInfo.WindowStyle = ProcessWindowStyle.Maximized;

            Console.WriteLine(1);

            var process = Process.Start(startInfo);
            Console.WriteLine(2);

            Thread.Sleep(400);
            Console.WriteLine(3);

            LeftMouseClick(1000, 200);
            Console.WriteLine(4);
        }

        static void CursorFun()
        {
            Point cursorPos = new Point();
            GetCursorPos(ref cursorPos);

            cursorPos.X += 100;
            Thread.Sleep(1000);
            SetCursorPos(cursorPos.X, cursorPos.Y);

            cursorPos.X += 100;
            Thread.Sleep(1000);
            SetCursorPos(cursorPos.X, cursorPos.Y);

            cursorPos.X += 100;
            Thread.Sleep(1000);
            SetCursorPos(cursorPos.X, cursorPos.Y);

            cursorPos.X += 100;
            Thread.Sleep(1000);
            SetCursorPos(cursorPos.X, cursorPos.Y);

        }

        [DllImport("user32.dll")]
        static extern bool SetCursorPos(int x, int y);

        [DllImport("user32.dll")]
        static extern bool GetCursorPos(ref Point lpPoint);

        [DllImport("user32.dll")]
        public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);

        public static void LeftMouseClick(int xpos, int ypos) //Make a click at specified coords and return mouse back
        {
            Point retPoint = new Point();
            GetCursorPos(ref retPoint); // set retPoint as mouse current coords
            SetCursorPos(xpos, ypos); //set mouse cursor position
            mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0);
            mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0); //click made
            SetCursorPos(retPoint.X, retPoint.Y); //return mouse position to coords
        }
        struct Point
        {
            public int X;
            public int Y;
        }

        private const int MOUSEEVENTF_LEFTDOWN = 0x02;
        private const int MOUSEEVENTF_LEFTUP = 0x04;
        private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
        private const int MOUSEEVENTF_RIGHTUP = 0x10;
    }
}

How to programmatically (C# or Java) launch an app in Windows and invoke click in it's window?

6 Answers6

Linked