JavaFX.Scene.Robot crashes / UI freezes

Question

I am trying to write a simple JavaFX app which acts as an auto clicker for a game I play. I choose two Points that the mouse should click alternately. Everything works fine until the Robot needs to do his work. When I put it like this:

robot.mouseMove(join);
Thread.sleep(2000);
robot.mouseClick(MouseButton.PRIMARY);
Thread.sleep(2000);
robot.mouseMove(accept);
Thread.sleep(2000);
robot.mouseClick(MouseButton.PRIMARY);
Thread.sleep(2000);

my App crashes. I've read things up online and it seems like you should not sleep in the JavaFX application thread. My new approach was to create a new thread that takes care of the clicking from the application thread like this:

clicker = new Clicker(join, accept);
Thread clickerThread = new Thread(clicker);
clickerThread.start();

And here how it looks in Clicker:

 public void run() {
        while (running){
            try {
                robot.mouseMove(join);
                Thread.sleep(2000);
                robot.mouseClick(MouseButton.PRIMARY);
                Thread.sleep(2000);
                robot.mouseMove(accept);
                Thread.sleep(2000);
                robot.mouseClick(MouseButton.PRIMARY);
                Thread.sleep(2000);
            } catch (InterruptedException e) {
                System.out.println("Clicker sleep interrupted!");
            }
        }
    }

However with the new approach I suddenly get this error:

Exception in thread "Thread-3" java.lang.IllegalStateException: This operation is permitted on the event thread only; currentThread = Thread-3

Does anyone know how I could fix this problem?

[Use an animation](https://stackoverflow.com/questions/9966136/javafx-periodic-background-task). — Slaw, Jul 03 '20 at 09:32

Slaw · Answer 1 · 2020-07-03T10:24:53.453

1

When you want to execute a periodic foreground task on the JavaFX Application Thread you should first consider using an animation. Here's an example of using a Timeline:

Point2D join = ...;
Point2D accept = ...;
Robot robot = ...;

Timeline timeline = new Timeline(
    new KeyFrame(Duration.ZERO, e -> robot.mouseMove(join)),
    new KeyFrame(Duration.seconds(2), e -> robot.mouseClick(MouseButton.PRIMARY)),
    new KeyFrame(Duration.seconds(4), e -> robot.mouseMove(accept)),
    new KeyFrame(Duration.seconds(6), e -> robot.mouseClick(MouseButton.PRIMARY)),
    new KeyFrame(Duration.seconds(8))
);
timeline.play();

The above will execute each KeyFrame's on-finished handler two seconds after the previous one (the first one immediately after the animation is started). The last KeyFrame matches your final call to sleep though it may not be necessary. All this will occur on the JavaFX Application Thread.

You can configure an animation to replay a certain number of times, or even forever, by setting its cycleCount property. For example:

timeline.setCycleCount(5); // play 5 times then stop
// or
timeline.setCycleCount(Animation.INDEFINITE); // play forever

edited Jul 03 '20 at 10:24

answered Jul 03 '20 at 10:01

Slaw

37,820
8
53
80

Thank you for your fast answer! But how do I manage to play the timeline over and over again? When I try to run timeline.play() in a while loop the app crashes again. – arrabiataHunter Jul 03 '20 at 10:20
@moanever Don't use a `while` loop. I updated my answer. Note that the call to `play()` is _asynchronous_. The calling thread will return immediately and the animation will play asynchronously (assuming you don't monopolize the FX thread). – Slaw Jul 03 '20 at 10:25
I see, thank you. Though .setCycleCount() might not do the trick for me as I want to stop the bot when a certain key is pressed thus I need some kind of boolean running that should be checked before the animation is played again. Maybe the approach with Platform.runLater() is better in that case? – arrabiataHunter Jul 03 '20 at 10:42
@moanever Just call `timeline.stop()` when you want it to stop, then call `play()` again when you want it to start playing again. This can be accomplished from within your key-event handler. – Slaw Jul 03 '20 at 10:44

score 0 · Answer 2 · answered Jul 03 '20 at 09:51

You need to execute the mouseMove on the UI thread try wrapping the calls with Platform.runlater(() -> robot.mouseMove(MouseButton.PRIMARY));

public void run() {
        while (running){
            try {
                Platform.runlater(() -> robot.mouseMove(join));
                Thread.sleep(2000);
                Platform.runlater(() -> robot. mouseClick(MouseButton.PRIMARY));
                Thread.sleep(2000);
                Platform.runlater(() -> robot.mouseMove(accept));
                Thread.sleep(2000);
            } catch (InterruptedException e) {
                System.out.println("Clicker sleep interrupted!");
            }
        }
    }

Platform.runLater() will schedule an action on the main thread and return immediately. It may take some time before the action is actually executed, in small applications this delay is usually not perceptible. If the delay were long it may happen that you schedule your action, sleep and schedule again before the first action was executed. If you want your clickerThread to wait for the action to execute before continuing then you will need some form of synchronisation, see below for an example of a method that wraps Platform.runlater() with a lock.

public void waitUntilExecutedOnMainThread(Runnable runnable){
    Semaphore semaphore = new Semaphore(0);
    Platform.runLater(() -> {
        try {
            runnable.run();
        } finally {
            semaphore.release();
        }
    });
    try {
        semaphore.acquire();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}

waitUntilExecutedOnMainThread(() -> robot. mouseClick(MouseButton.PRIMARY))

Thanks for your fast answer! Although this looks like it wasn't intended to be used like this it still works haha — arrabiataHunter, Jul 03 '20 at 10:22

JavaFX.Scene.Robot crashes / UI freezes

2 Answers2