Pyautogui: Mouse Movement with bezier curve

Question

I'm trying to move the mouse in a bezier curve motion in Pyautogui to simulate more of a human movement as seen here:

There is some tweening / easing functions within pyautogui but none of which represent a bezier curve type movement. I created a small script to calculate the random places it will hit before ultimately hitting its destination.

Default "Robot" linear path:

Unfortunately, which each destination the mouse temporarily stops.

import pyautogui
import time
import random
print "Randomized Mouse Started."
destx = 444;
desty = 631;
x, y = pyautogui.position() # Current Position
moves = random.randint(2,4)
pixelsx = destx-x
pixelsy = desty-y
if moves >= 4:
        moves = random.randint(2,4)
avgpixelsx = pixelsx/moves
avgpixelsy = pixelsy/moves
print "Pixels to be moved X: ", pixelsx," Y: ",pixelsy, "Number of mouse movements: ", moves, "Avg Move X: ", avgpixelsx, " Y: ", avgpixelsy

while moves > 0:
        offsetx = (avgpixelsx+random.randint(-8, random.randint(5,10)));
        offsety = (avgpixelsy+random.randint(-8, random.randint(5,10)));
        print x + offsetx, y + offsety, moves
        pyautogui.moveTo(x + offsetx, y + offsety, duration=0.2)
        moves = moves-1
        avgpixelsx = pixelsx / moves
        avgpixelsy = pixelsy / moves

Info:

• Windows 10
• Python 2.7
• Willing to use other libraries, Python version if necessary

I've seen this post: python random mouse movements

but can't figure out how to define a "start and stop" position. The answer is pretty close to what I'm looking for.

Any ideas on how to accomplish this?

Answer 1

Using scipy, numpy and anything that can simply move mouse cursor:

import pyautogui
import random
import numpy as np
import time
from scipy import interpolate
import math

def point_dist(x1,y1,x2,y2):
    return math.sqrt((x2 - x1) ** 2 + (y2 - y1) ** 2)

cp = random.randint(3, 5)  # Number of control points. Must be at least 2.
x1, y1 = pyautogui.position()  # Starting position

# Distribute control points between start and destination evenly.
x = np.linspace(x1, x2, num=cp, dtype='int')
y = np.linspace(y1, y2, num=cp, dtype='int')

# Randomise inner points a bit (+-RND at most).
RND = 10
xr = [random.randint(-RND, RND) for k in range(cp)]
yr = [random.randint(-RND, RND) for k in range(cp)]
xr[0] = yr[0] = xr[-1] = yr[-1] = 0
x += xr
y += yr

# Approximate using Bezier spline.
degree = 3 if cp > 3 else cp - 1  # Degree of b-spline. 3 is recommended.
                                  # Must be less than number of control points.
tck, u = interpolate.splprep([x, y], k=degree)
# Move upto a certain number of points
u = np.linspace(0, 1, num=2+int(point_dist(x1,y1,x2,y2)/50.0))
points = interpolate.splev(u, tck)

# Move mouse.
duration = 0.1
timeout = duration / len(points[0])
point_list=zip(*(i.astype(int) for i in points))
for point in point_list:
    pyautogui.moveTo(*point)
    time.sleep(timeout)

And you can remove any built-in delay in pyautogui by setting:

# Any duration less than this is rounded to 0.0 to instantly move the mouse.
pyautogui.MINIMUM_DURATION = 0  # Default: 0.1
# Minimal number of seconds to sleep between mouse moves.
pyautogui.MINIMUM_SLEEP = 0  # Default: 0.05
# The number of seconds to pause after EVERY public function call.
pyautogui.PAUSE = 0  # Default: 0.1

P.S.: Example above doesn't require any of those settings as it doesnt use public moveTo method.

Answer 2

For a simple solution, you can try using numpy with the bezier library:

import pyautogui
import bezier
import numpy as np


# Disable pyautogui pauses (from DJV's answer)
pyautogui.MINIMUM_DURATION = 0
pyautogui.MINIMUM_SLEEP = 0
pyautogui.PAUSE = 0

# We'll wait 5 seconds to prepare the starting position
start_delay = 5 
print("Drawing curve from mouse in {} seconds.".format(start_delay))
pyautogui.sleep(start_delay)

# For this example we'll use four control points, including start and end coordinates
start = pyautogui.position()
end = start[0]+600, start[1]+200
# Two intermediate control points that may be adjusted to modify the curve.
control1 = start[0]+125, start[1]+100
control2 = start[0]+375, start[1]+50

# Format points to use with bezier
control_points = np.array([start, control1, control2, end])
points = np.array([control_points[:,0], control_points[:,1]]) # Split x and y coordinates

# You can set the degree of the curve here, should be less than # of control points
degree = 3
# Create the bezier curve
curve = bezier.Curve(points, degree)
# You can also create it with using Curve.from_nodes(), which sets degree to len(control_points)-1
# curve = bezier.Curve.from_nodes(points)

curve_steps = 50  # How many points the curve should be split into. Each is a separate pyautogui.moveTo() execution
delay = 1/curve_steps  # Time between movements. 1/curve_steps = 1 second for entire curve

# Move the mouse
for i in range(1, curve_steps+1):
    # The evaluate method takes a float from [0.0, 1.0] and returns the coordinates at that point in the curve
    # Another way of thinking about it is that i/steps gets the coordinates at (100*i/steps) percent into the curve
    x, y = curve.evaluate(i/curve_steps)
    pyautogui.moveTo(x, y)  # Move to point in curve
    pyautogui.sleep(delay)  # Wait delay

I came up with this trying to write something to draw SVG Paths with the mouse. Running the above code will make your mouse move along the same path as below. The red dots are positioned at each of the control points that define the curve.

Note that you'll have to add pyautogui.mouseDown() before and pyautogui.mouseUp() after the loop at the end of the script if you want to click and drag like I did here in GIMP:

You can check out the bezier docs here: https://bezier.readthedocs.io/en/stable/index.html

Answer 3

you just need know is the move_mouse((300,300))will let you mouse arrive (300,300),then never change.look at the implement,it just call the WIN32 api mouse_event.read something about it,you will find there are no "start and stop" position.i don't know how to draw bezier curve.

    while True:
        pos = (random.randrange(*x_bound),random.randrange(*y_bound))
        move_mouse(pos)
        time.sleep(1.0/steps_per_second)

look,that is the secret of animation.all you need do is write a pos = draw_bezier_curve(t)

Answer 4

A bit of an update (2023) - I've been using this for a while. This method uses a combination of numpy, pytweening and pyautogui:

import numpy as np
import pyautogui as pag
import pytweening
from pyclick import HumanCurve

class MouseUtils:
    def move_to(destination: tuple, **kwargs):
        # sourcery skip: use-contextlib-suppress
        """
        Use Bezier curve to simulate human-like mouse movements.
        Args:
            destination: x, y tuple of the destination point
            destination_variance: pixel variance to add to the destination point (default 0)
        Kwargs:
            knotsCount: number of knots to use in the curve, higher value = more erratic movements
                        (default determined by distance)
            mouseSpeed: speed of the mouse (options: 'slowest', 'slow', 'medium', 'fast', 'fastest')
                        (default 'fast')
            tween: tweening function to use (default easeOutQuad)
        """
        offsetBoundaryX = kwargs.get("offsetBoundaryX", 100)
        offsetBoundaryY = kwargs.get("offsetBoundaryY", 100)
        knotsCount = kwargs.get("knotsCount", MouseUtils.__calculate_knots(destination))
        distortionMean = kwargs.get("distortionMean", 1)
        distortionStdev = kwargs.get("distortionStdev", 1)
        distortionFrequency = kwargs.get("distortionFrequency", 0.5)
        tween = kwargs.get("tweening", pytweening.easeOutQuad)
        mouseSpeed = kwargs.get("mouseSpeed", "fast")
        mouseSpeed = MouseUtils.__get_mouse_speed(mouseSpeed)

        dest_x = destination[0]
        dest_y = destination[1]

        start_x, start_y = pag.position()
        for curve_x, curve_y in HumanCurve(
            (start_x, start_y),
            (dest_x, dest_y),
            offsetBoundaryX=offsetBoundaryX,
            offsetBoundaryY=offsetBoundaryY,
            knotsCount=knotsCount,
            distortionMean=distortionMean,
            distortionStdev=distortionStdev,
            distortionFrequency=distortionFrequency,
            tween=tween,
            targetPoints=mouseSpeed,
        ).points:
            pag.moveTo((curve_x, curve_y))
            start_x, start_y = curve_x, curve_y

    def move_rel(self, x: int, y: int, x_var: int = 0, y_var: int = 0, **kwargs):
        """
        Use Bezier curve to simulate human-like relative mouse movements.
        Args:
            x: x distance to move
            y: y distance to move
            x_var: random upper-bound pixel variance to add to the x distance (default 0)
            y_var: random upper-bound pixel variance to add to the y distance (default 0)
        Kwargs:
            knotsCount: if right-click menus are being cancelled due to erratic mouse movements,
                        try setting this value to 0.
        """
        if x_var != 0:
            x += np.random.randint(-x_var, x_var)
        if y_var != 0:
            y += np.random.randint(-y_var, y_var)
        self.move_to((pag.position()[0] + x, pag.position()[1] + y), **kwargs)


    def __calculate_knots(destination: tuple):
        """
        Calculate the knots to use in the Bezier curve based on distance.
        Args:
            destination: x, y tuple of the destination point
        """
        # calculate the distance between the start and end points
        distance = np.sqrt((destination[0] - pag.position()[0]) ** 2 + (destination[1] - pag.position()[1]) ** 2)
        res = round(distance / 200)
        return min(res, 3)

    def __get_mouse_speed(speed: str) -> int:
        """
        Converts a text speed to a numeric speed for HumanCurve (targetPoints).
        """
        if speed == "slowest":
            return rd.randint(85, 100)
        elif speed == "slow":
            return rd.randint(65, 80)
        elif speed == "medium":
            return rd.randint(45, 60)
        elif speed == "fast":
            return rd.randint(20, 40)
        elif speed == "fastest":
            return rd.randint(10, 15)
        else:
            raise ValueError("Invalid mouse speed. Try 'slowest', 'slow', 'medium', 'fast', or 'fastest'.")