Product
Introducing License Enforcement in Socket
Ensure open-source compliance with Socket’s License Enforcement Beta. Set up your License Policy and secure your software!
For more information, see the GitHub Repository.
AutoPy is a simple, cross-platform GUI automation library for Python. It includes functions for controlling the keyboard and mouse, finding colors and bitmaps on-screen, and displaying alerts.
Currently supported on macOS, Windows, and X11 with the XTest extension.
First, see if a binary wheel is available for your machine by running:
$ pip install -U autopy
If that fails, install rustup and then run:
$ rustup default nightly-2019-10-05
$ pip install -U setuptools-rust
$ pip install -U autopy
Another option is to build from the latest source on the GitHub repository:
$ git clone git://github.com/autopilot-rs/autopy-rs.git
$ cd autopy
$ make
$ make install
Note: AutoPy currently requires the 2019-10-05
Rust nightly in order to
build from source. This is to maintain compatibility with an older version of
PyO3, as the latest version has dropped Python 2 support. Python 2 support will
likely be dropped from AutoPy as well sometime later this year, depending on
how necessary it is to upgrade to a more recent version of PyO3 or Rust. In the
meantime, it may be necessary to install the required nightly via the following
when building locally:
rustup install nightly 2019-10-05 --force
This is due to rustup complaining that it doesn't include certain components
such as rustfmt
.
Additional instructions for installing from source on Windows are available here.
The following is the source for a "hello world" script in autopy. Running this code will cause an alert dialog to appear on every major platform:
import autopy
def hello_world():
autopy.alert.alert("Hello, world")
hello_world()
AutoPy includes a number of functions for controlling the mouse. For a full list, consult the API Reference. E.g., to immediately "teleport" the mouse to the top left corner of the screen:
>>> import autopy
>>> autopy.mouse.move(1, 1)
To move the mouse a bit more realistically, we could use:
>>> import autopy
>>> autopy.mouse.smooth_move(1, 1)
Even better, we could write our own function to move the mouse across the screen as a sine wave:
import autopy
import math
import time
import random
import sys
TWO_PI = math.pi * 2.0
def sine_mouse_wave():
"""
Moves the mouse in a sine wave from the left edge of
the screen to the right.
"""
width, height = autopy.screen.size()
height /= 2
height -= 10 # Stay in the screen bounds.
for x in range(int(width)):
y = int(height * math.sin((TWO_PI * x) / width) + height)
autopy.mouse.move(x, y)
time.sleep(random.uniform(0.001, 0.003))
sine_mouse_wave()
The following will enter the keys from the string "Hello, world!" in the currently focused input at 100 WPM:
import autopy
autopy.key.type_string("Hello, world!", wpm=100)
Alternatively, individual keys can be entered using the following:
import autopy
autopy.key.tap(autopy.key.Code.TAB, [autopy.key.Modifier.META])
autopy.key.tap("w", [autopy.key.Modifier.META])
All of autopy's bitmap routines can be found in the module autopy.bitmap
. A
useful way to explore autopy is to use Python's built-in help()
function, for
example in help(autopy.bitmap.Bitmap)
. AutoPy's functions are documented with
descriptive docstrings, so this should show a nice overview.
>>> import autopy
>>> autopy.bitmap.capture_screen()
<Bitmap object at 0x12278>
This takes a screenshot of the main screen, copies it to a bitmap, displays its memory address, and then immediately destroys it. Let's do something more useful, like look at its pixel data:
>>> import autopy
>>> autopy.bitmap.capture_screen().get_color(1, 1)
15921906
AutoPy uses a coordinate system with its origin starting at the top-left, so
this should return the color of pixel at the top-left corner of the screen. The
number shown looks a bit unrecognizable, but we can format it with Python's
built-in hex
function:
>>> import autopy
>>> hex(autopy.bitmap.capture_screen().get_color(1, 1))
'0xF2F2F2'
Alternatively, we can use:
>>> import autopy
>>> autopy.color.hex_to_rgb(autopy.screen.get_color(1, 1))
(242, 242, 242)
which converts that hex value to a tuple of (r, g, b)
values. (Note that
autopy.screen.get_color()
, used here, is merely a more convenient and
efficient version of autopy.bitmap.capture_screen().get_color()
.)
To save the screen capture to a file, we can use:
>>> import autopy
>>> autopy.bitmap.capture_screen().save('screengrab.png')
The filetype is either parsed automatically from the filename, or given as an optional parameter. Currently only jpeg and png files are supported.
>>> import autopy
>>> autopy.bitmap.Bitmap.open('needle.png')
<Bitmap object at 0x1001d5378>
Aside from analyzing a bitmap's pixel data, the main use for loading a bitmap is finding it on the screen or inside another bitmap. For example, the following prints the coordinates of the first image found in a bitmap (scanned from left to right, top to bottom):
import autopy
def find_image_example():
needle = autopy.bitmap.Bitmap.open('needle.png')
haystack = autopy.bitmap.Bitmap.open('haystack.png')
pos = haystack.find_bitmap(needle)
if pos:
print("Found needle at: %s" % str(pos))
find_image_example()
It's also possible to do a bounded search by passing a tuple ((x, y), (width, height))
:
haystack.find_bitmap(needle, rect=((10, 10), (100, 100)))
Hope you enjoy using autopy! For a more in depth overview, see the API Reference.
If you are interested in this project, please consider contributing. Here are a few ways you can help:
This project is licensed under either the Apache-2.0 or MIT license, at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
FAQs
A simple, cross-platform GUI automation library for Python.
We found that autopy demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Ensure open-source compliance with Socket’s License Enforcement Beta. Set up your License Policy and secure your software!
Product
We're launching a new set of license analysis and compliance features for analyzing, managing, and complying with licenses across a range of supported languages and ecosystems.
Product
We're excited to introduce Socket Optimize, a powerful CLI command to secure open source dependencies with tested, optimized package overrides.