Note

This document is a supplement to Airtest framework and AirtestIDE editor. Please refer to Airtest project documentationt to get more information on API details during the process of writing scripts.

Airtest

Introduction to Airtest

Airtest is a cross-platform UI automation testing framework, based on image recognition principles, suitable for games and apps.

Visit Airtest source code on Github to get more information. You are welcome to help improve the project and submit PR. You can also submit bug or advice via the issues page.

The project documentation can be found here.

Quick start

For installation, usage instructions or simple examples, please check the Quick start section in Airtest document.

Airtest API

The airtest.core.api module of Airtest provides a series of cross-platform APIs, including operations such as tapping screen touch(), dragging swipe() and typing texts text(). For more information on API call arguments and return values, please check the Cross-platform API document.

All interface arguments in the Cross-platform API document are common cross-platform arguments. For details on specific platforms, you can check API document for more information on arguments.

Take touch interface as an example, suppose you need to click on an Android platform, you can write this:

# import all APIs from airtest.core.api, please make sure to keep this line
from airtest.core.api import *
# The touch interface for Android platform supports an additional parameter ‘duration’ to control the amount of screen clicks
# check airtest.core.android.android for more information on Android touch method
touch((600, 500), duration=1)

In an .air script, if you want to call a public function encapsulated in another .air script, you can do this:

from airtest.core.api import using
using("common.air")
from common import common_function
common_function()

For full information about other available methods and usage, please refer to Complete documentation for Airtest framework .

Writing Airtest scripts in AirtestIDE

In Airtest, you need to write loads of screenshot statements, such as:

touch(Template("image_of_a_button.png"))

In the statement above, Template(“xx.png”) is an image object. If you manually write the code and take each screenshot, the workload would be large. Therefore we recommend using AirtestIDE to develop screenshot statements, which makes the process more effective.

Download AirtestIDE via the official website and unzip it. The included assistant window is handy for writing Airtest script statements.

../_images/airtest_assistant.png

Hover over the button in Airtest assistant window to see relevant argument information of the particular statement. Click the button to directly generate the corresponding statements or take a screenshot of the screen of the currently connected device. Once screenshot is taken, a statement with the screenshot will be generated in the script window.

Advanced: Improve script success rate

It is not difficult to get started with Airtest. All you need is some basic knowledge of Python scripts. However, as more script code produced, we will find that sometimes the result of our scripts may be out of our control. For example, we want to determine if an icon exists on the current screen; if it does, we proceed to the next step. Sometimes, such icon might not exist, yet Airtest find some other content on the screen to be that icon. We can go through the report to find out if this was the case.

Another common situation would be: we frame a few words on the screen and want them recognized, but the results come up mixed up. Airtest often assumes that the content we select does not exist, but sometimes it does.

This problem is caused by how Airtest works: We use image recognition technology to find corresponding images in the game. However, image recognition is not always as accurate as human eyes, it can only find the result that fits the expectation best. This often leads to Airtest determining something that does not exist, as existing or something that we can easily see on the screen, as not exiting.

In one sentence: image recognition is not a panacea!!! ** There’s something called **success rate. Assume there are 10 images in a script, it has a 95% success rate to recognize each image, but the odds that it recognizes all 10 images is only 60%. What’s more, there are many images that, for a variety of reasons, have a far lower success rate than 95%, making it harder for the script to run 100% correctly. Therefore, after writing scripts, we can run them several times, and then make improvements to the parts with low success rate. These are several noteworthy improvements:

While taking a screenshot, make sure it is clear and is independent. For example, if you are taking a screenshot of a button, avoid a complex background so it won’t be difficult to recognize once the background is changed The image recognition algorithm is more suitable for identifying buttons (with frames) or icons. It has a much lower success rate at recognizing text. Please adjust the content in the screenshot to achieve better results and try to avoid interception of images, which is harder to recognize.

Airtest strives to adapt to as many devices with various resolution as possible, but, some games might have custom resolution adaptation rules, for them you can define your own settings. See tutorials here If there are lots of similar and repeated icons stacked together, you may get a poor recognition result. For our eyes, these icons may seem different; but Airtest might find them similar. We can try to modify the screenshots and use different background styles to create images that seem as different by Airtest.

Although we provide a convenient automatic recording capability to directly translate all current operations into code step by step, in some cases the automatically captured images are not ideal and need to be adjusted manually.

Settings for image recognition

During Airtest’s image recognition, there are some settings that can be adjusted to improve the success rate of the scripts: In AirtestIDE, double-click images to modify the threshold value of image recognition. The higher the value, the higher the requirement of image matching accuracy. While recognizing images, Airtest will first convert them into grayscale images for recognition. When there are two buttons with same shape but different colors, Airtest will determine they are identical. However, you can force color image recognition by double-clicking the image and checking the RGB option in the Settings. In addition to modifying threshold and rgb value, by double-clicking the image you can also use target_pos to set the position Airtest will click on once it’s recognized. For example, once we recognize an icon, we can make Airtest to click the upper left /bottom right corner of the icon. For detailed configuration methods, see Airtest scripts settings .