Performing automation testing with Airtest

Airtest in an automation test framework with main focus on mobile games however it can be used for all native Android apps as well. Airtest automation test framework uses image recognition technologies and provides cross-platform APIs that allows among other to install applications, perform simulated input, make assertions, etc.

Learn more from project homepage or Fork Airtest on GitHub,

Airtest integration with AirtestIDE

Airtest assistant window is placed on the bottom left side of AirtestIDE GUI layout by default (as shown on screenshot below).

../_images/airtest_intro.png

We can divide the buttons into three main groups - operations, assistant funtions and assertions:

Operations Auxiliary functions Assertions
touch text assert_exists
swipe keyevent assert_not_exists
wait server_call assert_equal
exist sleep assert_not_equal
snapshot    

These buttons can be easily filtered out using dropdown menu as follows:

../_images/3_filter_airtest_assist_btn.png

Recording Airtest script in AirtestIDE

The instructions above can be combined with the Python code, combining Python scripting with predefined set of Airtest instructions, we can easily create a powerful fully automated test script.

There are two ways how to record the Airtest script - manually create screenshots or automatically capture all click actions on the screen and generate the screenshots automatically.

Following demonstrates how to record the script manually:

  1. select action you want to perform in Airtest window
  2. to select the object of interest, draw a rectangle on the target object on the screen device, the code will be automatically generated to your script
  3. repeat the steps for all objects you need to check
../_images/1-airtest_manual_record.gif

Sometimes, creating the screenshots manually is not very effective way, that’s why AirtestIDE also provides automatic recording features. In each step, wherever you click, the IDE will automatically capture the screenshots generated according to the current click area.

Automatic script recording can be turned on by click camera icon in Airtest window. To stop the automatic script recording, press the camera icon again.

../_images/camera_icon.png

Following demonstrates how to record the script automatically:

../_images/2-airtest_auto_record.gif

Running Airtest script and inspecting the report

AirtestIDE also provides the features to run/replay the script and inspect the test report which is generated once the script execution is finished. To do so, there are 3 buttons available to manage run script, stop script and inspect the log.

../_images/4_run_stop_checklog.png

To run the script, perform following steps:

  1. click the start button (the most left button in the screenshot above)
  2. or press F5 key shortcut

To stop script, perform following steps:

  1. click the stop button (the middle button in the screenshot above)
  2. or press F10 key shortcut

To inspect the log, perform following steps:

  1. click the log button (the most right button in the screenshot above)
  2. or press Ctrl+L key combination

New .air script and .py script[new]

In Airtest, the original default script suffix is ​​``.air``. In fact, it is a folder that stores the .py files with the same name as .air and related images file. When you run the script, it actually use Python to call the .py file inside, because Airtest is essentially a Python third-party library. (For how to run the script, please click here .)

So in fact, in the ordinary .py file, you can also call the Airtest interface. We have added a convenient function to create py scripts in AirtestIDE and Airtest. The sample code of the reference can be clicked here, where the key initialization code is as follows:

from airtest.core.api import *
from airtest.cli.parser import cli_setup

if not cli_setup():
    auto_setup(__file__, logdir=True, devices=[
        "Android:///?cap_method=javacap&ori_method=adbori",
    ])

The above code means that when you use python xxx.py to run this file without any command line arguments, the auto_setup interface is automatically used to initialize the airtest-related parameters. This only needs to fill in the desired parameters when writing the py script to run the script directly with the python xx.py directive. The original ``airtest run xx.air –devices xx `` is also unaffected. As long as the script detects that the command line arguments are passed, the command line arguments are still used first to initialize the airtest.

At the same time, in AirtestIDE, if you want to create a new py file directly, please select .py Pure Python (For Advanced Users) when creating a new script, and then the python file settings window will pop up:

../_images/pycli.png

The configuration items inside are:

  • Save Path: File save path, required, select the path before you can continue to configure other options
  • Log Directory: The log save path when the script is running. The default is log/ directory under the directory where the .py file is located.
  • Devices: You can choose to automatically initialize the devices parameters using the currently connected mobile device, or you can fill in them later.
  • Project Root: The parent directory of the project, for future use of using statements to introduce other subfolders of the same parent directory

After configuring the required options, click the ok button, it will automatically create a template python file with the configuration content, then just write a script like a normal py file.

Report generation

A report on how to generate the .air script is described in detail in the documentation Running scripts-Generating a test report. If you need to generate a report in code in a pure .py script, you can also refer to what is described in the documentation.

In AirtestIDE, if you open the .py file and want to be able to directly use the IDE’s one-click report generation function after running the script, you need to manually specify the path of the log directory when running the script when you click the View Report button:

../_images/pycli-report.png

Image Editor Feature

AirtestIDE provides not only the ability to take the screenshots but also you can edit these captured screenshots using the Image Editor feature to adjust the image recognition parameters such as threshold, target position area and RGB channel to increase or adjust the image matching results.

To invoke the Image Editor dialog, double-click on the picture in script area. Click OK to accept your modification, or click Cancel to abandon it.

We provide a function to preview the matching rate of the screenshot on the current screen. Clicking the Snapshot Recognition button in the photo editor will capture the current screen of the mobile phone and perform a recognition. If the recognition is successful, the identified area of the screenshot will be marked to make the debugging easier.

../_images/5_image_editor.png

At the same time, a right click in the screenshot display area will pop-up a menu where you can choose to take another screenshot of the mobile device, specify the use of a certain type of identification algorithm and other operations.

Following values can be adjusted for each captured screenshot:

threshold (float) - it sets the image matching threshold, the valid values are from interval [0.0, 1.0], the default value is 0.6

target_pos (int) - set the position to operate on image matching result, the valid values for position flag are from interval 1-9

RGB (bool) - set whether to use RGB channel or not when calculating the confidence of image matching result, the default value is False, i.e. use the gray image to calculate the threshold