Getting started with AirtestIDE

Introduction

AirtestIDE is a cross-platform UI automation test editor for mobile games and apps. The main features are as follows:

• supports automated script recording, one-click playback, and report inspection - automation testing has never been easier
• supports image recognition based on Airtest framework for Android and Windows games and apps
• supports UI controls search based on Poco framework for Unity3d, Cocos2d and Android apps
• runs on Windows and macOS
• Netease has been successfully used AirtestIDE to dozens of projects internally, using mobile cloud lab for large-scale automated testing

Visit official websites to learn more about the features and watch the tutorial videos.

Through this tutorial you will learn how to get started with the automated testing, and how to write scripts to play the games automatically. Trust us, it's gonna be a lot of fun ~

Installation

Currently, AirtestIDE provides two versions of Windows and macOS clients. Just download the installer from official websites and unzip it.

Connect the devices

AirtestIDE currently supports testing of Android, Windows and iOS applications.

Connecting the Android device

Connect the Android device with your PC and allow USB debugging using adb. ADB is an official Android debugging tool developed by Google. AirtestIDE relies on ADB to communicate with the Android devices.

Next, open the AirtestIDE and follow the steps bellow to connect the device:

1. turn on the usb debugging option in Settings - Developers Options - USB debugging, for more details refer to official Android documentation
2. in AirtestIDE devices panel, click Refresh ADB button to see the connected phones.
3. if there is no device connected, try to Restart ADB. If it still fails, refer to FAQ
4. click next to the corresponding device on Connect button to connect the device with AirtestIDE

When the device is successfully connected, you can see the device screen mirrored in AirtestIDE. Now you can start to perform the real-time operations.

If the device connection fails, please provide the report including the information about the phone model and AirtestIDE error log to Github Issues. Our developers will try to solve the problem as soon as possible. Due to the fragmentation of Android phones, we are very grateful for any feedback which help us to make this project better.

Connecting the Windows window

When we test Windows desktop programs, we usually want to test a window. AirtestIDE can embed the selected window for script recording and debugging.

• in AirtestIDE devices panel, click Select Window button
• move the mouse to the desired program window, a green border box will be displayed to frame the corresponding window
• use left click to embed the corresponding window into the AirtestIDE

If the above can't find the window of tested program, follow the steps described in our Solving troubles with embedding the windows.

Connecting iOS device

Now we have supported iOS connection, please see the documentation for more information.

Record automation scripts

Now, we can start to record the automated test scripts.

Automatic script recording

Let's learn how to record the script automatically:

Based on image recognition

In the Airtest assistant window, click Record button , and since now as you operate the phone in the devices panel area, the code is automatically generated into the script window.

Immediately verify and click Run button to run your first automation script!

If the automatic recording feature is not accurate enough for you, in the Airtest assistant window you can click on touch button and then select the desired object on the device screen, or you can automatically generate the touch statement.

Similarly, you can automatically capture swipe action - click swipe button, select the desired object as the starting point of the swipe action on the device screen, then click to end swipe position. This will automatically generate a swipe statement.

Following is the list of other APIs for automatic script recording:

• text: text input
• kevent: key input, including HOME, BACK, MENU, etc.
• sleep: wait for certain time period
• snapshot: take a screenshot

Based on UI-hierarchy tree

If the image recognition is not accurate enough for you, you can create the automation tests based on UI-hierarchy tree controls.

AirtestIDE currently supports Unity3d and Cocos2d game engines and Android source apps. As the game engines use the graphics interface such as OpenGL for direct rendering without using Android UI system, we need to communicate with the runtime of the game to obtain the entire UI structure. For Unity3d and Cocos2d-js, we provide a very convenient SDK access method. More details can be found here.

For other game engines and UI systems we provide SDK self-extension capability.

In fact, inside NetEase we are using this method to support multiple self-developed engines such as Messiah/NeoX/Fantasy.

• Unity3d and Cocos2d games: SDK integration docs
• Android native apps: start straight!
• Other game engines and UI systems: extend the SDK

After the UI controls access is completed, we can begin. Start the game on your device and using Poco assistant window switch the mode to corresponding engine. You should see the entire UI structure now.

Click the record button, and all your actions will automatically generate a Poco statements.

In the same way, you can also view the UI controls more precisely through the UI tree structure. To automatically generate the Poco statements, double-click to UI element. Better attribute selection usually enhances the robustness and readability of the entire automated script. Check tutorials and examples .

Remember to run the test after the recording the script.

As the UI structure of native Android apps is accessible via Accessibility , we can start using it directly. Switch the Poco assistant window mode to Android and you can see the entire UI tree structure.

In fact, the above two methods are based on two frameworks:

• Image recognition based Airtest framework for all Android and Windows games.
• UI controls search based Poco framework for Unity3d, Cocos2d and Android.

Both of these frameworks are developed by our team. According to the experience of the actual project use, we are sure that by the combination of these two frameworks you can achieve the best results.

Use Python syntax

The code recorded and run throughout the AirtestIDE is based on the Python2.7 language. Python syntax is simple and powerful, and there are already many existing third-party libraries and tools.

For newbies: it is pretty straightforward to learn Python basics to be able to write logic statements needed in automated scripts.

touch("OpenCardPackage.png")
if exists("RewardPanel.png"):
for i in range(5):
    Poco("Reward-%s" % i).click()

For experienced: you can use various third-party libraries in the AirtestIDE to make your automation scripts more powerful, just add PYTHONPATH.

In addition to the statements provided by assistant windows, more about APIs is available in Airtest and Poco docs.

Until now, we have talked about variety of functions and logical control statements to operate the mobile device. However, there is one very important step in automated testing: the way how results are verified. So, let's have a look how to declare the assertions.

Verify the UI interface

The recording method is similar as described earlier:

• assert_exists: assertion that the given object exists
• assert_not_exists: assertion that the given object does not exists

Verification

For attribute values obtained from Poco there are following assertions available:

• assert_equal: assertion that the attribute is equal to given value
• assert_not_equal: assertion that the attribute is not equal to given value

For example:

# ... after simulating the input, get the score 20
value = Poco("ScoreButton").attr("num")
assert_equal(value, 20, "get score 20")

View the test report

After the script finishes the running, click on View Report button, or use key shortcut (CTRL+L) to open the report page with the results. The report will show the details of each step including the screenshot of the actual executed operation and the test result.

Command line interface

Until now, we have learned how to automate testing with AirtestIDE. Next step is to use the command line interface to combine automated testing with the continuous integration. What is continuous integration?

When the AirtestIDE runs the script, the run command output is printed in the Log Viewer window.

You can use that command to start a test script on the command line without opening the IDE itself, for example:

"D:\AirtestIDE\AirtestIDE" runner "D:\AirtestIDE_2018-01-24_83\untitled.air"  --device Android://127.0.0.1:5037/F8UDU16409004135 --log "C:\Users\gzliuxin\AppData\Local\Temp\AirtestIDE\scripts\cdfc40e8c297b6ad88e09de64d8bafa3"

With the AirtestIDE you can easily record the test scripts and save them in .air project directory. Please note, that .air project directory should not contain too much content, it is recommended to organize and name the test scripts with proper naming convention into proper directory structure.

You can also run the test scripts across multiple platforms and different devices. Refer to Running .air from cli for more information. Airtest project provides cross-platform APIs and commands, so you can easily use the same set of test scripts to run on multiple platforms, e.g on Android and Windows.

Inside NetEase, our games usually have hundreds of test scripts that cover common game play tests. Each week, these hundreds of test scripts are run on more than 200 devices for compatibility testing. Have a look on our Demo

Advanced Features

Editor Settings

Third-party Python libraries in AirtestIDE

Mastering Scripts Recording - Advanced Features

Building a cloud testing platform

Interation with Google Firebase