# 1.1 How to write a test script for your game In this tutorial, we will use a game based on the `Unity` engine, `The big fish is coming` as an example to show you how to use our `AirtestIDE` to quickly write your first game automation test script. It is not as hard as it might seem. Before we start, we strongly recommend that you read the [5 minutes to get started automation test](http://airtest.netease.com/tutorial/Tutorial.html) tutorial provided in the official website, which roughly shows the connection from the mobile phone to writing scripts, running scripts, and generating reports, as well as a detailed introduction to game scripting. **This article may take more than 10 minutes to read carefully (but it will save you a lot of detours!)**。 ## Start！ Suppose now that we have `AirtestIDE` [installed](http://airtest.netease.com/changelog.html) on the computer, and connect an Android phone to the computer via USB cable (please refer to the connection method [tutorial](http://airtest.netease.com/docs/docs_AirtestIDE-en_US/1_quick_start/2_device_connection.html)), and also open the game you want to test: `The big fish is coming`. When everything is ready, we will see the following on `AirtestIDE`: ![image](1_how_to_write_the_first_script_for_your_game/1_AirtestIDE_front.png) ## Image recognition based script First, we will show you how to use **image recognition** to write scripts, we provide an open source framework [Airtest](https://github.com/AirtestProject/Airtest), which is very intuitive, to find out our target elements by image recognition on the current game screen and operate on it. ### Airtest touch statement For example, our first test case is: Click the “go through” button on the screen to enter the level selection screen, then select “First pass”. This use case can be done using the `touch` statement. In `AirtestIDE` you can generate a statement by clicking the `touch` button in the Airtest helper window and then selecting the button: ![image](1_how_to_write_the_first_script_for_your_game/touch.gif) For this use case, we wrote two lines of code: ```touch(```![image](1_how_to_write_the_first_script_for_your_game/button1.png)```)``` ```touch(```![image](1_how_to_write_the_first_script_for_your_game/button2.png)```)``` Run the code, the effect is that the "pass through" button and the "first pass" button in the game are pressed successively. The principle of the code operation is shown in the figure below: ![image](1_how_to_write_the_first_script_for_your_game/airtest_touch.png) #### Advanced reading: touch statement in the API documentation The code of the `touch` statement is as simple and intuitive as it seems. It looks for the coordinates that match the content of the picture in the current game screen. If found, it will click on the coordinates. If no matching picture is found, an exception will be thrown. If you want to do more complicated operations, you can get more details in the API documentation of `Airtest`. For example, to learn more parameters and usage. Since `Airtest` is cross-platform, all the interfaces are defined in `airtest.core.api`. You could start in the section `airtest.core.api` to understand the [`touch` interface](http://airtest.readthedocs.io/en/latest/all_module/airtest.core.api.html#airtest.core.api.touch)。 In the interface documentation, the description of the `touch` statement is: clicks on the screen of the device. It is cross-platform: can be used for `Android`, `Windows` and `iOS`. The common parameters are: the coordinates of the click position: `v` (or a picture, as airtest will find the position coordinates of the picture on the screen) and the number of clicks: `times`. There are some other platform-specific parameters. Since we are using an Android phone to record the script in this example, we will continue to consult the platform-related chapters in the documentation. Find the [`touch` interface in the `airtest.core.android.android` module](http://airtest.readthedocs.io/en/latest/all_module/airtest.core.android.android.html#airtest.core.android.android.Android.touch). You can see the `touch` on the `android` platform interface accepts 2 parameters, one is `pos`, which is the coordinates of the actual click position, and the other is `duration`, which can control the length of time we click on the screen. The default is 0.01 second, which is a light touch, but with longer duration it can become a long press. This is another example of the `touch` statement. We replaced the image with coordinates and added a parameter `duration`, so this line of code long presses a coordinate position for one second: ``` touch((500, 600), duration=1) ``` > If you feel that it is a bit troublesome to consult the API documentation, you can directly hover the mouse over each button in the `Airtest Assist Window` of `AirtestIDE`, and you will directly see the parameter information corresponding to this statement. Of course, we also highly recommend reading [Airtest source code](https://github.com/AirtestProject/Airtest). #### More action statements In addition to `touch`, we also provide several other common operation statements, that you can use to achieve more complex operations: - `swipe`, to slide from one location to another - `wait`, to wait for an image to appear in the screen - `exists`，to determine whether there is an image in the screen - `text`，to call the input method and enter a paragraph of text - ... In addition to the operation statement, we also need to verify that the running result is correct during the process of writing the test script. `Airtest` provides a dedicated assertion statement for us to verify: - `assert_exists`，to assert that the image exists on the current screen - `assert_not_exists`， to assert that the image does not exist - ... ### A complete test case Now that you know the common statements, you can implement a use case like this with a small piece of code: - Open our game to be tested - Select game mode - Swipe the screen to the far right and select the level you want to play "20th" - Verify that you have successfully entered the level 20 in the preparation screen Sample code: ![image](1_how_to_write_the_first_script_for_your_game/script.png) In this code, we show the vast majority of the `Airtest` statements, which make it easy to write logically complex automated test scripts. ### Further reading: improving the success rate of running code `Airtest` is not complicated to get started. It can be said that it is easy to get started with a little `Python` script base. However, as more and more script code is produced, we will find a very serious problem: our script run results sometimes seem to be out of our control. For example, we want to first determine whether there is an icon on the current screen. Only when it exists, the next step should be performed, otherwise, if there is no clear image on the screen, the step should not be performed. However, `Airtest` still thinks that there is an existing icon. When you open the report, you find that the content of another area on the screen is judged to be the icon we are looking for. This problem is caused by the operation principle of `Airtest`, since it uses image recognition technology to find the corresponding image in the current game screen. However, the image recognition system can still not achieve the accuracy of human eye. It can only find, as far as possible, an image that best meets the expected one. This often leads to images that we do not think exist, but `Airtest` thinks they do, or some icon we think we can find on the screen, `Airtest` thinks it does not exist. Therefore, after writing the script, we can make it run a few times and then improve the part with low success rate. This has a few notable improvements: - When taking the screenshoots **try to ensure that the captured images are highly recognizable and independent**. For example, when capturing a button image, try not to bring too much noise into the background in order to avoid problems of recognition when the background changes. - In `AirtestIDE`, you can change the threshold of image recognition by double-clicking the image and modifying the value of `threshold`. The higher the threshold, the higher requirement of accuracy for image matching. You can refer to [document address](http:/ /airtest.netease.com/docs/docs_AirtestIDE-en_US/2_airtest_script/3_image_editor.html#id3) - When recognizing an image, `Airtest` will first convert the image to **grayscale** and then recognize it. The result may not be as accurate as expected when judging certain gray icons that become colored when hovering over them. However, we can force the use of color images by double-clicking on the image and checking the `RGB` option in the settings. - `Airtest` will try to adapt to different resolutions of the phone as much as possible, but in some games there may be custom resolution adaptation rules, which can be customized according to the resolution of the game, the strategy to follow can be found [here](https://github.com/AirtestProject/Airtest/issues/4) - If there is a large number of repetitive, very similar icons stacked together, it is possible that the recognition is not good. In our eyes, perhaps the text on each icon is different, but in the 'eyes' of `Airtest` they are too similar. We can try to modify the screenshot and modify it to a more recognizable image with some other background styles. - We provide a convenient automatic recording function, which can directly convert all current operations into code step by step. Nevertheless, in some cases, the automatically intercepted images are not ideal, and the screenshots need to be manually adjusted. ### New use cases and challenges `Airtest` satisfies our need to write game automation test scripts quickly and easily. However, its core technology is based on image recognition, and in addition to the lower success rate proposed in the previous chapter, we find it somewhat inadequate to test some of the more complex requirements. For example, for this use case: - Open the mall - Upgrade "invincible bubble" item, which will deduct a certain amount of gold coins - Verify that the gold coins in the backpack were successfully deducted ![image](1_how_to_write_the_first_script_for_your_game/item.png) As shown in the figure, although we can easily identify the gold coin control by taking a screenshot, the values inside are difficult to obtain. Similarly, we can write the `touch` statement to click the upgrade button (although some tricks are needed to distinguish which one is the "upgrade" button for the "invincible bubble"), it is difficult to verify whether the backpack gold coin is correctly deducted by the "400" gold coin marked in the image. The good news is that we have another solution to better solve this kind of problem, which is a framework based on control search [Poco](https://github.com/AirtestProject/Poco/)。 ## Poco based on control recognition Let's put aside the use cases mentioned above and take a brief look at the difference between `Poco` and `Airtest`. ![image](1_how_to_write_the_first_script_for_your_game/ide_poco.png) This is a screenshot of the Poco plugin in `AirtestIDE`. We use Poco to accurately locate the position of the elements on the current game screen in the actual UI tree. For example we can get the name of the "Go" button, coordinates, etc. We can also get this button by writing a certain filter statement, and then click and perform other operations on it. Suppose we now want to click on the "Go" button in the screen, using `Airtest`, we will write a statement like this: ```touch(```![image](1_how_to_write_the_first_script_for_your_game/button3.png)```)``` In `Poco`, the statements written are quite different, but simple and elegant: ```python poco("Go").click() ``` This is a simplified diagram that compares the operating principles of `Airtest` and `Poco`: ![image](1_how_to_write_the_first_script_for_your_game/poco_touch.png) As you can see, the biggest difference between `Poco` is that there is a `Poco-SDK` module. We need to **embed `Poco-SDK` into the game under test** to get the UI tree smoothly and for the next parsing and processing operations. ### How to access Poco In this chapter, we will show you how to get your game connected to `Poco` and enjoy its power. `Poco` The following modes are currently supported (not just the game engine) - Cocos2dx-js，Cocos2dx-lua -> [Access document](https://poco.readthedocs.io/en/latest/source/doc/integration.html#cocos2dx-lua) - Unity3D -> [Access document](https://poco.readthedocs.io/en/latest/source/doc/integration.html#unity3d) - Android Native APP -> does not require access - iOS -> [Help document](http://airtest.netease.com/docs/docs_AirtestIDE-en_US/1_quick_start/5_ios_connection.html) - Netease self-developed engine - Other engines -> available [self-access](https://poco.readthedocs.io/en/latest/source/doc/implementation_guide.html) - Stay tuned for more platforms and engines **In short, for game testing, if you want to use `Poco`, you need to connect `Poco-SDK` to the game you want to test according to the access document. If you have questions about the document, you can invite your project team to help with reading and accessing.** ### Start the handwriting Poco script Ok, let's assume that with the help of the project team, we have already connected the `The big fish is coming` game to Poco's Unity3D version of the SDK. The next scripting is very simple: ![image](http://airtest.netease.com/tutorial/gif/poco_auto_record.gif) The basic usage process should be: - In the Poco Assist window **select the corresponding mode**, for example here we select Unity - The script editing window will prompt if **to insert the corresponding initialization statement**, click yes - The contents of the initialization statement of poco (two lines) will be inserted in the blank script. - Wait a few seconds, the **UI tree structure** of the current game screen will be displayed in the Poco Assist window - Click the node on the UI tree to display the node content in the lower log window; double-click the node on the UI tree to automatically insert the code corresponding to the node into the edit box; click the pause button to freeze the current UI tree and view more detailed information easily. - After getting the node we need, we can manipulate it, with instructions like `.click()` and `.swipe()` Next, going back to the use case we mentioned before that was difficult to do with `Airtest`: - Open the mall - Upgrade "invincible bubble" item, which will deduct a certain amount of gold coins - Verify that the gold coins in the backpack were successfully deducted We can write a sample script in poco like this: ```python # Please be sure to start the app under test before initializing poco start_app("fish.package") # Initialize Poco, very important! from poco.drivers.unity3d import UnityPoco poco = UnityPoco() money = int(poco("Money").child('Text').get_text()) # .get_text() Can help us get its value cost = int(poco("2024").child("Upgrade").child("NonFullPanel").child("Cost").child("Number").get_text()) # Click the upgrade button for the item under test, please note the end ".click()" poco("2024").child("Upgrade").child("NonFullPanel").child("UpgradeBtn").click() current_money = int(poco("Money").child('Text').get_text()) assert_equal(money-cost, current_money, "The gold coin has to be deducted successfully") ``` For beginners, there are a few things that can go wrong: - **Poco initialization should be done after the game is started**, because poco needs to communicate with the poco-sdk in the game, you need to wait for the game to initialize poco-sdk before you can initialize poco - Please be sure to choose **the correct Poco mode**, first figure out what engine your game is made on, and then perform the corresponding SDK access. After the access is successful, you can start using it. - The UI object in poco is just a **agent**, which needs to be further manipulated (eg `.click()`, or `.exists()`) to achieve the desired effect, for example: ```python # For example, this is a bad way to determine that the if statement with the Go button in the screen is spelled incorrectly # because Poco("Go") is a UIObjectProxy object if Poco("Go"): Poco("Go").click() # The correct way to write it is if Poco("Go").exists(): Poco("Go").click() ``` ### Advanced reading: how to improve Poco scripts The code before uses the Poco plugin of `AirtestIDE` to automatically generate the corresponding node selection of the code. This is done by simply double-clicking on the node in the UI tree. Additionally, we provide an automatic recording option. This function can directly record the current mouse operation as a statement. This is very easy for beginners, but, it should be noted that sometimes the automatically generated code may not meet our needs. For example, in the item selection screen of `The big fish is coming`, there are multiple nodes with price tags, as shown in the figure: ![image](1_how_to_write_the_first_script_for_your_game/poco_coin.png) If you use `AirtestIDE` to select a node directly, the node selection code generated after double-clicking may be like this: ```python # Gold coin tags for magnetic bubble items poco(text="600") ``` At first glance, this line of automatically generated code is concise and clear, and can also fulfill our needs for "get magnetic bubble item corresponding to the value of gold the gold", but this code directly uses `text="600"` as the selection criteria. If the value of this node changes, or if there are other items that cost 600 gold coins, the result of this line of code may not meet our expectations, because it can not accurately locate the specific node we want. We need to have some understanding of the `Poco` selector, to modify it by writing your own certain conditions. We suggest: - To read our [Introduction to Use Cases](https://poco.readthedocs.io/en/latest/source/README.html#tutorials-and-examples), that provides demo games and applications and access to ` Poco - SDK `, in order to experience the fastest way to get started with ` Poco ` - Read the `Poco` documentation carefully and learn how to use [Poco to select UI objects](https://poco.readthedocs.io/en/latest/source/README.html#working-with-poco-objects), for mastering the selector which allow us to write a UI selection code that better suits our needs. - The documentation of [Operation UI Objects](https://poco.readthedocs.io/en/latest/source/README.html#object-proxy-related-operation) describes some of the most common operations, which can be good to get started - After getting started, you can read [Poco instance API](https://poco.readthedocs.io/en/latest/source/poco.pocofw.html) and in more depth [Poco proxy object API](https://poco.readthedocs.io/en/latest/source/poco.proxy.html), as well as other sections of the API documentation ## Summary This article introduces some basic guidelines for writing game test scripts using `Airtest` and `Poco`, and answers to some of the most frequently asked questions. For more advanced content, please refer to the official documentation manual. It's worth noting that **Poco and Airtest are both Python libraries. In essence, the scripts we write are all ordinary Python scripts, so they can be combined to other python libraries to achieve our test needs more accurately and to write more complex and powerful scripts**.