Advanced Usage

This guide covers some of the more advanced features, patterns and configuration options of AltTester Unity SDK.

AltTester input

AtTester Unity SDK has an Input class which overrides the Input class implemented by Unity. This way AltTester intercepts the input actions to be performed in the instrumented app and simulates them through this class. In case you are using assembly definitions inside your project, you will have to reference the AltTesterUnitySDK.asmdef in all your .asmdef files which use input actions.

AltTester input vs. regular input

AltTester’s custom input is active, by default, in any instrumented build. This means that certain input related actions (the ones that are part of Unity’s Input class) will be inactive for regular input (the device’s input). Because of this, pressing a key from the keyboard for example will not have any effect on the app. However, the simulated input from the tests, like the PressKey command, will be able to manipulate the object within the scene. While the AltTester input is active, the icon from the right bottom corner is green. You can change this behaviour by clicking on the AltTester’s icon and unchecking the box with the AltTester Input message. Now the icon will turn darker, signaling that the regular input is active. In this state, you can interfere with the object from the app using the keyboard or other input. Keep in mind that, input actions from the AltTester Desktop won’t have any effect while regular input is active. At the same time, if you want to run some automated tests, the AltTester input will be activated automatically for you.

Build apps from the command line

To build your Unity application from command line you need a static method in your project that handles the build logic. To instrument your Unity application with AltTester Unity SDK, your build method must define ALTTESTER scripting symbol and must insert AltTester Prefab in the first scene of the app.

Depending on your project’s setup, there are two ways in which apps can be built from the command line:

Note

AltTester Unity SDK does not work by default in release mode. If you instrument your app in release mode, AltTester Prefab self removes from the scenes and the socket server does not start. Best case practice is to customize your build script to insert AltTester Prefab only in Debug mode.

If you do want to use AltTester Unity SDK in release mode see Using AltTester Unity SDK in Release mode section.

1. If you already have a custom build method for your app

If you already have a custom build method for your app, you can add the following lines to your build method. Also, the BuildPlayerOptions should check for BuildOptions.Development and BuildOptions.IncludeTestAssemblies.

var buildTargetGroup = BuildTargetGroup.Android;
AltBuilder.AddAltTesterInScriptingDefineSymbolsGroup(buildTargetGroup);
if (buildTargetGroup == UnityEditor.BuildTargetGroup.Standalone) {
    AltBuilder.CreateJsonFileForInputMappingOfAxis();
}
var instrumentationSettings = new AltInstrumentationSettings();
AltBuilder.InsertAltInScene(FirstSceneOfTheApp, instrumentationSettings);

Note

Change buildTargetGroup above to the target group for which you are building.

2. If you create a new custom build method for your app

The following example script can be used. It sets all the project settings needed and uses the same two important lines from point 1 above.

This example method is configured for the Android platform, so make sure to update it based on your target platform.

static void BuildFromCommandLine () {
    try {
        BuildPlayerOptions buildPlayerOptions = new BuildPlayerOptions();
        buildPlayerOptions.scenes = new string[] {
            "Assets/AltTester/Examples/Scenes/Scene 1 AltDriverTestScene.unity",
            "Assets/AltTester/Examples/Scenes/Scene 2 Draggable Panel.unity",
            "Assets/AltTester/Examples/Scenes/Scene 3 Drag And Drop.unity",
            "Assets/AltTester/Examples/Scenes/Scene 4 No Cameras.unity",
            "Assets/AltTester/Examples/Scenes/Scene 5 Keyboard Input.unity",
            "Assets/AltTester/Examples/Scenes/Scene 7 Drag And Drop NIS.unity",
            "Assets/AltTester/Examples/Scenes/Scene 7 New Input System Actions.unity",
            "Assets/AltTester/Examples/Scenes/Scene 8 Draggable Panel NIP.unity",
            "Assets/AltTester/Examples/Scenes/Scene 9 NIS.unity",
            "Assets/AltTester/Examples/Scenes/Scene 10 Sample NIS.unity,
            "Assets/AltTester/Examples/Scenes/Scene 11 ScrollView Scene.unity",
            "Assets/AltTester/Examples/Scenes/Scene6.unity"
        };

        buildPlayerOptions.locationPathName = "sampleGame.apk";
        buildPlayerOptions.target = BuildTarget.Android;
        buildPlayerOptions.options = BuildOptions.Development | BuildOptions.IncludeTestAssemblies | BuildOptions.AutoRunPlayer;

        // Setup for AltTester
        var buildTargetGroup = BuildTargetGroup.Android;
        AltBuilder.AddAltTesterInScriptingDefineSymbolsGroup (buildTargetGroup);
        if (buildTargetGroup == UnityEditor.BuildTargetGroup.Standalone)
            AltBuilder.CreateJsonFileForInputMappingOfAxis();
        var instrumentationSettings = new AltInstrumentationSettings();
        AltBuilder.InsertAltInScene (buildPlayerOptions.scenes[0], instrumentationSettings);

        var results = BuildPipeline.BuildPlayer (buildPlayerOptions);
        AltBuilder.RemoveAltTesterFromScriptingDefineSymbols (BuildTargetGroup.Android);

    } catch (Exception exception) {
        Debug.LogException (exception);
    }
}

The following command is used to call the build method:

<UnityPath>/Unity -projectPath $CI_PROJECT_DIR -executeMethod BuilderClass.BuildFromCommandLine -logFile logFile.log -quit

You can find more information about the build command and arguments here.

Note

After building from the command line you can run the tests by using the commands from the next section.

How to make a production build

There is no need to remove the AltTester package entirely from the project, only the ALTTESTER Scripting Define Symbol should be deleted from the Player Settings. Also, make sure that the Keep ALTTESTER symbol defined checkbox is unchecked. After that, you can build your app normally as you would do in Unity.

Run tests from the command line

In order to run tests from the command line you can use the following example commands:

Available AltTester SDK command line arguments:

-testsClass - runs tests from given class/classes

Example command running tests from a single test class:

<UnityPath>/Unity -projectPath $PROJECT_DIR -executeMethod AltTester.AltTesterUnitySDK.Editor.AltTestRunner.RunTestFromCommandLine -testsClass MyTestClass -logFile logFile.log -batchmode -quit

Example command running tests from two test classes:

<UnityPath>/Unity -projectPath $PROJECT_DIR -executeMethod AltTester.AltTesterUnitySDK.Editor.AltTestRunner.RunTestFromCommandLine -testsClass MyTestClass1 MyTestClass2 -logFile logFile.log -batchmode -quit

-tests - runs given test/tests

Example command running a single test:

<UnityPath>/Unity -projectPath $PROJECT_DIR -executeMethod AltTester.AltTesterUnitySDK.Editor.AltTestRunner.RunTestFromCommandLine -tests MyTestClass.MyTestName -logFile logFile.log -batchmode -quit

Example command running two tests:

<UnityPath>/Unity -projectPath $PROJECT_DIR -executeMethod AltTester.AltTesterUnitySDK.Editor.AltTestRunner.RunTestFromCommandLine -tests MyTestClass1.MyTestName1 MyTestClass2.MyTestName2 -logFile logFile.log -batchmode -quit

-testsAssembly - runs tests from given assembly/assemblies

Example command running all tests from given assembly:

<UnityPath>/Unity -projectPath $PROJECT_DIR -executeMethod AltTester.AltTesterUnitySDK.Editor.AltTestRunner.RunTestFromCommandLine -testsAssembly MyAssembly -logFile logFile.log -batchmode -quit

Example command running tests from two assemblies:

<UnityPath>/Unity -projectPath $PROJECT_DIR -executeMethod AltTester.AltTesterUnitySDK.Editor.AltTestRunner.RunTestFromCommandLine -testsAssembly MyAssembly1 MyAssembly2 -logFile logFile.log -batchmode -quit

-reportPath - the xml test report will be generated here

<UnityPath>/Unity -projectPath $PROJECT_DIR -executeMethod AltTester.AltTesterUnitySDK.Editor.AltTestRunner.RunTestFromCommandLine -tests MyFirstTest.TestStartGame -reportPath $PROJECT_DIR/testReport.xml -logFile logFile.log -batchmode -quit

Run tests on a Continuous Integration Server

  1. Instrument your app build with AltTester Unity SDK from Unity or by building from the command line.

  2. Start the app build on a device.

  3. Run your tests - see commands in the “Run tests from the command line” section.

An example CI configuration file can be viewed in the GitLab repository.

What is reverse port forwarding and when to use it

Reverse port forwarding, is the behind-the-scenes process of intercepting data traffic and redirecting it from a device’s IP and/or port to the computer’s IP and/or port.

When you run your app instrumented with AltTester Unity SDK on a device, you need to tell your build how to connect to the AltServer.

Reverse port forwarding can be set up either through the command line or in the test code by using the methods available in the AltTester SDK classes.

The following are some cases when reverse port forwarded is needed:

  1. Connect to the app running on a USB connected device

  2. Connect to multiple devices running the app

How to setup reverse port forwarding

In case of Android

Reverse port forwarding can be set up in two ways:

  • through the command line using ADB

  • in the test code by using the methods available in the AltTester SDK classes

All methods listed above require that you have ADB installed.

For further information including how to install ADB, check this article.

In case of iOS

Unfortunately, IProxy does not have a way of setting up reverse port forwarding. As a workaround, to connect the device via USB you should follow the steps below:

  • set the iOS device as a Personal Hotspot

  • enable Hotspot via USB on the machine running the AltServer

    • for this to work, you need to make sure that you have the Disable unless needed toggle disabled in the Network settings for the USB connection

    ../_images/connect-via-hotspot-USB_iOS.png
    • the hotspot network and the first device to connect to it are most of the time on 172.20.10.2 so you could set this IP for builds for iOS

  • add the IP of the machine running the AltServer to the first input field in the green popup from the instrumented app/game

In the routing table, the personal hotspot network would be secondary, therefore the traffic shouldn’t be redirected through the hotspot:

../_images/workaround_iOS.png
  • Reverse port forwarding using the following command:

    adb [-s UDID] reverse tcp:device_port tcp:local_port.
    

Note

The default port on which the AltTester Unity SDK is running is 13000. The port can be changed from the green popup. Make sure to press Restart after modifying its value.

Connect AltTester Unity SDK running inside the app to AltServer

There are multiple scenarios:

Establish connection when the instrumented app and the test code are running on the same machine

reverse port forwarding case 1

  1. Start AltServer on your machine by opening AltTester Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented app on the same machine. It will automatically connect to AltServer. The server identifies the app using the appName.

  3. Connect your tests to the server using the line below in your OneTimeSetup(). Start your tests on the machine used before. Make sure that AltServer, the instrumented app and your tests are using the same port. Data transmission happens on localhost.

altDriver = new AltDriver (host = "127.0.0.1", port = 13000, appName = "MyApp");

In this case reverse port forwarding is not needed as both the app and tests are using localhost:13000.

Establish connection when the app is running on a device connected via USB

reverse port forwarding case 2

  1. Start AltServer on your machine by opening AltTester Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented app on your device.

  3. Use Reverse Port Forwarding to direct the data traffic from the device’s port to the computer’s port. After this, your app will be connected to AltServer. The server identifies the app using the appName.

  4. Connect your tests to AltServer using the line below in your OneTimeSetup(). Start your tests on the machine used before. Make sure that AltServer, the instrumented app and your tests are using the same port. Data transmission happens on localhost.

altDriver = new AltDriver (host = "127.0.0.1", port = 13000, appName = "MyApp");

Establish connection via IP when the app is running on a device

reverse port forwarding case 3

  1. Start AltServer on your machine by opening AltTester Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented app on your device.

  3. Change the host from the green popup in your instrumented build to the machine’s IP AltServer is running on. The server identifies the app using the appName.

  4. Connect your tests to AltServer using the line below in your OneTimeSetup(). Start your tests on the machine used before. Make sure that AltServer, the instrumented app and your tests are using the same port. Data transmission between tests and server happens on localhost; transmission between device and server happens on the host’s IP.

altDriver = new AltDriver (host = "127.0.0.1", port = 13000, appName = "MyApp");

In this case Reverse Port Forwarding is not needed. Despite that, it is recommended to use reverse port forwarding since IP addresses could change and would need to be updated more frequently.

Establish connection when different instances of the same app are running on multiple devices

Connection through IP

reverse port forwarding case 4

  1. Start AltServer on your machine by opening AltTester Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented app on your devices. Make sure they have different names. In case you want to change the name, you can do that in the green popup. There is no need to make another instrumented build.

  3. Change the hosts from the green popups in your instrumented builds to the machine’s IP AltServer is running on. The server identifies the apps using the appName.

  4. Connect your tests to AltServer using the line below in your OneTimeSetup(). You will need to create 2 AltDrivers as you have 2 devices. AltDriver1 will communicate with device1 and AltDriver2 with device2. Start your tests on the machine used before. Make sure that AltServer, the instrumented app and your tests are using the same port. Data transmission between tests and server happens on localhost; transmission between devices and server happens on the host’s IP.

altDriver1 = new AltDriver (host = "127.0.0.1", port = 13000, appName = "MyApp1");
altDriver2 = new AltDriver (host = "127.0.0.1", port = 13000, appName = "MyApp2");

The same happens with n devices. Repeat the steps n times.

Connection through USB

Use reverse port forwarding for both devices. Data transmission happens exclusively on localhost. Ex. with 2 Android devices:

adb -s deviceId1 reverse tcp:13000 tcp:1300
adb -s deviceId2 reverse tcp:13000 tcp:1300

Establish connection when multiple instances of the same application are running on the same device

Connection through IP

reverse port forwarding case 5

  1. Start AltServer on your machine by opening AltTester Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented apps on your device. Make sure they have different names. In case you want to change the name, you can do that in the green popup. There is no need to make another instrumented build.

  3. Change the hosts from the green popups in your instrumented builds to the machine’s IP AltServer is running on. The server identifies the apps using the appName.

  4. Connect your tests to AltServer using the line below in your OneTimeSetup(). You will need to create 2 AltDrivers as you have 2 apps. AltDriver1 will communicate with app1 and AltDriver2 with app2. Start your tests on the machine used before. Make sure that AltServer, the instrumented app and your tests are using the same port. Data transmission between tests and server happens on localhost; transmission between device and server happens on the host’s IP.

altDriver1 = new AltDriver (host = "127.0.0.1", port = 13000, appName = "MyApp1");
altDriver2 = new AltDriver (host = "127.0.0.1", port = 13000, appName = "MyApp2");

Connection through USB

Use Reverse Port Forwarding. Data transmission happens exclusively on localhost.

Important

On mobile devices, AltDriver can interact only with a single app at a time and the app needs to be in focus. In case of 2 drivers and 2 apps, you need to switch (in your test scripts) between the applications. This is due to the fact that on Android/iOS only one application is in focus at a time, even when using split screen mode.

Using AltTester Unity SDK in Release mode

By default AltTester Unity SDK does not run in release mode. We recommended that you do not instrument your Unity application in release mode with AltTester Unity SDK. That being said, if you do want to instrument your application in release mode, you need to uncheck RunOnlyInDebugMode flag on AltRunnerPrefab inside AltTester Unity SDK asset folder AltTester/Prefab/AltRunnerPrefab.prefab

Logging

There are two types of logging that can be configured in AltTester Unity SDK. The logs from AltDriver (from the tests) and the logs from the AltTester Unity SDK (from the instrumented Unity application)

Note

From version 1.7.0 on logs from Server are referred to as logs from Tester.

AltTester Unity SDK logging

Logging inside the instrumented Unity application is handled using a custom NLog LogFactory. The Server LogFactory can be accessed here: AltTester.AltTesterUnitySDK.Logging.ServerLogManager.Instance

There are two logger targets that you can configure on the server:

  • FileLogger

  • UnityLogger

Logging inside the instrumented app can be configured from the driver using the SetServerLogging command:

altDriver.SetServerLogging(AltLogger.File, AltLogLevel.Off);
altDriver.SetServerLogging(AltLogger.Unity, AltLogLevel.Info);

AltDriver logging

Logging on the driver is handled using NLog in C#, loguru in python and log4j in Java. By default logging is disabled in the driver (tests). If you want to enable it you can set the enableLogging in AltDriver constructor.

Logging is handled using a custom NLog LogFactory. The Driver LogFactory can be accessed here: AltTester.AltTesterUnitySDK.Driver.Logging.DriverLogManager.Instance

There are three logger targets that you can configure on the driver:

  • FileLogger

  • UnityLogger //available only when runnning tests from Unity

  • ConsoleLogger //available only when runnning tests using the Nuget package

If you want to configure different level of logging for different targets you can use AltTester.AltTesterUnitySDK.Driver.Logging.DriverLogManager.SetMinLogLevel(AltLogger.File, AltLogLevel.Info)

/* start AltDriver with logging disabled */
var altDriver = new AltDriver (enableLogging: false);

/* start AltDriver with logging enabled for Debug.Level; this is the default behaviour*/
var altDriver = new AltDriver (enableLogging: true);

/* disable AltDriver logging */
altDriver.SetLogging(enableLogging: false);

/* enable AltDriver logging */
altDriver.SetLogging(enableLogging: true);

/* set logging level to Info for File target */
AltTester.AltTesterUnitySDK.Driver.Logging.DriverLogManager.SetMinLogLevel(AltLogger.File, AltLogLevel.Info);

Logging in WebGL

The logs for a WebGL instrumented build are displaied in the browser’s console. You can open the Console tab by pressing F12. To download the logs right click inside the Console and choose Save as....

Save as...

Code Stripping

AltTester Unity SDK is using reflection in some of the commands to get information from the instrumented application. If you application is using IL2CPP scripting backend then it might strip code that you would use in your tests. If this is the case we recommend creating an link.xml file. More information on how to manage code stripping and create an link.xml file is found in Unity documentation