Advanced Usage

This guide covers some of the more advanced features, patterns and configuration options of AltTester® Unreal SDK.

What is reverse port forwarding and when to use it

Reverse port forwarding, is the behind-the-scenes process of intercepting data traffic and redirecting it from a device’s IP and/or port to the computer’s IP and/or port.

When you run your app instrumented with AltTester® Unreal SDK on a device, you need to tell your build how to connect to the AltTester® Server.

Reverse port forwarding can be set up either through the command line or in the test code by using the methods available in the AltTester® SDK classes.

The following are some cases when reverse port forwarded is needed:

  1. Connect to the app running on a USB connected device

  2. Connect to multiple devices running the app

How to setup reverse port forwarding

In case of Android

Reverse port forwarding can be set up in two ways:

  • through the command line using ADB

  • in the test code by using the methods available in the AltTester® SDK classes

All methods listed above require that you have ADB installed.

For further information including how to install ADB, check this article.

In case of iOS

Unfortunately, IProxy does not have a way of setting up reverse port forwarding. As a workaround, to connect the device via USB you should follow the steps below:

  • set the iOS device as a Personal Hotspot

  • enable Hotspot via USB on the machine running the AltTester® Server

    • for this to work, you need to make sure that you have the Disable unless needed toggle disabled in the Network settings for the USB connection

    ../_images/connect-via-hotspot-USB_iOS.png

    • the hotspot network and the first device to connect to it are most of the time on 172.20.10.2 so you could set this IP for builds for iOS

  • add the IP of the machine running the AltTester® Server to the first input field in the green popup from the instrumented app/game

In the routing table, the personal hotspot network would be secondary, therefore the traffic shouldn’t be redirected through the hotspot:

../_images/workaround_iOS.png

  • Reverse port forwarding using the following command:

    adb [-s UDID] reverse tcp:device_port tcp:local_port.

Note

The default port on which the AltTester® Unreal SDK is running is 13000. The port can be changed from the green popup. Make sure to press Restart after modifying its value.

Connect AltTester® Unreal SDK running inside the app to AltTester® Server

There are multiple scenarios:

Establish connection when the instrumented app and the test code are running on the same machine

reverse port forwarding case 1

  1. Start AltTester® Server on your machine by opening AltTester® Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented app on the same machine. It will automatically connect to AltTester® Server. The server identifies the app using the appName.

  3. Connect your tests to the server using the line below in your OneTimeSetup(). Start your tests on the machine used before. Make sure that AltTester® Server, the instrumented app and your tests are using the same port. Data transmission happens on localhost.

altDriver = new AltDriver (host: "127.0.0.1", port: 13000, appName: "MyApp");

In this case reverse port forwarding is not needed as both the app and tests are using localhost:13000.

Establish connection when the app is running on a device connected via USB

reverse port forwarding case 2

  1. Start AltTester® Server on your machine by opening AltTester® Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented app on your device.

  3. Use Reverse Port Forwarding to direct the data traffic from the device’s port to the computer’s port. After this, your app will be connected to AltTester® Server. The server identifies the app using the appName.

  4. Connect your tests to AltTester® Server using the line below in your OneTimeSetup(). Start your tests on the machine used before. Make sure that AltTester® Server, the instrumented app and your tests are using the same port. Data transmission happens on localhost.

altDriver = new AltDriver (host: "127.0.0.1", port: 13000, appName: "MyApp");

Establish connection via IP when the app is running on a device

reverse port forwarding case 3

  1. Start AltTester® Server on your machine by opening AltTester® Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented app on your device.

  3. Change the host from the green popup in your instrumented build to the machine’s IP AltTester® Server is running on. The server identifies the app using the appName.

  4. Connect your tests to AltTester® Server using the line below in your OneTimeSetup(). Start your tests on the machine used before. Make sure that AltTester® Server, the instrumented app and your tests are using the same port. Data transmission between tests and server happens on localhost; transmission between device and server happens on the host’s IP.

altDriver = new AltDriver (host: "127.0.0.1", port: 13000, appName: "MyApp");

In this case Reverse Port Forwarding is not needed. Despite that, it is recommended to use reverse port forwarding since IP addresses could change and would need to be updated more frequently.

Establish connection when different instances of the same app are running on multiple devices

Connection through IP

reverse port forwarding case 4

  1. Start AltTester® Server on your machine by opening AltTester® Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented app on your devices. Make sure they have different names. In case you want to change the name, you can do that in the green popup. There is no need to make another instrumented build.

  3. Change the hosts from the green popups in your instrumented builds to the machine’s IP AltTester® Server is running on. The server identifies the apps using the appName.

  4. Connect your tests to AltTester® Server using the line below in your OneTimeSetup(). You will need to create 2 AltDrivers as you have 2 devices. AltDriver1 will communicate with device1 and AltDriver2 with device2. Start your tests on the machine used before. Make sure that AltTester® Server, the instrumented app and your tests are using the same port. Data transmission between tests and server happens on localhost; transmission between devices and server happens on the host’s IP.

altDriver1 = new AltDriver (host: "127.0.0.1", port: 13000, appName: "MyApp1");
altDriver2 = new AltDriver (host: "127.0.0.1", port: 13000, appName: "MyApp2");

The same happens with n devices. Repeat the steps n times.

Connection through USB

Use reverse port forwarding for both devices. Data transmission happens exclusively on localhost. Ex. with 2 Android devices:

adb -s deviceId1 reverse tcp:13000 tcp:1300
adb -s deviceId2 reverse tcp:13000 tcp:1300

Establish connection when multiple instances of the same application are running on the same device

Connection through IP

reverse port forwarding case 5

  1. Start AltTester® Server on your machine by opening AltTester® Desktop. The server will be listening on port 13000 by default.

  2. Open your instrumented apps on your device. Make sure they have different names. In case you want to change the name, you can do that in the green popup. There is no need to make another instrumented build.

  3. Change the hosts from the green popups in your instrumented builds to the machine’s IP AltTester® Server is running on. The server identifies the apps using the appName.

  4. Connect your tests to AltTester® Server using the line below in your OneTimeSetup(). You will need to create 2 AltDrivers as you have 2 apps. AltDriver1 will communicate with app1 and AltDriver2 with app2. Start your tests on the machine used before. Make sure that AltTester® Server, the instrumented app and your tests are using the same port. Data transmission between tests and server happens on localhost; transmission between device and server happens on the host’s IP.

altDriver1 = new AltDriver (host: "127.0.0.1", port: 13000, appName: "MyApp1");
altDriver2 = new AltDriver (host: "127.0.0.1", port: 13000, appName: "MyApp2");

Connection through USB

Use Reverse Port Forwarding. Data transmission happens exclusively on localhost.

Important

On mobile devices, AltDriver can interact only with a single app at a time and the app needs to be in focus. In case of 2 drivers and 2 apps, you need to switch (in your test scripts) between the applications. This is due to the fact that on Android/iOS only one application is in focus at a time, even when using split screen mode.

Execute tests concurrently

In the AltDriver constructor you have the option to specify multiple tags. The available tags are: app name, platform, platform version, device instance id and app id. The app id can be used to uniquely identify an app. In case you specify no tags, the tests will be run on a randomly chosen app.

Keep in mind that, the tags given in the constructor will choose one random free app satisfying the requirements. Only one test can run on one app simultaneously. If you want to run the same tests on multiple apps concurrently, you have to start the dotnet test command multiple times, once for each app/device that you want your tests to be executed on. Depending on your setup, you might want to replace the dotnet test command with pytest or any other command that you usually use to start your tests.

Note

In order to ensure that the dotnet test command is executed multiple times concurrently within the same terminal add an & at the end of the command to run it in the background.

Note

Make sure that your product names are different in case you started multiple instrumented apps on the same device, otherwise your tests might fail because they are using the same resources.

Ex1. Let’s say we want to run a set of tests on all apps started on Windows 11 (the exact platform version is displayed in the green popup and in AltTester® Desktop). For that, use the following code snippet:

altDriver = new AltDriver (host: "127.0.0.1", port: 13000, platformVersion: "Windows 11  (10.0.22621) 64bit");

Ex2. Let’s say we want to run the same set of tests on Windows and Android platforms. If you run your tests with pytest, use the following code snippets:

In your test file:

def test(platform):
    alt_driver = AltDriver(host="127.0.0.1", port=13000, platform=platform)

In your conftest.py file:

def pytest_addoption(parser):
    parser.addoption("--platform", action="store", default="default name")


def pytest_generate_tests(metafunc):
    option_value = metafunc.config.option.platform
    if 'platform' in metafunc.fixturenames and option_value is not None:
        metafunc.parametrize("platform", [option_value])

Then you can run from the command line with a command line argument:

pytest --platform "WindowsPlayer" &
pytest --platform "Android"

Another way of doing this is with environment variables:

In your test file:

def test():
    alt_driver = AltDriver(host="127.0.0.1", port=13000, platform=get_platform())

In your conftest.py file:

def get_platform():
    return os.environ.get("PLATFORM", "")

Then you can set the environment variables and run from the command line the pytest command:

export PLATFORM="WindowsPlayer"
pytest &
export PLATFORM="Android"
pytest

AltDriver logging

Logging on the driver is handled using NLog in C#, loguru in python and log4j in Java. By default logging is disabled in the driver (tests). If you want to enable it you can set the enableLogging in AltDriver constructor.

Logging is handled using a custom NLog LogFactory. The Driver LogFactory can be accessed here: AltTester.AltTesterUnitySDK.Driver.Logging.DriverLogManager.Instance

There are three logger targets that you can configure on the driver:

  • FileLogger

  • ConsoleLogger //available only when runnning tests using the Nuget package

If you want to configure different level of logging for different targets you can use AltTester.AltTesterUnitySDK.Driver.Logging.DriverLogManager.SetMinLogLevel(AltLogger.File, AltLogLevel.Info)

/* start AltDriver with logging enabled */
var altDriver = new AltDriver (enableLogging: true);

/* start AltDriver with logging disabled */
var altDriver = new AltDriver (enableLogging: false);

/* disable AltDriver logging */
altDriver.SetLogging(enableLogging: false);

/* enable AltDriver logging */
altDriver.SetLogging(enableLogging: true);

/* set logging level to Info for File target */
AltTester.AltTesterUnitySDK.Driver.Logging.DriverLogManager.SetMinLogLevel(AltLogger.File, AltLogLevel.Info);

Generate testing reports using Allure

NUnit

Prerequisites

  1. Allure installed on your system:

  2. NUnit project where all the test classes belong to a certain namespace.

  3. (Not a must) VS Code installed with the Live Server extention.

* surely you can use any other IDE if it has these features.

Setup

  1. Add the Allure NUnit package to your project:

    dotnet add package Allure.NUnit --version 2.9.5-preview.1

  • Other versions: https://www.nuget.org/packages/Allure.NUnit/

  1. Create two folders called allure-report and allure-results under your project.

  2. Add an allureConfig.json file at the following path /bin/Debug/netcoreappX (where X is the version of your dotnet)

    • Config file example.

    • the value of the directory property should be the full path to the allure-results previously created folder.

  3. In the tests files, import the AllureNUnit adapter using NUnit.Allure.Core.

  4. Use the attribute [TestFixture] and the [AllureNUnit] under it.

    • Additionally, you can add more attributes that increase the diversity of your report. See more examples here.

How to run the tests to obtain an Allure report

  1. Execute tests to generate the output in the allure-results folder by using the command:

    dotnet test --results-directory allure-results

  2. Generate a report in the allure-report folder:

    allure generate allure-results -o allure-report

How to check the results

  • Using VS Code and Live Server:

    • Navigate to the allure-report folder and open the index.html file with Live server.

  • Using an allure command:

    allure serve allure-results

This command will generate a new report but not in a specific output. To find the report’s location, check the terminal output and there will be a message like Report successfully generated to PATH where the path is the report’s location.

More details related to Allure can be found at the official Allure documentation.

Pytest

Prerequisites

  1. Allure installed on your system:

  2. Pytest installed (pip install pytest)

Setup

  1. Add the allure-pytest dependency to your project:

    pip install allure-pytest

  2. Create a folder called allure-report by using the following command in your terminal:

    allure generate

How to run the tests to obtain an Allure report

  1. Execute tests to generate the output in the allure-report folder by using the command:

    pytest -v --alluredir=allure-report/ test_suite.py

  2. For viewing the allure report use the following command after the previous:

    allure serve allure-report/

How to obtain a single html report

In order to obtain a single html file with the whole report, you should use allure-combine. Please follow the steps:

  1. Install allure-combine using the following command in your terminal:

    pip install allure-combine

  2. Generate a non-combined report by using the follosing command:

    allure generate -c allure-report -o allure-results-html

  3. Generate a single html file with the whole report:

    allure-combine ./allure-results-html

! For MacOS, you should replace pip with pip3. The name of the combined report is combined.html and it is under allure-results-html folder.

More details related to Allure can be found at the official Allure documentation.

Java

Prerequisites

  1. Allure installed on your system:

  2. Maven installed on your system:

  3. (Not a must) Allure-combine python package: pip install allure-combine.

Updating the pom.xml

  1. Update the properties section of your pom.xml with the following info:

    <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <junit.version>4.13.2</junit.version>
    <allure.junit4.version>2.14.0</allure.junit4.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <maven-surefire-plugin-version>3.0.0-M5</maven-surefire-plugin-version>
</properties>

  2. Update the build section of your pom.xml with the following info:

    <build>

    <plugins>
<!-- Compiler plug-in -->

        <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> <!--For JAVA 8 use 1.8-->
                    <target>${maven.compiler.target}</target> <!--For JAVA 8 use 1.8-->
                </configuration>
            </plugin>

    <!-- Added Surefire Plugin configuration to execute tests -->       
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin-version}</version>
            <configuration>
                <testFailureIgnore>false</testFailureIgnore>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                </argLine>
                <properties>
                    <property>
                        <name>listener</name>
                        <value>io.qameta.allure.junit4.AllureJunit4</value>
                    </property>
                </properties>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

  3. Include in your dependecies section the following items:

    <dependencies>
        <dependency>
            <groupId>com.alttester</groupId>
            <artifactId>alttester</artifactId>
            <version>2.2.0</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-junit4</artifactId>
        <version>${allure.junit4.version}</version>
        <scope>test</scope>
    </dependency> 
    </dependencies>

For more information, check QA Automation expert tutorial.

How to run the tests to obtain an Allure report

  1. Execute tests to generate the output in the allure-results folder by using the command:

    mvn test

  2. For viewing the allure report use the following command after the previous:

    allure serve allure-results

How to obtain a single html report

In order to obtain a single html file with the whole report, you should use allure-combine which is a python package. Please follow the steps:

  1. Install allure-combine using the following command in your terminal:

    pip install allure-combine

  2. Generate a non-combined report by using the follosing command:

    allure generate -c allure-results -o allure-results-html

  3. Generate a single html file with the whole report:

    allure-combine ./allure-results-html

! For MacOS, you should replace pip with pip3. The name of the combined report is combined.html and it is under allure-results-html folder.

More details related to Allure can be found at the official Allure documentation.