step.yml

title: Virtual Device Testing for Android
summary: Run Android UI tests on virtual devices
description: |-
  Run Android UI tests on virtual devices. This Step collects the built APK/AAB file from the `$BITRISE_APK_PATH` and in case of instrumentation tests, the `$BITRISE_TEST_APK_PATH` Environment Variables and uses Firebase Test Lab to run UI tests on them.

  The available test types are instrumentation, robo, gameloop.

  ### Configuring the Step

  You can read [our detailed guide about using the Step](https://devcenter.bitrise.io/en/testing/device-testing-for-android.html) with other Bitrise Steps. Here we'll go over the configuration options of the Step.

  1. Make sure the **App path** input points to the path of the APK or AAB file of your app. If you use our **Android Build** or **Android Build for UI Testing** Steps, you don't need to change the default value.
  1. Add the devices you want the tests to run on in the **Test devices** input.

     You need to set the device ID, the version, the orientation, and the language. Read the input description for more information and available devices.
  1. Choose a test type.

     The available options are:
     - instrumentation
     - robo
     - gameloop

  For detailed configuration options related to the different test types, please check out the [full guide](https://devcenter.bitrise.io/en/testing/device-testing-for-android.html).

  You can also export the results of the Step to the Test reports add-on. All you need to do is to add a **Deploy to Bitrise.io** Step to the end of your Workflow.

  ### Troubleshooting

  If you get the **Build already exists** error, it is because you have more than one instance of the Step in your Workflow. This doesn't work as Bitrise sends the build slug to Firebase and having the Step more than once in the same Workflow results in sending the same build slug multiple times.

  ### Useful links

  - [Device testing for Android](https://devcenter.bitrise.io/en/testing/device-testing-for-android.html)
  - [Test reports](https://devcenter.bitrise.io/en/testing/test-reports.html)

  ### Related Steps

  - [Android Build](https://www.bitrise.io/integrations/steps/android-build)
  - [Android Build for UI Testing](https://www.bitrise.io/integrations/steps/android-build-for-ui-testing)
  - [Deploy to Bitrise.io](https://www.bitrise.io/integrations/steps/deploy-to-bitrise-io)

website: https://github.com/bitrise-steplib/steps-virtual-device-testing-for-android
source_code_url: https://github.com/bitrise-steplib/steps-virtual-device-testing-for-android
support_url: https://github.com/bitrise-steplib/steps-virtual-device-testing-for-android/issues

project_type_tags:
- android
- cordova
- flutter
- ionic
- kotlin-multiplatform
- react-native
type_tags:
- test

is_always_run: false
is_skippable: false
run_if: .IsCI

meta:
  bitrise.io:
    addons_required:
    - addons-testing

toolkit:
  go:
    package_name: github.com/bitrise-steplib/steps-virtual-device-testing-for-android

inputs:
- app_path: $BITRISE_APK_PATH
  opts:
    title: App path
    summary: |
      The path to the app to test (APK or AAB).
    description: |
      The path to the app to test (APK or AAB). By default `android-build` and `android-build-for-ui-testing` Steps export the `BITRISE_APK_PATH` Env Var, so you won't need to change this input.
      Can specify an APK (`$BITRISE_APK_PATH`) or AAB (Android App Bundle) as input (`$BITRISE_AAB_PATH`).

      If nothing is specified then the Step will use a default empty Application APK. This will help the library instrumentation tests as it can be used as a shell where the tests will be running.
- test_type: robo
  opts:
    title: Test type
    summary: |
      The type of your test you want to run on the devices. Find more properties below in the selected test type's group.
    description: |
      The type of your test you want to run on the devices. Find more properties below in the selected test type's group.
    is_required: true
    value_options:
    - instrumentation
    - robo
    - gameloop
- test_devices: MediumPhone.arm,33,en,portrait
  opts:
    title: Test devices
    summary: One device configuration per line, each in the `deviceID,version,language,orientation` format. See table below for the available devices.
    description: |
      One device configuration per line, each in the `deviceID,version,language,orientation` format. See table below for the available devices.

      For example:
      ```
      MediumPhone.arm,33,en,portrait
      MediumTablet.arm,30,en,landscape
      ```

      Available devices and their OS versions, generally available models first, newest OS first (generated on 2026-07-28):
      ```
      ┌────────────────────────────────────────────────┬──────────────────────────────────┬──────────────────────────────────┬────────────────────────┬─────────┬─────────────┬─────────┐
      │                   MODEL_NAME                   │             MODEL_ID             │          OS_VERSION_IDS          │          TAGS          │   MAKE  │  RESOLUTION │   FORM  │
      ├────────────────────────────────────────────────┼──────────────────────────────────┼──────────────────────────────────┼────────────────────────┼─────────┼─────────────┼─────────┤
      │ Medium Phone, 6.4in/16cm (Arm)                 │ MediumPhone.arm                  │ 26,27,28,29,30,31,32,33,34,35,36 │                        │ Generic │ 2400 x 1080 │ VIRTUAL │
      │ Medium Tablet, 10.05in/25cm (Arm)              │ MediumTablet.arm                 │ 26,27,28,29,30,31,32,33,34,35    │                        │ Generic │ 2560 x 1600 │ VIRTUAL │
      │ Small Phone, 4.65in/12cm (Arm)                 │ SmallPhone.arm                   │ 26,27,28,29,30,31,32,33,34,35    │                        │ Generic │ 1280 x 720  │ VIRTUAL │
      │ Pixel 2 (Arm)                                  │ Pixel2.arm                       │ 26,27,28,29,30,31,32,33          │                        │ Google  │ 1920 x 1080 │ VIRTUAL │
      │ Generic 720x1600 Android tablet @ 270dpi (Arm) │ AndroidTablet270dpi.arm          │ 30                               │                        │ Generic │ 1600 x 720  │ VIRTUAL │
      │ Google TV Amati                                │ AmatiTvEmulator                  │ 29                               │ beta=29, deprecated=29 │ Google  │ 1080 x 1920 │ VIRTUAL │
      │ Google TV                                      │ GoogleTvEmulator                 │ 30                               │ beta=30, deprecated=30 │ Google  │  720 x 1280 │ VIRTUAL │
      │ Medium Phone (16K page size), 6.4in/16cm (Arm) │ MediumPhone_ps16k.arm            │ 36,37                            │ preview=36, preview=37 │ Generic │ 2400 x 1080 │ VIRTUAL │
      │ Medium Phone (16K page size), 6.4in/16cm (Arm) │ MediumPhone_ps16k_backcompat.arm │ 36                               │ preview=36             │ Generic │ 2400 x 1080 │ VIRTUAL │
      └────────────────────────────────────────────────┴──────────────────────────────────┴──────────────────────────────────┴────────────────────────┴─────────┴─────────────┴─────────┘
      ```

      For the authoritative list, see [Available devices in Test Lab](https://firebase.google.com/docs/test-lab/android/available-testing-devices).
    is_required: true
- num_flaky_test_attempts: "0"
  opts:
    title: Number of times a test execution is reattempted
    summary: Specifies the number of times a test execution should be reattempted if one or more of its test cases fail for any reason.
    description: |-
      Specifies the number of times a test execution should be reattempted if one or more of its test cases fail for any reason.

      An execution that initially fails but succeeds on any reattempt is reported as FLAKY.
      The maximum number of reruns allowed is 10. (Default: 0, which implies no reruns.)
    is_required: true
- test_apk_path: $BITRISE_TEST_APK_PATH
  opts:
    category: Instrumentation Test
    title: Test APK path
    summary: The path to the APK that contains instrumentation tests
    description: The path to the APK that contains instrumentation tests. To build this, you can run the [Build for UI testing](https://bitrise.io/integrations/steps/android-build-for-ui-testing) Step (before this Step).
- inst_test_runner_class:
  opts:
    category: Instrumentation Test
    title: Test runner class
    summary: The fully-qualified Java class name of the instrumentation test runner (leave empty to use the last name extracted from the APK manifest).
    description: The fully-qualified Java class name of the instrumentation test runner (leave empty to use the last name extracted from the APK manifest).
- inst_test_targets:
  opts:
    category: Instrumentation Test
    title: |
      Test targets, seperated with the "," character.
    summary: |
      A list of one or more instrumentation test targets to be run (default: all targets). Each target must be fully qualified with the package name or class name, in one of these formats:
      - `package package_name`
      - `class package_name.class_name`
      - `class package_name.class_name#method_name`
      For example:
      `class com.my.company.app.MyTargetClass,class com.my.company.app.MyOtherTargetClass`
    description: |
      A list of one or more instrumentation test targets to be run (default: all targets). Each target must be fully qualified with the package name or class name, in one of these formats:
      - `package package_name`
      - `class package_name.class_name`
      - `class package_name.class_name#method_name`
      For example:
      `class com.my.company.app.MyTargetClass,class com.my.company.app.MyOtherTargetClass`
- inst_use_orchestrator: "false"
  opts:
    category: Instrumentation Test
    title: Use Orchestrator
    summary: |
      The option of whether running each test within its own invocation of instrumentation with Android Test Orchestrator or not.
    description: |
      The option of whether running each test within its own invocation of instrumentation with Android Test Orchestrator or not.
    is_required: true
    value_options:
    - "false"
    - "true"
- robo_initial_activity:
  opts:
    category: Robo Test
    title: Initial activity
    summary: The initial activity used to start the app during a robo test. (leave empty to get it extracted from the APK manifest)
    description: The initial activity used to start the app during a robo test. (leave empty to get it extracted from the APK manifest)
- robo_max_depth:
  opts:
    category: Robo Test
    title: Max depth
    summary: |
      The maximum depth of the traversal stack a robo test can explore. Needs to be at least 2 to make Robo explore the app beyond the first activity(leave empty to use the default value: `50`)
    description: |
      The maximum depth of the traversal stack a robo test can explore. Needs to be at least 2 to make Robo explore the app beyond the first activity(leave empty to use the default value: `50`)
- robo_max_steps:
  opts:
    category: Robo Test
    title: Max steps
    summary: |
      The maximum number of steps/actions a robo test can execute(leave empty to use the default value: `no limit`).
    description: |
      The maximum number of steps/actions a robo test can execute(leave empty to use the default value: `no limit`).
- robo_directives:
  opts:
    category: Robo Test
    title: Robo directives
    summary: |
      To complete text fields in your app, use robo-directives and provide a comma-separated list of key-value pairs, where the key is the Android resource name of the target UI element, and the value is the text string. EditText fields are supported but not text fields in WebView UI elements.
      For example, you could use the following parameter for custom login:
      ```
      username_resource,username,ENTER_TEXT
      password_resource,password,ENTER_TEXT
      loginbtn_resource,,SINGLE_CLICK
      ```
      One directive per line, the parameters are separated with `,` character. For example: `ResourceName,InputText,ActionType`
    description: |
      To complete text fields in your app, use robo-directives and provide a comma-separated list of key-value pairs, where the key is the Android resource name of the target UI element, and the value is the text string. EditText fields are supported but not text fields in WebView UI elements.
      For example, you could use the following parameter for custom login:
      ```
      username_resource,username,ENTER_TEXT
      password_resource,password,ENTER_TEXT
      loginbtn_resource,,SINGLE_CLICK
      ```
      One directive per line, the parameters are separated with `,` character. For example: `ResourceName,InputText,ActionType`
- robo_scenario_file:
  opts:
    category: Robo Test
    title: Robo scenario file path
    summary: A path to a JSON file with a sequence of recorded actions Robo should perform before the Robo crawl.
- loop_scenarios:
  opts:
    category: Game Loop Test
    title: Loop scenarios
    summary: "A list of game-loop scenario numbers which will be run as part of the test (default: all scenarios)."
    description: |
      A list of game-loop scenario numbers which will be run as part of the test (default: all scenarios).
      A maximum of 1024 scenarios may be specified in one test matrix.
      Format: int,[int,...]
      For example:
      ```
      1,2
      ```
- loop_scenario_labels:
  opts:
    category: Game Loop Test
    title: Loop scenario labels
    summary: "A list of game-loop scenario labels (default: None)."
    description: |
      A list of game-loop scenario labels (default: None).
      Each game-loop scenario may be labeled in the APK manifest file with one or more arbitrary strings, creating logical groupings (e.g. GPU_COMPATIBILITY_TESTS).
- test_timeout: "900"
  opts:
    category: Debug
    title: Test timeout
    summary: |
      Max time a test execution is allowed to run before it is automatically canceled. The default value is 900 (15 min), the maximum is 3600 (60 min).  Duration in seconds with up to nine fractional digits. Example: "3.5".
    description: |
      Max time a test execution is allowed to run before it is automatically canceled. The default value is 900 (15 min), the maximum is 3600 (60 min).  Duration in seconds with up to nine fractional digits. Example: "3.5".
    is_required: true
- obb_files_list:
  opts:
    category: Test setup
    title: OBB files
    summary: Upload OBB files before tests will run
    description: |
      A list of one or two Android OBB file names which will be copied to each test device before the tests will run (default: None).
      Each OBB file name must conform to the format as specified by Android (e.g. [main|patch].0300110.com.example.android.obb) and will be installed into `[shared-storage]/Android/obb/[package-name]/` on the test device.
      Files should be seperated by newline.
      For example:
      ```
      main.0300110.com.example.android.obb
      patch.0300110.com.example.android.obb
      ```
- auto_google_login: "false"
  opts:
    category: Test setup
    title: Add preconfigured Google account
    summary: Add preconfigured Google account before tests will run
    description: |-
      Automatically log into the test device using a preconfigured Google
      account before beginning the test.
    is_required: true
    value_options:
    - "false"
    - "true"
- environment_variables:
  opts:
    category: Test setup
    title: Environment Variables
    summary: |
      One variable per line, key and value seperated by `=`
      For example:
      ```
      coverage=true
      coverageFile=/sdcard/tempDir/coverage.ec
      ```
    description: |
      One variable per line, key and value seperated by `=`
      For example:
      ```
      coverage=true
      coverageFile=/sdcard/tempDir/coverage.ec
      ```
- directories_to_pull:
  opts:
    category: Debug
    title: Directories to pull
    summary: |
      A list of paths that will be downloaded from the device's storage after the test is complete.

      For example

      ```
      /sdcard/tempDir1
      /data/tempDir2
      ```

      If `download_test_results` input is set to `false` then these files will be downloaded to the dashboard only. To have them downloaded set that input to `true` as well.
    description: |
      A list of paths that will be downloaded from the device's storage after the test is complete.

      For example

      ```
      /sdcard/tempDir1
      /data/tempDir2
      ```

      If `download_test_results` input is set to `false` then these files will be available on the dashboard only. To have them downloaded set that input to `true` as well.
- download_test_results: "false"
  opts:
    category: Debug
    title: Download files
    summary: |
      If this input is set to `true` all files generated in the test run and the files you downloaded from the device (if you have set `directories_to_pull` input as well) will be downloaded. Otherwise, no any file will be downloaded.
    description: |
      If this input is set to `true` all files generated in the test run and the files you downloaded from the device (if you have set `directories_to_pull` input as well) will be downloaded. Otherwise, no any file will be downloaded.
    is_required: true
    value_options:
    - "false"
    - "true"
- use_verbose_log: "false"
  opts:
    category: Debug
    title: Verbose log
    summary: |
      If set to `true` will enable verbose level logging.
    is_required: true
    value_options:
    - "false"
    - "true"
- apk_path:
  opts:
    category: Deprecated
    title: APK path
    summary: |
      Deprecated. Use 'App path' input instead of this one.
      The path to the APK you want the tests run with. By default `gradle-runner` step exports `BITRISE_APK_PATH` env, so you won't need to change this input.
    description: |
      Deprecated. Use 'App path' input instead of this one.
      The path to the APK you want the tests run with. By default `gradle-runner` step exports `BITRISE_APK_PATH` env, so you won't need to change this input.
    is_required: false
- app_package_id:
  opts:
    category: Deprecated
    title: App package ID
    summary: |
      Deprecated: If not specified will be automatically extracted from the App manifest.
      The Java package of the application under test.
    description: |
      Deprecated: If not specified will be automatically extracted from the App manifest.
      The Java package of the application under test.
- inst_test_package_id:
  opts:
    category: Deprecated
    title: Test package ID
    summary: |
      Deprecated: If not specified will be automatically extracted from the Test App manifest.
      The Java package name of the instrumentation test.
    description: |
      Deprecated: If not specified will be automatically extracted from the Test App manifest.
      The Java package name of the instrumentation test.
- api_base_url: https://vdt.bitrise.io/test
  opts:
    category: Debug
    title: Test API's base URL
    summary: The URL where test API is accessible.
    description: |
      The URL where test API is accessible.
    is_required: true
    is_dont_change_value: true
- api_token: $ADDON_VDTESTING_API_TOKEN
  opts:
    category: Debug
    title: API Token
    summary: The token required to authenticate with the API.
    description: |
      The token required to authenticate with the API.
    is_required: true
    is_dont_change_value: true
    is_sensitive: true
- quarantined_tests: $BITRISE_QUARANTINED_TESTS_JSON
  opts:
    category: Debug
    title: Quarantined tests
    summary: JSON list of tests added to quarantine on Bitrise.io, quarantined tests are excluded from test runs.
outputs:
- VDTESTING_DOWNLOADED_FILES_DIR: ""
  opts:
    title: Downloaded files directory
    summary: The directory containing the downloaded files if you have set `directories_to_pull` and `download_test_results` inputs above.

- BITRISE_FLAKY_TEST_CASES:
  opts:
    title: List of flaky test cases
    summary: A list of flaky test cases. A test case is considered flaky if it has failed at least once, but passed at least once as well.
    description: |-
      A list of flaky test cases. A test case is considered flaky if it has failed at least once, but passed at least once as well.

      The list contains the test cases in the following format:
      ```
      - TestSuit_1.TestClass_1.TestName_1
      - TestSuit_1.TestClass_1.TestName_2
      - TestSuit_1.TestClass_2.TestName_1
      - TestSuit_2.TestClass_1.TestName_1
      ...
      ```

      To export `BITRISE_FLAKY_TEST_CASES` Step Output `download_test_results` Step Input should be set to `true`.