Skip to content

Latest commit

 

History

History
406 lines (319 loc) · 14.7 KB

config.md

File metadata and controls

406 lines (319 loc) · 14.7 KB

Configuration

Appwright provides a set of configuration options that you can use to customize the test environment and thus the behavior of the tests.

Device Providers

Device providers make Appium compatible mobile devices available to Appwright. These providers are supported:

  • local-device
  • emulator
  • browserstack
  • lambdatest

BrowserStack

BrowserStack App Automate can be used to provide remote devices to Appwright.

These environment variables are required for the BrowserStack

  • BROWSERSTACK_USERNAME
  • BROWSERSTACK_ACCESS_KEY

BrowserStack also requires name and osVersion of the device to be set in the projects in appwright config file.

LambdaTest

LambdaTest Real Device Cloud can be used to provide remote devices to Appwright.

These environment variables are required for the LambdaTest

  • LAMBDATEST_USERNAME
  • LAMBDATEST_ACCESS_KEY

LambdaTest also requires name and osVersion of the device to be set in the projects in appwright config file.

Local Device & Emulator Configuration

When using provider: "local-device" or provider: "emulator", additional configuration options are available for fine-grained control over app behavior and iOS WebDriverAgent settings.

App Reset Behavior

uninstallAppBeforeTest (optional, default: false)

Controls whether the app should be completely uninstalled before starting each test session.

  • true: The app will be fully uninstalled before each session, ensuring a completely clean state
  • false: The app is not uninstalled (existing installation is used)

Note: On real devices, app data might be cached under certain circumstances even with this enabled.

preserveAppState (optional, default: true)

Controls whether the app state and data should be preserved between test sessions.

  • true: The app continues running and its data is preserved between sessions
  • false: The app is terminated and all its data is cleaned before each session

Example:

{
  name: "android-fresh-install",
  use: {
    platform: Platform.ANDROID,
    device: {
      provider: "local-device",
      uninstallAppBeforeTest: true,  // Clean install each time
      preserveAppState: false,        // Clear app data
    },
    buildPath: "app-release.apk",
  },
}

iOS WebDriverAgent Configuration

updatedWDABundleId (optional, iOS only)

Specifies a custom WebDriverAgent bundle ID to use. This is useful when you've built a custom WebDriverAgent with a different bundle identifier.

Default: com.facebook.WebDriverAgentRunner

Example:

{
  name: "ios-custom-wda",
  use: {
    platform: Platform.IOS,
    device: {
      provider: "local-device",
      updatedWDABundleId: "co.tulip.WebDriverAgentRunner",
    },
    buildPath: "MyApp.ipa",
  },
}

xcodeOrgId / xcodeSigningId (optional, iOS only)

Sign WebDriverAgent for a physical iOS device using the XCUITest driver's automatic configuration. They map to appium:xcodeOrgId (your Apple Developer Team ID) and appium:xcodeSigningId (defaults to "Apple Development" in the driver). Simulators don't need signing.

These values usually differ per developer, so keep them out of the config file and load them from a gitignored .env. Unset values are not sent to Appium, so the same config also works on machines without them.

import "dotenv/config"; // or process.loadEnvFile() on Node >= 20.12

export default defineConfig({
  projects: [
    {
      name: "ios-device",
      use: {
        platform: Platform.IOS,
        device: {
          provider: "local-device",
          xcodeOrgId: process.env.XCODE_ORG_ID,
          xcodeSigningId: process.env.XCODE_SIGNING_ID,
          updatedWDABundleId: process.env.UPDATED_WDA_BUNDLE_ID,
        },
        buildPath: "MyApp.ipa",
      },
    },
  ],
});
# .env
XCODE_ORG_ID=ABCDE12345
UPDATED_WDA_BUNDLE_ID=co.company.WebDriverAgentRunner

Extra Appium capabilities

capabilities (optional, local-device and emulator)

Any other Appium capabilities, merged over the ones appwright builds so a value here wins. Keys without a vendor prefix get appium: unless they are W3C capabilities (platformName, timeouts, …).

device: {
  provider: "local-device",
  capabilities: {
    showXcodeLog: true,                        // sent as appium:showXcodeLog
    allowProvisioningDeviceRegistration: true, // sent as appium:allowProvisioningDeviceRegistration
  },
}

Complete Configuration Example

import { defineConfig, Platform } from "appwright";

export default defineConfig({
  projects: [
    {
      name: "android-local",
      use: {
        platform: Platform.ANDROID,
        device: {
          provider: "local-device",
          udid: "emulator-5554",  // Optional: specific device
          uninstallAppBeforeTest: false,
          preserveAppState: true,
        },
        buildPath: "android/app/build/outputs/apk/release/app-release.apk",
      },
    },
    {
      name: "ios-local",
      use: {
        platform: Platform.IOS,
        device: {
          provider: "local-device",
          updatedWDABundleId: "co.company.WebDriverAgentRunner",
          uninstallAppBeforeTest: true,
          preserveAppState: false,
        },
        buildPath: "ios/build/MyApp.ipa",
      },
    },
  ],
});

Android Emulator

To run tests on the Android emulator, ensure the following installations are available. If not, follow these steps:

  1. Install Android Studio: If not installed, download and install it from here.
  2. Set Android SDK location: Open Android Studio, copy the Android SDK location, and set the ANDROID_HOME environment variable to the same path.
  3. Check Java Installation: Verify if Java is installed by running java -version. If it's not installed:
    • Install Java using Homebrew: brew install java.
    • After installation, run the symlink command provided at the end of the installation process.

To check for available emulators, run the following command:

$ANDROID_HOME/emulator/emulator --list-avds

iOS Simulator

To run tests on the iOS Simulator, ensure the following installations are available. If not, follow these steps:

  1. Install Xcode: If not installed, download and install it from here.
  2. Download iOS Simulator: While installing Xcode, you will be prompted to select the platform to develop for. Ensure that iOS is selected.

To check for available iOS simulators, run the following command:

xcrun simctl list

Running on multiple local devices

Appwright runs one Playwright worker per device: every worker (parallelIndex 0, 1, 2, ...) opens its own Appium session on the device with the same index in the project's device.devices list. This works for both provider: "local-device" and provider: "emulator".

Each entry in devices has:

  • udid (required): the device identifier. For Android physical devices this is the adb serial (adb devices); for Android emulators it is emulator-<port> where the port is an even number between 5554 and 5584 (the console port range the emulator binary uses); for iOS physical devices it is the UDID from xcrun xctrace list devices; for iOS simulators it is the simulator UDID from xcrun simctl list devices.
  • avd (optional, emulator provider, Android only): the AVD name from $ANDROID_HOME/emulator/emulator -list-avds. Appwright boots this AVD on the port taken from udid when that emulator is not already running.

A single udid on the device config is still accepted and is shorthand for devices: [{ udid }]. Leaving both out keeps the old behaviour of picking the only connected device, but then only one worker can run.

Rules to keep in mind:

  • workers must not exceed the number of entries in devices. When devices is listed, Appwright checks this in global setup and fails fast with a message telling you to add devices or lower workers, before any session is opened. Configs that use the single-udid shorthand (or neither) are left to the old behaviour: they keep running as long as Playwright spawns one worker, and the second worker fails when it asks for a device it has not been given.
  • The worker slot is Playwright's parallelIndex, not workerIndex. When Playwright restarts a worker after a failure the new worker reuses the same parallelIndex, so a retried test runs on the same device as the original attempt.
  • One Appium server is started per run on a free port (4723 if available, otherwise the next free one) and shut down when the run ends. If the port is grabbed by another process between the check and the start, Appwright retries once on a fresh port. Workers only create and delete sessions on it; Appium is no longer restarted after every test. The Appium driver is installed once, and skipped if it is already installed.
  • Ports that must be unique per concurrent session on one host (systemPort on Android; wdaLocalPort, mjpegServerPort and derivedDataPath on iOS) are derived from the worker slot automatically, so you do not need to set them.
  • Emulators and simulators that Appwright booted for the run are shut down at the end. Emulators and simulators that were already running when the run started are left alone.
  • iOS simulators listed by udid are booted with xcrun simctl if they are not already booted.

Example: two Android emulators

import { defineConfig, Platform } from "appwright";

export default defineConfig({
  workers: 2,
  projects: [
    {
      name: "android",
      use: {
        platform: Platform.ANDROID,
        device: {
          provider: "emulator",
          devices: [
            { udid: "emulator-5554", avd: "Pixel_7_API_34" },
            { udid: "emulator-5556", avd: "Pixel_7_API_34" },
          ],
        },
        buildPath: "android/app/build/outputs/apk/release/app-release.apk",
      },
    },
  ],
});

Worker 0 uses emulator-5554 and worker 1 uses emulator-5556. The same AVD can back several entries; each one is booted as a separate emulator instance on its own port. The Android emulator only allows this when all instances of that AVD run with -read-only, so Appwright adds the flag to every instance of a shared AVD (their disk changes are discarded on shutdown). An entry without avd that is not already running boots the first installed AVD, with a warning.

Example: two iOS physical devices

import { defineConfig, Platform } from "appwright";

export default defineConfig({
  workers: 2,
  projects: [
    {
      name: "ios",
      use: {
        platform: Platform.IOS,
        device: {
          provider: "local-device",
          devices: [
            { udid: "00008110-000A1B2C3D4E5F67" },
            { udid: "00008120-001122334455AABB" },
          ],
          updatedWDABundleId: "co.company.WebDriverAgentRunner",
        },
        buildPath: "ios/build/MyApp.ipa",
      },
    },
  ],
});

Find the UDIDs with xcrun xctrace list devices. Both devices must be connected and trusted before the run starts; Appwright does not boot or pair physical devices.

Test results per run

Every run writes into its own folders, so two runs started side by side on one machine (for example iOS and Android at the same time) never overwrite each other's output:

test-results/<run>/               Playwright output: per-test artifacts, .last-run.json
test-results/<run>/videos-store/  Appwright worker videos and worker-info files
playwright-report/<run>/          HTML report
blob-report/<run>/                Blob report, when the blob reporter is enabled

Open a report with npx playwright show-report playwright-report/<run>. Global setup logs the folders of the current run when it starts.

Naming a run

  • npx appwright test --project android --run-name nightly names the run nightly. The flag is handled by the appwright CLI and is not passed on to Playwright.
  • APPWRIGHT_RUN_NAME=nightly npx appwright test --project android does the same through the environment, which is convenient in CI. The flag wins when both are given.
  • Without either, the run is named <project>-<YYYYMMDD>-<HHmmss>-<4 random chars> in local time, for example android-20260910-143201-k3x9. Several --project values are joined with +.

Names are used as folder names, so anything other than letters, digits, ., _, - and + is replaced with -, and leading dots are removed.

Interaction with Playwright options

  • A custom outputDir or html outputFolder in your config is kept as the base folder; the run name is nested under it (<outputDir>/<run>).

  • Playwright's own --output <dir> flag still overrides outputDir completely, as it always has.

  • --last-failed reads .last-run.json from the run's output folder. To rerun the failures of an earlier run, pass the same --run-name again.

  • Reporters that own a whole folder are namespaced automatically: the html reporter (playwright-report/<run>) and the blob reporter (blob-report/<run>). Both wipe their folder when they write a report, so without this the second run to finish would delete the first run's report. Merge blob reports from a run with npx playwright merge-reports blob-report/<run>, or collect the .zip files of several runs into one folder first.

  • Reporters that write a single file to a path you chose (json, junit) are left exactly where their options point, because silently moving a path your CI reads would be worse than the collision it avoids. Two concurrent runs do still overwrite each other there, so put the run name in the path yourself when you need those side by side:

    import { defineConfig, resolveRunName } from "appwright";
    
    const run = resolveRunName();
    
    export default defineConfig({
      reporter: [
        ["list"],
        ["html"],
        ["json", { outputFile: `test-results/${run}/results.json` }],
      ],
      // ...
    });
  • Run folders are never deleted automatically. Remove test-results/ and playwright-report/ when you want the disk space back.

Upgrading from a flat layout

Before this change every run wrote straight into test-results/ and playwright-report/. Results are now one level deeper, under the run folder. Update anything that reads a fixed path — CI artifact globs, npx playwright show-report, scripts that open playwright-report/index.html — to include the run folder, and pass --run-name <name> when you want that folder to have a known, stable name:

npx appwright test --project android --run-name ci
npx playwright show-report playwright-report/ci