React Native E2E tests on Expo Workflows

Expo Workflows can run warden e2e on a macOS worker with an iOS Simulator. The workflow on this page reuses the EAS build that matches the app's native fingerprint (it builds one only when none matches), installs warden, repacks this commit's JavaScript into the build in seconds and runs the specs. No native build when only JavaScript changed.

warden's own repository runs this workflow on its example app: apps/example/.eas/workflows/e2e.yml. It runs warden from that checkout by default, to test changes before they are released; -F warden=npm runs the npm install on this page.

Prerequisites

• An Expo project linked to EAS (eas init), with ios.bundleIdentifier in the app config.
• Continuous Native Generation: the fingerprint job hashes the generated native project.
• @delacour/warden-e2e as a dev dependency, and a suite in warden.config.ts that installs the app (app: true). See Suites.

export default {
  projects: [{ name: "my-app", bundleId: "com.example.app" }],
  e2e: {
    "my-app": {
      project: "my-app",
      specsDir: "e2e",
      platform: "ios",
      app: true,
    },
  },
};

Add a build profile

Add a profile that makes a Release iOS Simulator build without the dev client:

{
  "build": {
    "e2e-ios-simulator": {
      "environment": "preview",
      "developmentClient": false,
      "ios": {
        "simulator": true,
        "buildConfiguration": "Release"
      }
    }
  }
}

The e2e job hands this build to warden builds import --kind simulator-base, which files it as the base that warden repacks this checkout's JavaScript into. The import refuses a Debug or expo-dev-client build, a device build, a build with no arm64 slice (EAS macOS workers are Apple silicon) and a build with another bundle id. The JavaScript bundle the build embeds is stripped from the cached copy, and every repack writes its own, so any build of the right native fingerprint will do. EXPO_PUBLIC_* values on the profile do not reach the specs: the repack inlines the suite's bundle env (metro.env, see Build kinds).

environment: "preview" makes the build read the same EAS environment variables as the e2e job.

Create the workflow

name: e2e

on:
  workflow_dispatch: {}

jobs:
  # Hash the native project, then look for a finished build of the profile with that hash.
  fingerprint:
    type: fingerprint

  get_ios_build:
    needs: [fingerprint]
    type: get-build
    params:
      platform: ios
      profile: e2e-ios-simulator
      fingerprint_hash: ${{ needs.fingerprint.outputs.ios_fingerprint_hash }}

  # Build only when no build matches the fingerprint.
  build_ios:
    needs: [get_ios_build]
    if: ${{ !needs.get_ios_build.outputs.build_id }}
    type: build
    params:
      platform: ios
      profile: e2e-ios-simulator

  e2e_ios:
    name: warden e2e (iOS)
    after: [get_ios_build, build_ios]
    if: ${{ always() && (needs.get_ios_build.outputs.build_id || needs.build_ios.outputs.build_id) }}
    runs_on: macos-large
    environment: preview
    env:
      WARDEN_GOLDEN: '0'
    steps:
      - uses: eas/checkout

      - uses: eas/install_node_modules

      # warden must match the @delacour/warden-e2e your specs import.
      - name: Pick the warden version
        id: warden
        run: set-output version "$(node -p "require('./node_modules/@delacour/warden-e2e/package.json').version")"

      - uses: eas/restore_cache
        id: warden_cache
        with:
          key: warden-cli-darwin-arm64-${{ steps.warden.outputs.version }}
          path: /Users/expo/.warden-cli

      - name: Install warden
        run: |
          BIN="$HOME/.warden-cli/lib/node_modules/@delacour/warden/node_modules/@delacour/warden-darwin-arm64/bin"
          if [ ! -x "$BIN/warden" ]; then
            npm install --global --prefix "$HOME/.warden-cli" "@delacour/warden@${{ steps.warden.outputs.version }}"
          fi
          set-env PATH "$BIN:$PATH"

      - uses: eas/save_cache
        if: ${{ steps.warden_cache.outputs.cache_hit != 'true' }}
        with:
          key: warden-cli-darwin-arm64-${{ steps.warden.outputs.version }}
          path: /Users/expo/.warden-cli

      - uses: eas/download_build
        id: download_build
        with:
          build_id: ${{ needs.get_ios_build.outputs.build_id || needs.build_ios.outputs.build_id }}
          extensions: [app]

      - name: Use the EAS build as the simulator base
        run: warden builds import "${{ steps.download_build.outputs.artifact_path }}" --kind simulator-base

      - name: Run the specs
        run: warden e2e my-app --all --headless --report /Users/expo/warden-e2e/e2e-report.json

      - uses: eas/upload_artifact
        if: ${{ always() }}
        with:
          type: other
          name: warden-e2e
          ignore_error: true
          path: |
            /Users/expo/warden-e2e/**/*
            /Users/expo/.warden/batches/**/*
            /Users/expo/.warden/logs/**/*

What each part does:

macos-large (10 cores) is the worker size warden's own runs use; macos-medium works for one simulator. Add --count 2 to run two simulators on macos-large, and --retry 1 to rerun a failed spec once.

Why warden repacks, not eas/repack

EAS has its own repack (the repack job and @expo/repack-app). This workflow does not use it: warden e2e repacks the downloaded build itself (about a minute on macos-large).

• The same repack everywhere. It is what warden e2e does on a laptop and on GitHub Actions, for Expo apps (expo export:embed) and bare React Native apps (react-native bundle) alike.
• Cached by content. A repack is keyed on the native base and a hash of the JavaScript inputs, and every simulator in the run installs the same one. Save ~/.warden/builds/*/ios/*/simulator/repacks with eas/save_cache to reuse it on a later run with the same JavaScript.
• Made for testing. It turns expo-updates off in the repacked app, so the app runs the bundle under test rather than an update it fetched, and keeps the base's entitlements when it re-signs.
• One step. The repack job's output is a new EAS build, which the e2e job would still download and import before warden repacks it again.

Run the workflow

eas workflow:run .eas/workflows/e2e.yml

The run page on expo.dev shows each job; the e2e job's log prints each failing spec and the path of its evidence. To pass the suite and arguments per run, declare workflow_dispatch inputs and use them as ${{ inputs.suite }} in the run lines, then run with -F suite=my-app -F args='--specs app/todos' (inputs).

Run on pull requests

Link the GitHub repository to the Expo project (the Expo GitHub app), then add a pull_request trigger:

on:
  workflow_dispatch: {}
  pull_request:
    branches: [main]

To start the workflow from a GitHub Actions job instead, see Start an EAS workflow from GitHub Actions.

Environment variables and secrets

environment: preview on the e2e job reads the variables of the project's preview environment, the same ones the build profile reads. Store tokens there with the visibility set to Secret, for example NPM_TOKEN when eas/install_node_modules installs packages from a private registry. See Environment variables in EAS.

Read the results

The run page's Artifacts section holds warden-e2e: e2e-report.json, the batch directories (results.jsonl, and per failed test failure.png, screen-at-failure.txt, trace.jsonl) and warden's logs. Read screen-at-failure.txt first: it is the full accessibility tree at the moment the test failed. See Debugging.

Troubleshooting