E2E YAML reference

An e2e test is a YAML file in .argent/e2e/<name>.yaml. The agent records the file with the e2e tools. You can also write or edit the file by hand. This page lists the file shape, the selectors, the directives and the argent e2e command. For the concept, see Record and replay e2e tests.

Argent replays an e2e YAML file step by step on an iOS simulator

Formerly Argent flows

Earlier versions of Argent called this feature Argent flows. The old names still work:

Old name	New name
flow-* tools	e2e-* tools
tool: flow-execute step	tool: e2e-execute step
argent flow command	argent e2e command
.argent/flows/ directory	.argent/e2e/ directory
ARGENT_FLOW_FILES_LOG	ARGENT_E2E_FILES_LOG

Argent reads an e2e test from .argent/e2e/ first, then from .argent/flows/. When both directories hold an e2e test with the same name, Argent runs the file in .argent/e2e/ and prints a warning that names the other file. Argent writes a new e2e test to .argent/e2e/. Argent does not move the existing files. When you move an e2e test by hand, also move its __baselines__/<test>/ directory.

File shape

steps:
  - launch: com.example.app
  - await: { visible: { id: home-screen } }
  - await: { idle: true }

The file has two top-level keys:

Key	Required	Description
steps	yes	The list of steps. Argent runs the steps in order
executionPrerequisite	no	One sentence that names the start state. Only a fragment can declare it

An e2e test has one of two shapes:

Shape	First step that is not echo: or script:	Start state
Standalone e2e test	launch:	Argent starts the app from scratch
Fragment	Any other step	The app is already in the state that the e2e test needs

Argent skips leading echo and script steps when it classifies an e2e test. An e2e test that opens with an echo step and then a launch: step is still standalone. An e2e test that opens with a script step and then a launch: step is also standalone. Put the named start state of a standalone e2e test in a leading echo step.

A file does not store a device id. The runner selects the device. A launch: restarts the process but does not clear the app data, the account or the backend data.

When a step fails, Argent stops the e2e test and reports the later steps as skipped.

Selectors

A selector finds an element on the screen. Write a selector as an explicit map:

{ id: save-button }
{ text: Save }
{ role: button }
{ id: settings-row, text: Notifications }
{ text: { matches: '^Order #\d+$' } }

Field	Match
id	Exact, case-insensitive. The test id or accessibility id
text	Case-insensitive substring, or { matches: <regex> }
role	Case-insensitive substring of the element role
any	Any element. Takes only true. Pair it with a relational scope

All fields in a selector must match. An unqualified Android id also matches its qualified resource id. identifier is an alias of id. Use single quotes for a regex with a backslash. any: true is the only selector without its own id, text or role, so it needs a relational scope, for example { any: true, after: { text: Danger zone } }.

A bare string, for example tap: Save, is a loose selector. It tries id first, then text. Prefer the map form.

When several elements match:

Directive type	Chosen element
Action (tap, long-press, swipe, type, scroll-to, pinch, rotate)	The most specific visible match: exact text or id, then the smallest frame, then reading order
Condition (await, assert)	exists and visible hold when any match qualifies. hidden holds when no match qualifies. text reads the first visible match

Relational scopes

A selector can scope its matches by the frames of other elements:

Key	Meaning
within	The target is inside the frame of the anchor
after	The target follows the anchor in reading order
next	The target is the nearest match that follows the anchor

- tap: { text: Delete, within: { id: profile-card } }
- assert: { visible: { role: Button, after: { text: Danger zone } } }
- tap: { role: Switch, next: { text: Wi-Fi } }

within uses the visual frame, not the source tree. Reading order is top to bottom, left to right, as the user sees the UI. This is also true for a landscape UI, for example on a rotated device or an unfolded foldable. after, next, any: true and a text condition use this reading order. Scopes nest, with at most six scope keys in one selector. The await-ui-element tool does not support scopes.

Runner tree and discovery tree

The runner resolves a selector against a different tree than the describe tool:

Platform	Runner tree	describe tree
iOS simulator	Native UIView hierarchy	Accessibility tree
Physical iPhone	Accessibility snapshot of describe	Same snapshot
Android	Full accessibility hierarchy	Trimmed interactable nodes
Chromium	DOM nodes with an id, label, value or handler	Full DOM
Vega	Toolkit page source	Same source
HarmonyOS	None	uitest layout dump

On an iOS simulator and Android, an id absent from describe can still resolve in an e2e test. On Chromium, an element absent from describe cannot resolve. On an iOS simulator, do not copy a role from describe into a selector: the runner derives the role from the view class, describe from the accessibility traits. On a physical iPhone, the runner and describe read one tree: copy ids and roles from describe.

Directives

Each step contains one directive. A failed directive stops the e2e test.

Directive	Shape	Description
launch	<app id> or { native, ios, android, vega, chromium }	Terminate and start the app, then wait until the app is ready
tap	<selector>, { on: <selector>, times } or { x, y }	Tap an element. times: 2 double-taps. x, y are normalized coordinates
long-press	<selector> or { on: <selector>, duration }	Press and hold. duration in milliseconds
swipe	<direction> or { from, direction, to, by, momentum, duration }	Move one finger across the screen. direction is the travel of the finger. See Swipe
type	{ into: <selector>, text, submit }	Focus the element and type. Presses Enter unless submit: false
scroll-to	{ target: <selector>, direction, within }	Scroll until the target is visible. direction is down (default), up, left or right
pinch	{ on: <selector>, scale }	Pinch around the element, or the screen center when on is absent. scale > 1 zooms in
rotate	{ on: <selector>, by }	Two-finger rotation in degrees, clockwise positive. This is a gesture, not the device orientation
fold	<posture>, <angle> or { posture } / { angle }	Fold or unfold a foldable device. See Foldable devices
await	{ <condition>, timeout } or { idle: true, stableFor, timeout }	Wait for a condition. Default timeout 7500 ms
assert	{ <condition> }	Check a condition now, with a fixed 1000 ms grace. Rejects timeout
wait	<milliseconds>	Pause for a fixed time
snapshot	<name> or { name, maxMismatch, cropOn }	Compare a screenshot with a stored baseline. See Snapshots
run	<path>	Run another e2e test file inline. See Composition
script	{ path, timeout }	Run a local .mjs file in a new Node process. See Local scripts
when	{ <condition> } or { platform }, with steps:	Run the nested steps only when the condition holds. See Optional steps
echo	<message>	Print a message in the report
tool	<tool name>, with args: and optional delayMs:	Call any Argent tool with the given arguments

tap, type and long-press do not scroll. Add scroll-to before them when the target can be off-screen.

A direction in swipe and scroll-to is the direction that the user sees. This is also true for a landscape UI, for example on a rotated device or an unfolded foldable. Coordinates in tap: { x, y } and in the from, to and by of swipe use the space of the describe frames. Argent does not turn them.

A type step can contain a {{secret:NAME}} placeholder. The tool-server fills the value at run time and redacts the value in the report. See Secrets.

A gesture that resolves no selector passes with a warning when the runner cannot read the UI tree. This covers a coordinate tap, a swipe with no selector at either end, and a pinch or rotate without on. The warning says that Argent sent the gesture, not that the gesture reached the element.

Launch map

Use the map form for an e2e test that runs on more than one platform:

- launch: { native: com.acme.app, chromium: ../../app }
- launch: { ios: com.acme.app, android: com.acme.app.android, chromium: ../../app }
- launch: { ios: { app: com.acme.app, args: [-FeatureFlag, YES] }, android: com.acme.app }

native is one id for iOS, Android and Vega. A per-platform key replaces it for that platform. chromium accepts a relative or absolute app path, or { path, args }. ios accepts a bundle id, or { app, args }. On an iOS simulator or a physical iPhone, Argent passes args to the app process at launch. Each arg must be a string, so quote a number or boolean: "5", not 5. A launch that has no id for the platform of the run is an error.

A run on a remote simulator uses the ios entry. If the map has no ios entry, the run uses the native entry.

When the agent records restart-app with launchArgs on iOS, Argent saves a launch step with a native id and an ios: { app, args } entry.

An Android app that starts from a non-launcher activity has no launch: form. Record restart-app with activity as a tool step and keep the e2e test as a fragment.

Swipe

- swipe: left
- swipe: { from: { id: story-card }, direction: left }
- swipe: { by: { y: -0.4 }, duration: 600 }
- swipe: { from: { id: drag-handle }, to: { id: drop-zone }, momentum: false }

A swipe moves one finger across the screen. direction is the travel of the finger as the user sees it, not the travel of the content. To bring an element on screen use scroll-to not swipe.

Option	Value
from	The start point. A selector or { x, y }. Argent selects the start when from is absent
direction	up, down, left or right. Argent moves the finger a preset distance. With from, Argent clamps the travel on screen
to	The end point. A selector or { x, y }
by	A signed delta { x, y } in screen fractions. Argent delivers the exact delta. With from, a delta that leaves the screen fails the step
momentum	false removes the fling at the default duration. Default true
duration	The travel time in milliseconds. Default 300, minimum 150, maximum 10000

A step takes exactly one of direction, to and by. Each travel must cover at least 0.03 of the screen. Parsing rejects a shorter by. A shorter to fails the step during the run.

On Chromium, Argent runs a swipe as a mouse drag. On Vega, a swipe fails like the other touch directives.

Foldable devices

- fold: open
- fold: 120
- fold: { posture: closed }

A fold step moves the hinge of a foldable iOS simulator, for example the iPhone Duo, or of a foldable Android emulator. The step takes a posture (closed, half-open or open) or an angle from 0 to 180 degrees. The step waits until the device accepts input again. When you record an e2e test, Argent saves a call to the fold tool as a fold step.

After a fold step:

• The screen size and the coordinates change with the panel. Later selectors resolve against a new UI tree.
• A snapshot baseline is valid only for the posture that made it. Use a different snapshot name for each posture.
• On iOS, unfolded, the UI is landscape. Directions and reading order follow the UI, as on a rotated device.
• On iOS, a fold between two angles that are not 0 or 180 can keep the current panel. The step passes, and the report names the panel. To change panels, fold to closed or open.
• On Android, the report names the device state that Android committed, for example CLOSED.

Conditions

- await: { visible: { id: settings-screen } }
- await: { hidden: { id: loading-spinner }, timeout: 15000 }
- assert: { exists: { id: notifications-toggle } }
- assert: { text: { in: { id: preference-status }, equals: Enabled } }
- assert: { text: { in: { id: result-count }, matches: '^\d+ results$' } }

Condition	Holds when
exists	An element matches the selector
visible	A visible element matches the selector
hidden	No visible element matches the selector
text	The text of the element in in satisfies exactly one comparator

The text comparators:

Comparator	Match
contains	Case-insensitive substring
equals	Case-insensitive full match
matches	Case-sensitive JavaScript regex

matches reads an element with no text as an empty string. Use '^$' to check that a field is empty, or '^(?!.*@)' to check that it doesn't contain @. A pattern that can match an empty string, such as '(Saved)?', passes on an element with no text. An iOS text field is the exception: on a simulator Argent can't read its typed text, so matches never reads an iOS text field as empty.

A negative condition passes before the element appears, for a typo, and on the wrong screen. First prove the screen and the same selector with visible. Then perform the action and check hidden.

Prove a navigation

Each screen change needs two checks:

- await: { visible: { id: profile-screen } } # identity
- await: { idle: true } # readiness

The identity selector must exist only on the destination. idle waits until the screen has content and stops changing in the UI tree and in the pixels.

- await: { idle: true, stableFor: 400, timeout: 9000 }

Option	Default	Description
stableFor	250 ms	How long the screen must stay still
timeout	7500 ms	The budget of the whole wait

idle never fails a run. A screen that does not settle passes with a warning. The warning names the reason: the screen kept moving, a small part kept moving, the wait ended mid-hold, the tree stayed empty, only the tree settled, or too few reads. Read the warning before you accept the step.

One outcome stops the run: the runner cannot read the UI tree at all. Argent marks the step as errored and skips the later steps.

On HarmonyOS, the runner has no UI tree, so idle cannot run. The step errors immediately, names wait: as the replacement, and Argent skips the later steps.

idle has no assert form and no when form.

Optional steps

- when: { visible: { text: Got it } }
  steps:
    - tap: { text: Got it }

The guard accepts one exists, visible, hidden or text condition, or { platform: ios | android | chromium | vega }. A run on a remote simulator matches ios. A UI guard uses the assert grace and rejects timeout. There is no else. A skipped block reports as skipped. A failure inside an entered block is a real failure. Do not put a required check inside when:.

Composition

- run: ../shared/login.yaml

A run: path resolves against the directory of the e2e test file that contains the step. When nothing is there, the e2e test file is in .argent/e2e/ or .argent/flows/, and the path has no /, Argent looks in the other of these directories. The .yaml suffix is optional. When you record a nested e2e-execute of an e2e test that is only in the other directory, the recorder writes its path from the recording, for example run: ../flows/login.yaml.

Platform	Behavior
iOS, Android	A nested fragment or standalone e2e test runs inline. A nested launch restarts the app
Chromium	Each launch step boots one instance. A later launch boots a fresh instance and stops the previous one of that app. Argent runs swipe as a mouse drag. pinch and rotate are rejected
Vega	Use tool: tv-remote and tool: keyboard. The touch directives are unsupported
HarmonyOS	Pass the device with --device harmony-<connectKey>. Argent never detects a HarmonyOS device, and --platform and when: { platform } do not accept harmony. The runner has no UI tree, so each step with a selector fails. A coordinate tap, a swipe or pinch without a selector, launch with an app id or native, snapshot without cropOn, wait and tool steps run. long-press and rotate are rejected

A fragment whose run: chain reaches a launch cannot declare executionPrerequisite. Parsing accepts the file, the run rejects it.

A tool: e2e-execute step runs a different e2e test as a nested run. run: steps and nested runs make one chain of e2e tests. Argent stops the run at a step that closes a cycle in this chain, for example an e2e test that runs itself through tool: e2e-execute. The reason of the step contains cyclic e2e test reference: and the chain of e2e tests. Argent also stops the run at a step that makes the chain longer than 20 e2e tests. The reason contains max run depth exceeded. These rules apply with and without a link.

Snapshots

- snapshot: checkout-summary
- snapshot: { name: price-card, cropOn: { id: price-card }, maxMismatch: 0.2 }

A snapshot compares the current screen, or the frame of cropOn, with a stored baseline. A missing baseline fails the step. A mismatch above maxMismatch percent fails the step. The default maxMismatch is 0.5. A cropOn element whose size changed fails the step.

Baselines live in __baselines__/<test>/ beside the e2e test file, for example .argent/e2e/__baselines__/checkout/. Argent keys a baseline by platform and capture geometry, plus the selector for cropOn. A run on a remote simulator uses ios in the key. It uses the same baseline as a local iOS run with the same capture geometry. Run argent e2e run <name> --update-baselines to write the baselines from a known-good state. A run on a remote simulator with --update-baselines rewrites the same baseline file that a local run uses. The step reason then says that a remote simulator wrote the baseline. Review each baseline before you commit it.

An e2e test that a tool: e2e-execute step runs uses its own baselines, in __baselines__/<test>/ beside the real file of that e2e test. A run with --update-baselines also writes the baselines of each e2e test that a tool: e2e-execute step runs. When the step sets updateBaselines, Argent uses the value of the step.

Over a link, the baselines stay in the same directory in the project on the client. This is also true for the baselines of an e2e test that a tool: e2e-execute step runs. With the call, the client sends the baselines that the run compares. With --update-baselines, the client writes each new baseline when the run ends. See E2E tests over a link.

Use a snapshot for layout, color, spacing, typography, clipping and icons. Do not use a snapshot as the only proof of navigation, data or network behavior. Avoid timestamps, live data, ads and animation in the captured region. The runner pins the mobile status bar during a visual run.

Local scripts

- script: { path: ../../scripts/seed-order.mjs }
- script: { path: ../../scripts/seed-order.mjs, timeout: 60000 }

Use a script step for setup or cleanup that device steps cannot do. The script must be a local .mjs file.

Over a link, the tool-server rejects an e2e test with a script step, and the e2e-add-script tool refuses to record one. Record and replay an e2e test with a script step without a link. An argent client that is older than the tool-server can record a script step when the computer of the tool-server has the project_root path, for example over a link to 127.0.0.1. A replay over the link then rejects the e2e test. See E2E tests over a link.

A script step needs no device. An e2e test of script steps alone runs with no simulator, emulator or browser, and the report names no device. A run step or a when step beside it makes the e2e test resolve one again, so a script inside a when: { platform: ios } block needs a booted iOS device.

The value is always a map. Parsing rejects a bare script: scripts/seed.mjs.

Key	Required	Description
path	yes	An .mjs file, relative to the e2e test file that holds the step. Write the extension in lowercase, and match the letter case on disk
timeout	no	The time limit of the step in milliseconds. The minimum is 100, and the default is 30000

Argent runs the script with the project root as the working directory. Argent gives the script an allowlist of environment names. Argent copies each value from the tool-server, not from the shell that starts the run. The tool-server is a long-lived process, so the values are the ones it held when it started. A variable you export now reaches a script only after you restart the tool-server. See Environment variables. The allowlist holds these groups:

• PATH, HOME and the equivalent Windows names
• The identity, shell, locale, terminal, temporary-directory and cache names of the host
• The Windows platform names
• The proxy names and the TLS certificate names
• The Node, npm, Android, Java and Apple toolchain names
• CI, SSH_AUTH_SOCK, and each name that starts with npm_config_

Argent removes each other name. Argent removes its own token, its own port and each ARGENT_SECRET_ value. Argent does not read a project .env file, so a name such as NODE_ENV or DATABASE_URL is absent. There is no env key. Let the script read a secret or a URL from a file.

Three entries in the list give access to a credential. Each entry holds the value of the tool-server. SSH_AUTH_SOCK reaches the SSH agent of the session that started the tool-server. An npm_config_ name can hold a registry token. HOME is the home directory of that session, so the script reads ~/.npmrc, ~/.aws/credentials and ~/.argent/ from there. Run a script only when you trust it.

PATH is the value of the tool-server too. A script that runs psql, docker or pnpm finds the binary on that PATH. An editor that starts the tool-server gives it the PATH of the editor. The script also runs under the Node binary of the tool-server, not the binary on the PATH of your shell.

A failed step stops the e2e test. The verdict names the side that caused the failure.

Verdict	Cause	Examples
Failed	The script	A missing file, a load error, a thrown error, or a non-zero exit code
Errored	The host	A time limit, a heap limit, a signal, a process that did not start, a full queue, a cancelled run, or a mis-cased filename

A step that a thrown error stopped reports the message of the error and the frames of the script. Argent gives each frame a path relative to the project root. Argent removes the frames of Node and of its own runner, and keeps at most six.

The split holds only when the timeout is generous. Argent needs about a second to tell a script that never settles from a script the host stopped. Below that, the same defect of the script becomes a time limit, and the step is errored. Give a script a second or more before you read errored as a fact about the machine.

Parsing rejects a timeout below 100 milliseconds. The step starts a Node process before the script runs, and that start alone costs tens of milliseconds. A smaller limit ends the step at the time limit, or makes the result depend on the load of the host.

timeout bounds the run of the script, not the duration of the step. The tool-server runs a limited number of scripts at the same time, and it takes the number from the CPU count of the host. A step waits for a free slot before the script starts. The step reports the wait in its reason, and the wait can be longer than the timeout of the step. A step that finds the queue full is an errored step. Every project on the host shares one queue, because every project shares one tool-server.

scripts.maxTimeoutMs caps the timeout of each step, and scripts.heapLimitMb sets the heap of the process. Argent reads both keys from the global configuration file of the tool-server's own home directory. See Available keys.

On iOS, Android and Vega, a script step above the launch step runs before Argent restarts the app.

On Chromium, Argent boots the app before step 1 only when the run has no device and the leading launch names a Chromium app alone. A script step above that launch then runs while the app is up. In every other case Argent boots nothing before step 1: the script step runs first, and the launch step attaches to a running instance. Two cases reach it — a run you pin with --device, and a launch that names more than one platform when you pass no --platform.

The argent e2e command

argent e2e run replays an e2e test without an agent and exits non-zero on failure. The command sends the e2e test to the tool-server that the CLI uses. See E2E tests over a link.

Command	Description
argent e2e run <name>	Run .argent/e2e/<name>.yaml, or .argent/flows/<name>.yaml when the first file does not exist
argent e2e run <path>.yaml	Run any e2e test file. The path must not contain ..
argent e2e run <dir>	Run every e2e test in the directory, one after the other
argent e2e list	List the e2e test files in .argent/e2e and .argent/flows

Option	Description
--device <id>	The device to run on. Argent detects the device when you omit the option. A physical iPhone is never detected; name it here
--platform <p>	ios, android, chromium, vega or ios-remote. Narrows the detection. ios still never picks a physical iPhone, and never picks a remote simulator either
--update-baselines	Write the snapshot baselines instead of comparing them. Also writes the baselines of the e2e tests that tool: e2e-execute steps run, unless the step sets updateBaselines
--output <dir>	Write the failed baseline, current and diff images and the screenshots of failed steps to <dir>/<test>/, for a CI artifact upload
-r, --recursive	With a directory, also run the e2e tests in subdirectories. Dot-directories and node_modules are skipped
--json	Print the JSON report of the e2e test, or the JSON aggregate of the directory
--json-stream	Print one JSON record per line: each step, then the result. One e2e test, and not with --json
--	End of options, for an e2e test name that starts with -

argent e2e run checkout --platform ios
argent e2e run .argent/e2e/checkout.yaml --output e2e-artifacts --json
argent e2e run ~/shared-e2e/checkout.yaml --device <UDID> --update-baselines
argent e2e run .argent/e2e --recursive

The file name without .yaml names the report and the artifacts. It contains only letters, numbers, _ and -.

When a step fails, Argent takes a screenshot of the device. Argent shows its path under the step as screen. Argent takes no screenshot when an earlier step or the failed step types a {{secret:…}} value, because the value can be on the screen. With --output, Argent writes the screenshot to <dir>/<test>/step-<n>-screen.png, where <n> is the number of the step in the report.

Each step line shows the time of the step before the reason, for example (1.2s). A step that did not run shows no time. The time of a when or run step does not include the steps in its block or fragment. The last line of an e2e test run shows the time of the run. The summary of a directory run shows the time of all e2e tests, for example (1m 32s).

A directory run prints a numbered header for each e2e test. Under the header Argent prints each step of that e2e test that failed or carries a warning, with its artifact paths. Argent then prints one outcome line. After the last e2e test, Argent prints a Failed e2e tests section when one or more e2e tests failed. Argent prints a summary of all e2e tests at the end.

An e2e test that fails its steps lets the directory run continue. So does an e2e test that the tool-server rejects up front, such as an invalid file or a device that Argent cannot resolve. So does an e2e test when the upload of one of its fragments fails. A transport failure ends the directory run. So does a rejection that the tool-server does not mark as a validation error, and a reply that is not a report. Argent counts the e2e tests after the failure as skipped.

The outcome line of an e2e test that produced a report gives the result, the step counts and the time of that e2e test. The outcome line of an e2e test that produced no report states the reason. A single e2e test run prints the same reason under the name of the e2e test.

Outcome line	Meaning
not run (invalid e2e test)	The tool-server rejected the e2e test file or one of its steps
not run (no device resolved)	Argent did not resolve one device for the e2e test
not run (rejected)	The tool-server rejected the e2e test for another reason
not run (upload failed)	The upload of a run: fragment failed, so the CLI did not send the e2e test
did not finish (run error)	The run failed before it produced a report
did not finish (no run report)	The tool-server answered without a run report
not run (batch stopped)	An earlier failure ended the directory run. Directory runs only
not run (tool-server unreachable)	The CLI did not connect to the tool-server that argent link or ARGENT_TOOLS_URL names

The Failed e2e tests section has one entry for each e2e test that failed. Each entry holds these items:

• The e2e test path.
• The first step that failed or errored, with its number. An e2e test without a report gives its outcome line instead.
• The reason of that step, or the error message of an e2e test without a report.
• A re-run: command that runs only that e2e test. The command uses the same --device, --platform and --update-baselines options as the directory run. If the directory run has --output, the command writes the images of that e2e test to the same directory as the directory run.

The section does not include the e2e tests that Argent skipped.

This is the end of the output of argent e2e run ./e2e --recursive --platform chromium:

Failed e2e tests (2)

  ✗ a-login.yaml › step 3 assert visible "Dashboard"
    no element matched selector text="Dashboard"
    re-run: argent e2e run e2e/a-login.yaml --platform chromium

  ✗ sub/c-search.yaml › not run (invalid e2e test)
    [Tool:e2e-execute] Unrecognized e2e test entry (unrecognized step kind): {"swipe-left":{"text":"Results"}}
    re-run: argent e2e run e2e/sub/c-search.yaml --platform chromium

FAIL — 4 e2e tests: 2 passed, 2 failed, 0 skipped (10.2s)

When a single e2e test run fails its steps, Argent prints the first step that failed or errored, and its reason, again above the last line.

--json and --json-stream print no header, no outcome line, no Failed e2e tests section and no summary.

A directory run with --json writes one document. The document holds ok, total, passed, failed, skipped, durationMs and a flows array. durationMs is the time of the directory run in milliseconds. The array holds one entry for each e2e test. Each entry holds the path and the status. An entry holds report when the e2e test produced one, whether it passed or failed its steps. An entry holds error instead when the tool call produced no report: a rejected run, a run error, or a reply that is not a report. Such an entry also holds error_code and error_kind when the tool-server or the CLI sets them. An entry with the status skip holds neither, because Argent made no call for that e2e test. --json-stream puts the same two fields on its error record.

A single e2e test run with --json writes the report of that e2e test. The report holds startedAt, the start of the run in epoch milliseconds, and durationMs, the time of the run in milliseconds. Each step that ran holds its own durationMs. Each progress record of --json-stream holds one step, with the same durationMs. When the run gets no report, Argent writes one JSON object on stderr instead. The object holds event and error. It also holds error_code and error_kind when the tool-server or the CLI sets them. --json-stream writes the same object on stdout.

With --json, Argent writes these objects on stderr, one on each line. Argent writes one object for each of these errors:

• an e2e test in a directory run that failed without a report
• a usage error, such as an unknown flag

A usage error prints no help text and gives exit code 2. The warnings of --output stay plain text.

Pin --platform and --device on iOS, Android and Vega. On Chromium, pass --platform chromium and omit --device. Then the runner boots the declared app path with a reproducible window size.

E2E tests over a link

argent link and the ARGENT_TOOLS_URL variable route the CLI to a tool-server. In this section, both are a link. Over a link, argent e2e run uploads these files with the call:

• the e2e test file
• each run: fragment and each nested e2e test that the e2e test reaches. A nested e2e test is an e2e test that a tool: e2e-execute step runs
• the snapshot baselines of each run
• the file arguments of the tool: steps

The other files stay on the client.

A file argument is an argument that a tool: step reads as one file, named by an absolute path that ends in .png or .yaml. The case of the letters does not matter. An example is baselinePath of screenshot-diff.

E2E test content	Over a link
Directives that drive the device, echo, when	Runs
run:, snapshot:, a tool: step with a file argument	Runs. When the CLI or the tool-server is an older version, the tool-server rejects the e2e test before the first step
script:	The tool-server rejects the e2e test before the first step
A tool: step that names a file by a relative path, or a file that does not end in .png or .yaml	The tool-server rejects the e2e test before the first step. Use an absolute path to a .png or .yaml file
A tool: step that takes a directory, an app or an output directory, for example gather-workspace-data, reinstall-app or screenshot-diff with outputDir	The tool-server rejects the e2e test before the first step
A tool: step that builds a file path from two or more arguments, for example e2e-read-prerequisite with name	The tool-server rejects the e2e test before the first step
tool: e2e-execute with name and an absolute project_root without a .. segment	Runs. The CLI also uploads the e2e test that the step names, with its own files. The same rules apply to that e2e test. When the CLI or the tool-server is an older version, the tool-server rejects the e2e test before the first step
tool: e2e-execute with flow_path, or with name and a project_root that is relative or has a .. segment	The tool-server rejects the e2e test before the first step
A tool: step that records an e2e test, for example e2e-add-step	The tool-server rejects the e2e test before the first step
A relative launch: { chromium: <path> }	The launch fails. Use an absolute path on the tool-server

The rejections in the table apply to the e2e test file and to each fragment and nested e2e test that the CLI uploads. For each rejection, the error lists the steps that caused it. When an older CLI sends an e2e test with run: steps, snapshot: steps, nested e2e tests or file arguments, the error also tells you to update the argent CLI. For a tool: step that names a file by a relative path, the error tells you to use an absolute path. For a file that does not end in .png or .yaml, the error says that a tool: step can name only such a file.

The CLI uploads each fragment that a run: step names, and each nested e2e test that a tool: e2e-execute step names with name. This includes the steps in fragments, in nested e2e tests and in when: blocks. The CLI uploads a fragment or a nested e2e test only when its real location is under one of these directories:

• the project root, which is the working directory of the CLI
• the project of an e2e test saved under <project>/.argent/e2e/ or <project>/.argent/flows/
• the .argent/e2e and .argent/flows directories of the project root
• the directory of the e2e test file, and the directory of its real file when the e2e test file is a symlink

The CLI does not upload these fragments and nested e2e tests:

• a file outside these directories
• a .yaml name that links to a file that is not a YAML file
• a directory, or a file that the CLI cannot read
• a file larger than 32 MiB

The tool-server then rejects the e2e test before the first step. The error gives the reason for each such file.

A fragment or a nested e2e test that does not exist in these directories does not stop the e2e test before the first step. Its step reports error when the step runs. A when: block that does not run does not need its files.

For an e2e test with snapshot: steps, the CLI also uploads baselines from the __baselines__/<test>/ directory that a run without a link uses. A snapshot: step in a fragment uses the baselines of the e2e test file, as in a run without a link. A nested e2e test is a run of its own, with the baselines beside its own real file. It updates its baselines as Snapshots describes. The upload depends on the run:

Run	The CLI uploads
A run that compares baselines	The baselines of the snapshot: steps in the e2e test and its fragments, also with cropOn. With --platform, or a Chromium device in --device, only the baselines of that platform
A run that updates baselines	The name of each baseline, without its content

When a run that compares and a run that updates use the same directory, the CLI uploads the content of each baseline that the run that compares reads. It uploads the name of each other baseline.

When a run updates baselines, the tool-server returns each new baseline in the result, also the baselines of nested runs. The CLI writes each one into the directory of its run after the run, and into no other directory. When the CLI cannot write a baseline, it prints the path and the reason. The run then fails with exit code 1. When the connection closes before the result, the CLI writes no baseline.

The CLI does not upload a baseline that links out of the directories above, or a .png name that links to a file that is not a PNG file. A snapshot: step that reads or writes such a baseline reports error. When the __baselines__/<test>/ directory itself links out of these directories, the CLI uploads no baseline and writes none.

The CLI also uploads each file argument of the tool: steps in the e2e test, its fragments and its nested e2e tests, in each when: block. Only an argument that the tool declares as a file counts: an absolute .png path that a keyboard step types is not uploaded. When a step reads a baseline that an earlier step of the same call wrote, the step gets the new baseline. This is also true in a nested run.

The CLI does not upload these file arguments:

• a file argument whose real location is outside the directories above
• a name that links to a file that does not end in .png or .yaml
• a file argument that is not on the client

Such a file argument does not stop the e2e test before the first step. Its tool: step reports error when it runs. The tool-server never uses a file at the same path on its own computer.

The CLI puts fragments, nested e2e tests, baselines and file arguments in the call until they are 256 KiB in total. The call carries the e2e test file and these files as base64 text, which is about 4/3 of their size. The CLI sends each other file in a separate request to POST /upload. Thus a reverse proxy in front of the tool-server must accept these request bodies:

• on POST /tools/e2e-execute, the e2e test file and about 350 KiB more
• on POST /upload, one file at a time

When the proxy refuses a file with the status 413, the e2e test does not run. The error gives the body size that the proxy must accept on POST /upload. When the proxy refuses the call with the status 413, the error gives only the status. The Tool-server API reference documents the upload.

Set ARGENT_E2E_FILES_LOG=1 to see what the CLI does with each fragment, nested e2e test, baseline and file argument. The CLI then prints one [flow-files] line on stderr for each file before the call, and one line for each baseline that it writes. With --json, each line is a warning object: { "event": "warning", "warning": <text> }.

The tool-server runs the uploaded copy even when it has a file at the same path. This is also true over a link to 127.0.0.1. Only the copy from the client is certain to be the latest. Without a link, the tool-server reads the file in place, and all step kinds run.

A rejected e2e test gives exit code 1. With --json, the CLI writes one JSON object for each rejected e2e test on stderr: { "event": "error", "error": <message>, "error_code": "FLOW_FILE_INVALID", "error_kind": "validation" }. With --json-stream, the same object is the last line on stdout. A directory run continues with the next e2e test and keeps the rejected e2e test in its flows array.

When the upload of a fragment to POST /upload fails, the e2e test does not run. The CLI prints the reason. The exit code is 1. The JSON object holds "error_code": "FILE_INPUT_UPLOAD_FAILED" and "error_kind": "validation". A directory run continues with the next e2e test.

When the CLI cannot connect to the tool-server that a link names, the e2e test does not run. The CLI prints the address of that tool-server, the setting that names it and the reason, for example connection refused. The exit code is 2. The JSON object holds "error_code": "TOOL_SERVER_UNREACHABLE" and "error_kind": "network". A directory run stops at that e2e test and marks the e2e tests after it as skipped.

Remote runs

The e2e-execute tool takes one e2e test source: name for a saved e2e test, or flow_path for any file. Without a link, the tool-server reads the file in place, and all step kinds run.

Over a link, both sources send these files with the call, and the tool-server runs those copies:

• the e2e test file
• each run: fragment that the e2e test reaches
• each nested e2e test, that is an e2e test that a tool: e2e-execute step names with name and an absolute project_root
• the snapshot baselines of each run
• the file arguments of the tool: steps

The client sends these files with the rules of E2E tests over a link. The project_root argument is the project root. When a run of the call updates baselines, the client writes each new baseline into the __baselines__/<test>/ directory of its e2e test when the run ends. When the client cannot write a baseline, the report shows baseline not written with the path and the reason. When the call sets updateBaselines, a tool: e2e-execute step in the e2e test also updates the baselines of its e2e test, unless the step sets updateBaselines itself.

Before the first step, Argent checks the e2e test and each fragment and nested e2e test that the client sends, at any depth. Argent rejects these:

• each step that the table in E2E tests over a link rejects
• a fragment or a nested e2e test that the client does not send, except one that does not exist

The error lists each such step. A fragment, a nested e2e test or a file argument that does not exist fails its step when that step runs. See E2E tests over a link for the argent e2e run command.

YAML safety

Quote a string that contains #, : or quotes. Quote a number, true or false in a text slot. Use single quotes for a regex with backslashes. Parsing rejects an unknown directive, an invalid selector, an invalid regex, else, an unsupported option, and a standalone e2e test that declares executionPrerequisite.