E2E YAML reference
An e2e test is a YAML file in .argent/e2e/<name>.yaml. The agent records the file with the e2e tools. You can also write or edit the file by hand. This page lists the file shape, the selectors, the directives and the argent e2e command. For the concept, see Record and replay e2e tests.
Earlier versions of Argent called this feature Argent flows. The old names still work:
| Old name | New name |
|---|---|
flow-* tools | e2e-* tools |
tool: flow-execute step | tool: e2e-execute step |
argent flow command | argent e2e command |
.argent/flows/ directory | .argent/e2e/ directory |
ARGENT_FLOW_FILES_LOG | ARGENT_E2E_FILES_LOG |
flow_path param | e2e_test_path param |
flow_file param | e2e_test_file param |
flow result field | e2eTest result field |
flowFile result field | e2eTestFile result field |
flows array of --json | e2eTests array of --json |
.argent-flow-source file | .argent-e2e-source file |
Argent reads an e2e test from .argent/e2e/ first, then from .argent/flows/. When both directories hold an e2e test with the same name, Argent runs the file in .argent/e2e/ and prints a warning that names the other file. Argent writes a new e2e test to .argent/e2e/. Argent does not move the existing files. When you move an e2e test by hand, also move its __baselines__/<test>/ directory.
The results of the e2e tools and the --json document of argent e2e run hold each old field beside the new field, with the same value. An older tool-server sends only the old field.
argent e2e run --output marks each test directory that it writes with a .argent-e2e-source file. The file names the e2e test that wrote the directory. The CLI also reads the .argent-flow-source file that an older CLI wrote.
File shape
steps:
- launch: com.example.app
- await: { visible: { id: home-screen } }
- await: { idle: true }
The file has four top-level keys:
| Key | Required | Description |
|---|---|---|
steps | yes | The list of steps. Argent runs the steps in order |
executionPrerequisite | no | One sentence that names the start state. Only a fragment can declare it |
requires | no | The targets that the e2e test supports. See Requirements |
version | no | The file format version. The value is 1. Argent writes it. A file without version is version 1 |
Argent refuses a file with a version that is higher than the version it reads. Update Argent to run that file.
Argent releases from before the version key refuse a file that has the key. To run the file on such a release, delete the version line.
An e2e test has one of two shapes:
| Shape | First step that is not echo: or script: | Start state |
|---|---|---|
| Standalone e2e test | launch: | Argent starts the app from scratch |
| Fragment | Any other step | The app is already in the state that the e2e test needs |
Argent skips leading echo and script steps when it classifies an e2e test. An e2e test that opens with an echo step and then a launch: step is still standalone. An e2e test that opens with a script step and then a launch: step is also standalone. Put the named start state of a standalone e2e test in a leading echo step.
A file does not store a device id. The runner selects the device. A launch: restarts the process but does not clear the app data, the account or the backend data.
When a step fails, Argent stops the e2e test and reports the later steps as skipped.
Requirements
The requires block names the targets that an e2e test supports. An e2e test without the block runs on all targets.
requires:
platform: [ios, android]
runtimeKind: tv
steps:
- launch: com.example.tvapp
| Key | Value |
|---|---|
platform | ios, android, chromium, vega or macos, or a list of them. ios also covers a remote simulator |
runtimeKind | tv for Apple TV, Android TV and Fire TV. mobile for all other targets except a Mac app, which satisfies neither |
The block must have one key or both keys. A target must satisfy all keys in the block. Parsing rejects an unknown key and an unknown value.
The requirements change how Argent selects the device:
- Argent selects and checks the device before it asks for the execution prerequisite. When the block excludes the target, the run fails with
FLOW_REQUIREMENTS_UNMET. - Without
--device, Argent selects only from the booted devices that satisfy the requirements. An iOS-only e2e test selects the simulator when an Android emulator is also booted. - When devices are booted but none satisfies the requirements, the run fails with
FLOW_REQUIREMENTS_UNMET. A directory run skips that e2e test. See Theargent e2ecommand. - When
runtimeKindis set and Argent cannot read the runtime kind of a device that it must check, the run fails withFLOW_DEVICE_RESOLUTION. A directory run does not skip that e2e test. - An e2e test that has no step that acts on a device runs without a device. Argent checks its requirements only against
--platformor--device.
A run: fragment can also declare requires. When the run device does not satisfy the requirements of the fragment, the run: step fails.
e2e-start-recording replaces the whole file, including the requires block. To keep the block when you record an e2e test again, pass it in the requires argument of the tool.
Selectors
A selector finds an element on the screen. Write a selector as an explicit map:
{ id: save-button }
{ text: Save }
{ role: button }
{ id: settings-row, text: Notifications }
{ text: { matches: '^Order #\d+$' } }
| Field | Match |
|---|---|
id | Exact, case-insensitive. The test id or accessibility id |
text | Case-insensitive substring, or { matches: <regex> } |
role | Case-insensitive substring of the element role |
any | Any element. Takes only true. Pair it with a relational scope |
All fields in a selector must match. An unqualified Android id also matches its qualified resource id. identifier is an alias of id. Use single quotes for a regex with a backslash. any: true is the only selector without its own id, text or role, so it needs a relational scope, for example { any: true, after: { text: Danger zone } }.
A bare string, for example tap: Save, is a loose selector. It tries id first, then text. Prefer the map form.
When several elements match:
| Directive type | Chosen element |
|---|---|
Action (tap, long-press, swipe, type, scroll-to, pinch, rotate) | The most specific visible match: exact text or id, then the smallest frame, then reading order |
Condition (await, assert) | exists and visible hold when any match qualifies. hidden holds when no match qualifies. text reads the first visible match |
Relational scopes
A selector can scope its matches by the frames of other elements:
| Key | Meaning |
|---|---|
within | The target is inside the frame of the anchor |
after | The target follows the anchor in reading order |
next | The target is the nearest match that follows the anchor |
- tap: { text: Delete, within: { id: profile-card } }
- assert: { visible: { role: Button, after: { text: Danger zone } } }
- tap: { role: Switch, next: { text: Wi-Fi } }
within uses the visual frame, not the source tree. Reading order is top to bottom, left to right, as the user sees the UI. This is also true for a landscape UI, for example on a rotated device or an unfolded foldable. after, next, any: true and a text condition use this reading order. Scopes nest, with at most six scope keys in one selector. The await-ui-element tool does not support scopes.
Runner tree and discovery tree
The runner resolves a selector against a different tree than the describe tool:
| Platform | Runner tree | describe tree |
|---|---|---|
| iOS simulator | Native UIView hierarchy | Accessibility tree |
| Physical iPhone | Accessibility snapshot of describe | Same snapshot |
| Android | Full accessibility hierarchy | Trimmed interactable nodes |
| Chromium | DOM nodes with an id, label, value or handler | Full DOM |
| Vega | Toolkit page source | Same source |
| Mac app | The elements of the app tree that have an id, a label, a value, a clickable role or focus | The full app tree, nested |
| HarmonyOS | None | uitest layout dump |
On an iOS simulator and Android, an id absent from describe can still resolve in an e2e test. On Chromium, an element absent from describe cannot resolve. On an iOS simulator, do not copy a role from describe into a selector: the runner derives the role from the view class, describe from the accessibility traits. On a physical iPhone, the runner and describe read one tree: copy ids and roles from describe.
Selectors on Mac apps
Copy control identifiers and AX roles from describe. The runner uses the same app tree.
- Select controls inside a window or dialog. Windows, sheets, alerts, popovers, and scroll bars cannot be selector targets, even with an identifier. To wait for a sheet, select a button inside it.
- Use identifiers for password fields. Password fields expose no text value. An
idorroleselector can match them. - Expect foreground controls to take priority. When several elements match, menus and combo box lists come first, then the uppermost dialog, then the window. Text checks prefer a matching paragraph over its containing editor.
- Keep recorded coordinates when position matters. Sliders, disclosure triangles, and canvas clicks can behave differently at the element's center. Argent preserves coordinates for these clicks when needed.
Directives
Each step contains one directive. A failed directive stops the e2e test.
| Directive | Shape | Description |
|---|---|---|
launch | <app id> or { native, ios, android, vega, chromium, macos } | Terminate and start the app, then wait until the app is ready |
tap | <selector>, { on: <selector>, times, button } or { x, y } | Tap an element. times: 2 double-taps. x, y are normalized coordinates |
long-press | <selector> or { on: <selector>, duration } | Press and hold. duration in milliseconds |
swipe | <direction> or { from, direction, to, by, momentum, duration } | Move one finger across the screen. direction is the travel of the finger. See Swipe |
type | { into: <selector>, text, submit } | Focus the element and type. Presses Enter unless submit: false |
scroll-to | { target: <selector>, direction, within } | Scroll until the target is visible. direction is down (default), up, left or right |
pinch | { on: <selector>, scale } | Pinch around the element, or the screen center when on is absent. scale > 1 zooms in |
rotate | { on: <selector>, by } | Two-finger rotation in degrees, clockwise positive. This is a gesture, not the device orientation |
fold | <posture>, <angle> or { posture } / { angle } | Fold or unfold a foldable device. See Foldable devices |
await | { <condition>, timeout } or { idle: true, stableFor, timeout } | Wait for a condition. Default timeout 7500 ms |
assert | { <condition> } | Check a condition now, with a fixed 1000 ms grace. Rejects timeout |
wait | <milliseconds> | Pause for a fixed time |
snapshot | <name> or { name, maxMismatch, cropOn } | Compare a screenshot with a stored baseline. See Snapshots |
run | <path> | Run another e2e test file inline. See Composition |
script | { path, timeout } | Run a local JavaScript or TypeScript file in a new Node process. See Local scripts |
when | { <condition> } or { platform }, with steps: | Run the nested steps only when the condition holds. See Optional steps |
echo | <message> | Print a message in the report |
tool | <tool name>, with args: and optional delayMs: | Call any Argent tool with the given arguments |
tap, type and long-press do not scroll. Add scroll-to before them when the target can be off-screen.
The button of a tap is left (the default) or right. button: right is a right-click, for example to open a context menu. Argent supports it only on a Mac app. On other platforms, the step fails before the runner reads the tree. To open a context menu on iOS or Android, use long-press. A recorded gesture-tap with button: "right" becomes a tap with button: right.
A direction in swipe and scroll-to is the direction that the user sees. This is also true for a landscape UI, for example on a rotated device or an unfolded foldable. Coordinates in tap: { x, y } and in the from, to and by of swipe use the space of the describe frames. Argent does not turn them.
A type step can contain a {{secret:NAME}} placeholder. The tool-server fills the value at run time. The report of the step shows the placeholder, not the value. See Secrets and Typed secrets in a report.
A gesture that resolves no selector passes with a warning when the runner cannot read the UI tree. This covers a coordinate tap, a swipe with no selector at either end, and a pinch or rotate without on. The warning says that Argent sent the gesture, not that the gesture reached the element.
Typed secrets in a report
A text field that is not secure shows a typed secret on the screen. The reason of a failed text check quotes the text of its element. Argent keeps a typed secret out of the quoted text with two rules.
The first rule applies on each platform. Argent knows each value that a type step or the keyboard, fill or paste tool typed on a device. In the quoted text, Argent replaces a value with its placeholder when the screen shows the value exactly as typed. The comparison is case-sensitive:
element matched id="status" but its text was "Signed in with {{secret:API_TOKEN}}" (wanted to contain "Saved")
The second rule applies on an iOS simulator. A text field can change a typed value. For example, the field can capitalize, correct or format the value. Thus, after a tool typed a secret on an iOS simulator, the quoted text does not contain the text of a text field. The quoted text contains a note instead:
element matched id="token" but its text was "Token [text withheld: a secret was typed on this device]" (wanted to contain "Saved")
The second rule applies to each text field on that simulator. It also applies to an empty field, which reads as its placeholder. The label of the field and the text of other elements stay in the quoted text. The check reads the text of the field as before. Only the quoted text changes.
Argent keeps the values in memory, for each device, until the tool-server stops. Thus the two rules also apply in a later e2e test of a directory run, in a nested e2e-execute run and in the run around it. The rules also apply after the app starts again.
Argent changes only the text that the reason quotes from the screen. The other words of the reason and the text that the check expects do not change. The first rule also applies to the note of a failed text check of the await-ui-element tool.
A report can show a typed secret in these cases:
- An element that is not a text field shows a text that the app derives from the secret, for example a capitalized name.
- A
tool:step reads the screen, for example withdescribe. The result of the step contains the text of the screen. - On a physical iPhone, Android, Chromium or a Mac app, a text field shows the secret in a changed form. The second rule applies only on an iOS simulator.
- A later e2e test of a directory run fails. That e2e test typed no secret, so Argent takes the screenshot of the failed step.
- A different tool-server typed the secret, or no Argent tool typed it.
Launch map
Use the map form for an e2e test that runs on more than one platform:
- launch: { native: com.acme.app, chromium: ../../app }
- launch: { ios: com.acme.app, android: com.acme.app.android, chromium: ../../app }
- launch: { ios: { app: com.acme.app, args: [-FeatureFlag, YES] }, android: com.acme.app }
native is one id for iOS, Android, Vega and macOS. A per-platform key replaces it for that platform. macos takes the bundle id of a Mac app. chromium accepts a relative or absolute app path, or { path, args }. ios and macos accept a bundle id, or { app, args }. On an iOS simulator, a physical iPhone or a Mac app, Argent passes args to the app process at launch. A Mac app gets no arguments when the launch step has no args. Each arg must be a string, so quote a number or boolean: "5", not 5. A launch that has no id for the platform of the run is an error.
A run on a remote simulator uses the ios entry. If the map has no ios entry, the run uses the native entry.
When the agent records restart-app with launchArgs on iOS, Argent saves a launch step with a native id and an ios: { app, args } entry.
An Android app that starts from a non-launcher activity has no launch: form. Record restart-app with activity as a tool step and keep the e2e test as a fragment.
Launch a Mac app
Use the bundle identifier in launch.macos. Start your intended .app through Argent once before the first test. Later launches reuse that build path. Without a saved path, Argent uses the newest matching build found by Spotlight.
For example, save this test as .argent/e2e/open-document.yaml. Replace the bundle identifier, control identifier, and file path with your app's values:
steps:
- launch: { macos: com.example.notes }
- tap: { id: open-document }
- tool: file-panel
args: { action: choose, path: ~/Documents/note.txt }
Run the test with:
argent e2e run open-document --platform macos
The launch step restarts the app using the same rules as restart-app. Save prompts must be answered before a restart can proceed. The macos-apps flag must remain enabled.
| How you select the app | Behavior |
|---|---|
--device macos-app-<bundle id> | Uses that app. |
--platform macos with a leading launch | Starts the app named by the launch step. |
A launch with only the macos key, or macOS-only requires with a leading launch | Selects the named app without needing --platform. |
--platform macos without a launch step | Uses the only running Mac app. With several running apps, pass --device. |
Without a macOS selection, automatic device detection excludes Mac apps. Recorded tests with launch: <bundle id> need --platform macos or --device.
A Mac launch accepts { app, args } for launch arguments. Without args, the app receives no arguments. When recording a restart with arguments, Argent saves them under macos.
A launch from another git worktree produces a warning, including builds in that worktree's Xcode DerivedData folder. Start the intended build before retrying. Worktree warnings are unavailable for tests over a link.
Use tool steps for file-panel and menu-bar. For prepared file answers, set panelKind: open or panelKind: save explicitly. Otherwise, a save can create the destination and cause a later replay to infer an open answer. Recording fills in panelKind automatically.
Swipe
- swipe: left
- swipe: { from: { id: story-card }, direction: left }
- swipe: { by: { y: -0.4 }, duration: 600 }
- swipe: { from: { id: drag-handle }, to: { id: drop-zone }, momentum: false }
A swipe moves one finger across the screen. direction is the travel of the finger as the user sees it, not the travel of the content. To bring an element on screen use scroll-to not swipe.
| Option | Value |
|---|---|
from | The start point. A selector or { x, y }. Argent selects the start when from is absent |
direction | up, down, left or right. Argent moves the finger a preset distance. With from, Argent clamps the travel on screen |
to | The end point. A selector or { x, y } |
by | A signed delta { x, y } in screen fractions. Argent delivers the exact delta. With from, a delta that leaves the screen fails the step |
momentum | false removes the fling at the default duration. Default true |
duration | The travel time in milliseconds. Default 300, minimum 150, maximum 10000 |
A step takes exactly one of direction, to and by. Each travel must cover at least 0.03 of the screen. Parsing rejects a shorter by. A shorter to fails the step during the run.
On Chromium and on a Mac app, Argent runs a swipe as a mouse drag. On Vega, a swipe fails like the other touch directives.
Foldable devices
- fold: open
- fold: 120
- fold: { posture: closed }
A fold step moves the hinge of a foldable iOS simulator, for example the iPhone Duo, or of a foldable Android emulator. The step takes a posture (closed, half-open or open) or an angle from 0 to 180 degrees. The step waits until the device accepts input again. When you record an e2e test, Argent saves a call to the fold tool as a fold step.
After a fold step:
- The screen size and the coordinates change with the panel. Later selectors resolve against a new UI tree.
- A
snapshotbaseline is valid only for the posture that made it. Use a different snapshot name for each posture. - On iOS, unfolded, the UI is landscape. Directions and reading order follow the UI, as on a rotated device.
- On iOS, a fold between two angles that are not 0 or 180 can keep the current panel. The step passes, and the report names the panel. To change panels, fold to
closedoropen. - On Android, the report names the device state that Android committed, for example
CLOSED.
Conditions
- await: { visible: { id: settings-screen } }
- await: { hidden: { id: loading-spinner }, timeout: 15000 }
- assert: { exists: { id: notifications-toggle } }
- assert: { text: { in: { id: preference-status }, equals: Enabled } }
- assert: { text: { in: { id: result-count }, matches: '^\d+ results$' } }
| Condition | Holds when |
|---|---|
exists | An element matches the selector |
visible | A visible element matches the selector |
hidden | No visible element matches the selector |
text | The text of the element in in satisfies exactly one comparator |
The text comparators:
| Comparator | Match |
|---|---|
contains | Case-insensitive substring |
equals | Case-insensitive full match |
matches | Case-sensitive JavaScript regex |
matches reads an element with no text as an empty string. Use '^$' to check that a field is empty, or '^(?!.*@)' to check that it doesn't contain @. A pattern that can match an empty string, such as '(Saved)?', passes on an element with no text. An iOS text field is the exception: matches never reads an iOS text field as empty. On a Mac app, each text comparator reads only the text in a text field. An empty field reads as empty, also when its placeholder is its label.
On an iOS simulator, a text check reads a text field as follows:
- A text field reads as the text that it contains.
- An empty text field reads as its placeholder.
- A text field with an accessibility label reads as the label, then a space, then the text or the placeholder. For example, an empty field with the label
Emailand the placeholderRequiredreads asEmail Required. - Argent does not read the text in a secure text field.
- The runner knows a text field by the class name of its view. The name must contain
TextField,TextVieworSearchField. Argent does not read the text in a text field of a different class, for example aUITextFieldsubclass with the nameAmountField. For such a field,matches: '^$'passes also when the field contains text.
On an iOS simulator, Argent does not use the text in a text field as the selector of a recorded tap.
A negative condition passes before the element appears, for a typo, and on the wrong screen. First prove the screen and the same selector with visible. Then perform the action and check hidden.
Prove a navigation
Each screen change needs two checks:
- await: { visible: { id: profile-screen } } # identity
- await: { idle: true } # readiness
The identity selector must exist only on the destination. idle waits until the screen has content and stops changing in the UI tree and in the pixels.
- await: { idle: true, stableFor: 400, timeout: 9000 }
| Option | Default | Description |
|---|---|---|
stableFor | 250 ms | How long the screen must stay still |
timeout | 7500 ms | The budget of the whole wait |
idle never fails a run. A screen that does not settle passes with a warning. The warning names the reason: the screen kept moving, a small part kept moving, the wait ended mid-hold, the tree stayed empty, only the tree settled, or too few reads. Read the warning before you accept the step.
One outcome stops the run: the runner cannot read the UI tree at all. Argent marks the step as errored and skips the later steps.
On HarmonyOS, the runner has no UI tree, so idle cannot run. The step errors immediately, names wait: as the replacement, and Argent skips the later steps.
idle has no assert form and no when form.
Optional steps
- when: { visible: { text: Got it } }
steps:
- tap: { text: Got it }
The guard accepts one exists, visible, hidden or text condition, or { platform: ios | android | chromium | vega | macos }. A run on a remote simulator matches ios. A UI guard uses the assert grace and rejects timeout. There is no else. A skipped block reports as skipped. A failure inside an entered block is a real failure. Do not put a required check inside when:.
Composition
- run: ../shared/login.yaml
A run: path resolves against the directory of the e2e test file that contains the step. When nothing is there, the e2e test file is in .argent/e2e/ or .argent/flows/, and the path has no /, Argent looks in the other of these directories. The .yaml suffix is optional. When you record a nested e2e-execute of an e2e test that is only in the other directory, the recorder writes its path from the recording, for example run: ../flows/login.yaml.
| Platform | Behavior |
|---|---|
| iOS, Android | A nested fragment or standalone e2e test runs inline. A nested launch restarts the app |
| Chromium | Each launch step boots one instance. A later launch boots a fresh instance and stops the previous one of that app. Argent runs swipe as a mouse drag. pinch and rotate are rejected |
| macOS | A nested launch restarts the app. Argent runs swipe as a mouse drag and long-press as a press and hold. pinch and rotate are rejected |
| Vega | Use tool: tv-remote and tool: keyboard. The touch directives are unsupported |
| HarmonyOS | Pass the device with --device harmony-<connectKey>. Argent never detects a HarmonyOS device, and --platform and when: { platform } do not accept harmony. The runner has no UI tree, so each step with a selector fails. A coordinate tap, a swipe or pinch without a selector, launch with an app id or native, snapshot without cropOn, wait and tool steps run. long-press and rotate are rejected |
A fragment whose run: chain reaches a launch cannot declare executionPrerequisite. Parsing accepts the file, the run rejects it.
A tool: e2e-execute step runs a different e2e test as a nested run. run: steps and nested runs make one chain of e2e tests. Argent stops the run at a step that closes a cycle in this chain, for example an e2e test that runs itself through tool: e2e-execute. The reason of the step contains cyclic e2e test reference: and the chain of e2e tests. Argent also stops the run at a step that makes the chain longer than 20 e2e tests. The reason contains max run depth exceeded. These rules apply with and without a link.
Snapshots
- snapshot: checkout-summary
- snapshot: { name: price-card, cropOn: { id: price-card }, maxMismatch: 0.2 }
A snapshot compares the current screen, or the frame of cropOn, with a stored baseline. A missing baseline fails the step. A mismatch above maxMismatch percent fails the step. The default maxMismatch is 0.5. A cropOn element whose size changed fails the step.
Baselines live in __baselines__/<test>/ beside the e2e test file, for example .argent/e2e/__baselines__/checkout/. Argent keys a baseline by platform and capture geometry, plus the selector for cropOn. A run on a remote simulator uses ios in the key. It uses the same baseline as a local iOS run with the same capture geometry. Run argent e2e run <name> --update-baselines to write the baselines from a known-good state. A run on a remote simulator with --update-baselines rewrites the same baseline file that a local run uses. The step reason then says that a remote simulator wrote the baseline. Review each baseline before you commit it.
Mac snapshots capture the app window and attached dialogs or menus at one pixel per point. Baseline names include macos and the capture size, such as summary__macos-900x640.png. Keep the window size consistent between runs. After an intentional resize, update and review the baseline.
An e2e test that a tool: e2e-execute step runs uses its own baselines, in __baselines__/<test>/ beside the real file of that e2e test. A run with --update-baselines also writes the baselines of each e2e test that a tool: e2e-execute step runs. When the step sets updateBaselines, Argent uses the value of the step.
Over a link, the baselines stay in the same directory in the project on the client. This is also true for the baselines of an e2e test that a tool: e2e-execute step runs. With the call, the client sends the baselines that the run compares. With --update-baselines, the client writes each new baseline when the run ends. See E2E tests over a link.
Use a snapshot for layout, color, spacing, typography, clipping and icons. Do not use a snapshot as the only proof of navigation, data or network behavior. Avoid timestamps, live data, ads and animation in the captured region. The runner pins the mobile status bar during a visual run.
Local scripts
- script: { path: ../../scripts/seed-order.mjs }
- script: { path: ../../scripts/seed-order.mjs, timeout: 60000 }
Use a script step for setup or cleanup that device steps cannot do. The script must be a local file with one of these extensions:
| Extension | Language | Module type |
|---|---|---|
.mjs | JavaScript | ES module |
.mts | TypeScript | ES module |
.js | JavaScript | The package.json of the project decides |
.ts | TypeScript | The package.json of the project decides |
Parsing rejects each other extension, such as .cjs or .tsx. See Module type for a .js or .ts file, and TypeScript scripts for the limits of a .ts or .mts file.
Over a link, the tool-server rejects an e2e test with a script step, and the e2e-add-script tool refuses to record one. Record and replay an e2e test with a script step without a link. An argent client that is older than the tool-server can record a script step when the computer of the tool-server has the project_root path, for example over a link to 127.0.0.1. A replay over the link then rejects the e2e test. See E2E tests over a link.
A script step needs no device. An e2e test of script steps alone runs with no simulator, emulator or browser, and the report names no device. A run step or a when step beside it makes the e2e test resolve one again, so a script inside a when: { platform: ios } block needs a booted iOS device.
The value is always a map. Parsing rejects a bare script: scripts/seed.mjs.
| Key | Required | Description |
|---|---|---|
path | yes | An .mjs, .mts, .js or .ts file, relative to the e2e test file that holds the step. Write the extension in lowercase, and match the letter case on disk |
timeout | no | The time limit of the step in milliseconds. The minimum is 100, and the default is 30000 |
Argent runs the script with the project root as the working directory. Argent gives the script an allowlist of environment names. Argent copies each value from the tool-server, not from the shell that starts the run. The tool-server is a long-lived process, so the values are the ones it held when it started. A variable you export now reaches a script only after you restart the tool-server. See Environment variables. The allowlist holds these groups:
PATH,HOMEand the equivalent Windows names- The identity, shell, locale, terminal, temporary-directory and cache names of the host
- The Windows platform names
- The proxy names and the TLS certificate names
- The Node, npm, Android, Java and Apple toolchain names
- The adb server names
ANDROID_ADB_SERVER_PORT,ANDROID_ADB_SERVER_ADDRESSandADB_SERVER_SOCKET. Thusadbin a script uses the same adb server as the tool-server. This includes a server thatandroid.adbServerPortselects CI,SSH_AUTH_SOCK, and each name that starts withnpm_config_
Argent removes each other name. Argent removes its own token, its own port and each ARGENT_SECRET_ value. Argent does not read a project .env file, so a name such as NODE_ENV or DATABASE_URL is absent. There is no env key. Let the script read a secret or a URL from a file.
Three entries in the list give access to a credential. Each entry holds the value of the tool-server. SSH_AUTH_SOCK reaches the SSH agent of the session that started the tool-server. An npm_config_ name can hold a registry token. HOME is the home directory of that session, so the script reads ~/.npmrc, ~/.aws/credentials and ~/.argent/ from there. Run a script only when you trust it.
PATH is the value of the tool-server too. A script that runs psql, docker or pnpm finds the binary on that PATH. An editor that starts the tool-server gives it the PATH of the editor. The script also runs under the Node binary of the tool-server, not the binary on the PATH of your shell.
A failed step stops the e2e test. The verdict names the side that caused the failure.
| Verdict | Cause | Examples |
|---|---|---|
| Failed | The script | A missing file, a load error, a thrown error, or a non-zero exit code |
| Errored | The host | A time limit, a heap limit, a signal, a process that did not start, a full queue, a cancelled run, a mis-cased filename, or a TypeScript file on a Node that does not run TypeScript |
A step that a thrown error stopped reports the message of the error and the frames of the script. Argent gives each frame a path relative to the project root. Argent removes the frames of Node and of its own runner, and keeps at most six.
The split holds only when the timeout is generous. Argent needs about a second to tell a script that never settles from a script the host stopped. Below that, the same defect of the script becomes a time limit, and the step is errored. Give a script a second or more before you read errored as a fact about the machine.
Parsing rejects a timeout below 100 milliseconds. The step starts a Node process before the script runs, and that start alone costs tens of milliseconds. A smaller limit ends the step at the time limit, or makes the result depend on the load of the host.
timeout bounds the run of the script, not the duration of the step. The tool-server runs a limited number of scripts at the same time, and it takes the number from the CPU count of the host. A step waits for a free slot before the script starts. The step reports the wait in its reason, and the wait can be longer than the timeout of the step. A step that finds the queue full is an errored step. Every project on the host shares one queue, because every project shares one tool-server.
scripts.maxTimeoutMs caps the timeout of each step, and scripts.heapLimitMb sets the heap of the process. Argent reads both keys from the global configuration file of the tool-server's own home directory. See Available keys.
On iOS, Android, Vega and macOS, a script step above the launch step runs before Argent restarts the app.
On Chromium, Argent boots the app before step 1 only when the run has no device and the leading launch names a Chromium app alone, or the requires block allows only chromium. A script step above that launch then runs while the app is up. In every other case Argent boots nothing before step 1: the script step runs first, and the launch step attaches to a running instance. Two cases reach it — a run you pin with --device, and a launch that names more than one platform when you pass no --platform and the requires block does not allow only chromium.
Module type
An .mjs or .mts script is an ES module in every project.
A .js or .ts script has the module type that Node gives each .js file in the same directory. Node reads the type field of the nearest package.json:
type | Result |
|---|---|
module | The script is an ES module. A call to require makes the step fail |
commonjs | The script is a CommonJS module. ES module syntax makes the step fail |
| Not set | Node reads a script that holds ES module syntax as an ES module. Node reads each other script as a CommonJS module |
Node counts each of these as ES module syntax:
- An
importstatement or anexportstatement. Anexport {}statement alone is sufficient. - A reference to
import.meta. - An
awaitat the top level of the script. - A
constorletdeclaration at the top level with the namerequire,module,exports,__dirnameor__filename.
A call to import() is not ES module syntax. In a .ts script, Node looks for the syntax after it removes the types. Thus an import type statement is not ES module syntax.
A React Native project usually does not set type. When type is not set, Node usually prints a warning for a script that it reads as an ES module. Argent turns this warning off, so the log of the step does not hold it.
When type is module, the error of Node for a call to require in a .js script tells you to rename the script to .cjs. Parsing rejects a .cjs file and a .cts file. Use an import statement in place of the call to require.
Use the .mjs or .mts extension for a script that must behave the same in every project.
TypeScript scripts
Node runs a .ts or .mts script without a build step. Node removes the type annotations and runs the code that remains. Argent adds no compiler.
Each Node version that Argent supports runs TypeScript. The Node build must also include TypeScript support. The nodejs package of Debian and of Ubuntu does not include it. On such a host, use a Node build from nodejs.org.
A script that imports a TypeScript file has the same requirement, also when the script is an .mjs or .js file.
When the Node of the tool-server does not run TypeScript, the result depends on the place of the TypeScript file:
| Place | Result |
|---|---|
A .ts or .mts step of the e2e test | Argent rejects the e2e test before step 1, also when the step is in a when block. argent e2e run prints not run (rejected) |
A .ts or .mts step of a run: fragment | The run step is errored, and no step of the fragment runs |
A .ts or .mts file for e2e-add-script | The script does not start, and the tool records no step |
| A TypeScript file that a script imports | The step is errored when Node loads the file |
The reason names the Node version of the tool-server. A tool-server that runs keeps its Node build. To change the Node build, stop the tool-server with argent server stop. Then run Argent on the other Node build.
When you cannot change the Node build, write the script and the files that it imports in JavaScript. Use an .mjs file in place of an .mts file. Use a .js file in place of a .ts file, or an .mjs file when the file is an ES module.
On Node 23.8 to 24.2, type stripping is experimental. Node writes a warning to the log of each step that loads a TypeScript file.
These limits apply:
- Node does not check the types. See Type check.
- Node does not read
tsconfig.json. A path alias frompathsdoes not resolve. - Node accepts only the syntax that it can remove. An
enum, anamespacethat holds code, or a parameter property makes the step fail. - Each relative import needs the file extension, for example
./helpers.ts. - An import of a type needs
import type. - Node does not run a TypeScript file from
node_modules. - An imported
.tsor.jsfile can have a different module type. See Imported files.
A frame in a failed step has the line and the column of the TypeScript source. A step that fails on syntax that Node cannot remove reports the file and the line of that syntax. On Node 23, that step reports the line but not the file.
Imported files
The .mts and .mjs extensions set the module type of one file. An imported .mts or .mjs file is an ES module too. An imported .ts or .js file takes its module type from the type field of the nearest package.json, as a .ts or .js script does. See Module type.
type | An imported .ts or .js file that has an export statement |
|---|---|
module | Node loads the file as an ES module |
commonjs | The step fails. The reason says that the file does not provide the export, or that the file is a CommonJS module |
| Not set | Node loads the file as an ES module |
When type is not set, Node has a MODULE_TYPELESS_PACKAGE_JSON warning for a file that it loads as an ES module. Argent turns this warning off, so the log of the step does not hold it.
Give a file the .mts or .mjs extension when a script imports it and the project does not set type to module.
Type check
Node does not check the types of a TypeScript script. To check the types, run tsc with a tsconfig.json that has these options. erasableSyntaxOnly needs TypeScript 5.8 or later.
{
"compilerOptions": {
"noEmit": true,
"module": "nodenext",
"allowImportingTsExtensions": true,
"erasableSyntaxOnly": true,
"verbatimModuleSyntax": true
},
"include": ["scripts"]
}
| Option | Effect |
|---|---|
noEmit | tsc checks the files and writes no output |
allowImportingTsExtensions | tsc accepts an import that ends in .ts or .mts |
erasableSyntaxOnly | tsc reports the syntax that Node cannot remove |
verbatimModuleSyntax | tsc reports an import of a type that does not use import type |
Without these options, tsc rejects an import that ends in .ts or .mts, and accepts an enum that Node rejects.
The argent e2e command
argent e2e run replays an e2e test without an agent and exits non-zero on failure. The command sends the e2e test to the tool-server that the CLI uses. See E2E tests over a link.
| Command | Description |
|---|---|
argent e2e run <name> | Run .argent/e2e/<name>.yaml, or .argent/flows/<name>.yaml when the first file does not exist |
argent e2e run <path>.yaml | Run any e2e test file. The path must not contain .. |
argent e2e run <dir> | Run every e2e test in the directory, one after the other |
argent e2e list | List the e2e test files in .argent/e2e and .argent/flows |
argent e2e list <dir> | List the e2e tests that argent e2e run <dir> runs. The path must not contain .. |
argent e2e list <dir> -r | List the e2e tests that argent e2e run <dir> -r runs |
argent e2e list --json | Print the same list as a JSON document. See The argent e2e list JSON document |
argent e2e list skips the same subdirectories as argent e2e run <dir> --recursive: __baselines__, node_modules and dot-directories other than .argent, at any depth. Argent always reads the directory that you name, also when it is one of these directories.
argent e2e list and argent e2e run <dir> skip a YAML file that is not an e2e test. Such a file has no top-level steps key, for example a docker-compose.yaml. Argent does not skip an empty file. Argent does not skip a file that has a YAML syntax error and holds steps or a top-level version: line. A directory run reports such a file as a failed e2e test. Argent does not skip a file with a newer version. argent e2e run <name> and argent e2e run <path>.yaml always run the file that you name.
argent e2e list writes a warning on stderr for an empty file and for a file with a YAML syntax error, for example warning: <path> is empty. Stdout holds only the paths.
When neither .argent/e2e nor .argent/flows exists, argent e2e list writes warning: no .argent/e2e or .argent/flows directory in the current working directory on stderr. When the directories hold no e2e test, argent e2e list writes No e2e tests found in .argent/e2e or .argent/flows on stderr. In both cases, stdout is empty and the exit code is 0. When one of the directories exists but Argent cannot read it, argent e2e list writes Could not read e2e test directory: <dir> on stderr and gives exit code 2.
argent e2e list <dir> prints paths relative to the current directory for a relative <dir>, and absolute paths for an absolute <dir>. argent e2e run accepts each printed path. Argent puts ./ in front of a printed path that starts with -, so that argent e2e run does not read the path as an option.
Argent writes an error on stderr and gives exit code 2 when <dir>:
- does not exist
- is not a directory
- cannot be read
- is empty text
- has a
..segment
When you give -r without <dir>, Argent writes e2e list --recursive requires a directory path on stderr and gives exit code 2.
When Argent finds no e2e test in <dir>, Argent writes No e2e tests found in <dir> on stderr, prints nothing on stdout and gives exit code 0. Without -r, Argent writes Pass -r/--recursive to include subdirectories. on the next line. With --json, Argent prints { "e2eTests": [] }.
| Option | Description |
|---|---|
--device <id> | The device to run on. Argent detects the device when you omit the option. A physical iPhone is never detected; name it here |
--platform <p> | ios, android, chromium, vega, macos or ios-remote. Narrows the detection. ios still never picks a physical iPhone, and never picks a remote simulator either |
--update-baselines | Write the snapshot baselines instead of comparing them. Also writes the baselines of the e2e tests that tool: e2e-execute steps run, unless the step sets updateBaselines |
--output <dir> | Write the failed baseline, current and diff images and the screenshots of failed steps to <dir>/<test>/, for a CI artifact upload |
-r, --recursive | With a directory, also run the e2e tests in subdirectories. Argent skips __baselines__, node_modules and dot-directories other than .argent |
--json | Print the JSON report of the e2e test, or the JSON aggregate of the directory |
--json-stream | Print one JSON record per line: each step, then the result. One e2e test, and not with --json |
-- | End of options, for an e2e test name that starts with - |
argent e2e run checkout --platform ios
argent e2e run .argent/e2e/checkout.yaml --output e2e-artifacts --json
argent e2e run ~/shared-e2e/checkout.yaml --device <UDID> --update-baselines
argent e2e run .argent/e2e --recursive
The file name without .yaml names the report and the artifacts. It contains only letters, numbers, _ and -.
When a step fails, Argent takes a screenshot of the device. Argent shows its path under the step as screen. Argent takes no screenshot when an earlier step or the failed step types a {{secret:…}} value, because the value can be on the screen. With --output, Argent writes the screenshot to <dir>/<test>/step-<n>-screen.png, where <n> is the number of the step in the report.
Each step line shows the time of the step before the reason, for example (1.2s). A step that did not run shows no time. The time of a when or run step does not include the steps in its block or fragment. The last line of an e2e test run shows the time of the run. The summary of a directory run shows the time of all e2e tests, for example (1m 32s).
A directory run prints a numbered header for each e2e test. Under the header Argent prints each step of that e2e test that failed or carries a warning, with its artifact paths. Argent then prints one outcome line. After the last e2e test, Argent prints a Failed e2e tests section when one or more e2e tests failed. Argent prints a summary of all e2e tests at the end.
A directory run skips an e2e test when the target does not satisfy its requires block. The outcome line is skipped (requires not met). The next line gives the reason. When the directory run skips all e2e tests, the summary starts with NONE RAN and the exit code is 2.
An e2e test that fails its steps lets the directory run continue. So does an e2e test that the tool-server rejects up front, such as an invalid file or a device that Argent cannot resolve. So does an e2e test when the upload of one of its fragments fails. A transport failure ends the directory run. So does a rejection that the tool-server does not mark as a validation error, and a reply that is not a report. Argent counts the e2e tests after the failure as skipped.
The outcome line of an e2e test that produced a report gives the result, the step counts and the time of that e2e test. The outcome line of an e2e test that produced no report states the reason. A single e2e test run prints the same reason under the name of the e2e test.
| Outcome line | Meaning |
|---|---|
not run (invalid e2e test) | The tool-server rejected the e2e test file or one of its steps |
not run (needs a newer Argent) | The e2e test file has a version that is higher than the version this Argent reads |
not run (no device resolved) | Argent did not resolve one device for the e2e test |
not run (requires not met) | The target does not satisfy the requires block. Single e2e test runs only |
skipped (requires not met) | The target does not satisfy the requires block. Directory runs only |
not run (rejected) | The tool-server rejected the e2e test for another reason |
not run (upload failed) | The upload of a run: fragment failed, so the CLI did not send the e2e test |
did not finish (run error) | The run failed before it produced a report |
did not finish (no run report) | The tool-server answered without a run report |
not run (batch stopped) | An earlier failure ended the directory run. Directory runs only |
not run (tool-server unreachable) | The CLI did not connect to the tool-server that argent link or ARGENT_TOOLS_URL names |
The Failed e2e tests section has one entry for each e2e test that failed. Each entry holds these items:
- The e2e test path.
- The first step that failed or errored, with its number. An e2e test without a report gives its outcome line instead.
- The reason of that step, or the error message of an e2e test without a report.
- A
re-run:command that runs only that e2e test. The command uses the same--device,--platformand--update-baselinesoptions as the directory run. If the directory run has--output, the command writes the images of that e2e test to the same directory as the directory run.
The section does not include the e2e tests that Argent skipped.
This is the end of the output of argent e2e run ./e2e --recursive --platform chromium:
Failed e2e tests (2)
✗ a-login.yaml › step 3 assert visible "Dashboard"
no element matched selector text="Dashboard"
re-run: argent e2e run e2e/a-login.yaml --platform chromium
✗ sub/c-search.yaml › not run (invalid e2e test)
[Tool:e2e-execute] Unrecognized e2e test entry (unrecognized step kind): {"swipe-left":{"text":"Results"}}
re-run: argent e2e run e2e/sub/c-search.yaml --platform chromium
FAIL — 4 e2e tests: 2 passed, 2 failed, 0 skipped (10.2s)
When a single e2e test run fails its steps, Argent prints the first step that failed or errored, and its reason, again above the last line.
--json and --json-stream print no header, no outcome line, no Failed e2e tests section and no summary.
A directory run with --json writes one document. The document holds ok, total, passed, failed, skipped, durationMs and an e2eTests array. durationMs is the time of the directory run in milliseconds. The array holds one entry for each e2e test. Each entry holds the path and the status. An entry holds report when the e2e test produced one, whether it passed or failed its steps. An entry holds error instead when the tool call produced no report: a rejected run, a run error, or a reply that is not a report. Such an entry also holds error_code and error_kind when the tool-server or the CLI sets them. --json-stream puts the same two fields on its error record. An entry with the status skip holds neither. An entry that Argent skipped for its requires block holds reason. ok is false when the directory run skipped all e2e tests.
A single e2e test run with --json writes the report of that e2e test. The report holds e2eTest, the name of the e2e test, and startedAt, the start of the run in epoch milliseconds, and durationMs, the time of the run in milliseconds. Each step holds e2eTest, the e2e test or fragment that the step belongs to. Each step that ran holds its own durationMs. Each progress record of --json-stream holds one step, with the same durationMs. When the run gets no report, Argent writes one JSON object on stderr instead. The object holds event and error. It also holds error_code and error_kind when the tool-server or the CLI sets them. --json-stream writes the same object on stdout.
With --json, Argent writes these objects on stderr, one on each line. Argent writes one object for each of these errors:
- an e2e test in a directory run that failed without a report
- a usage error, such as an unknown flag
A usage error prints no help text and gives exit code 2. The warnings of --output stay plain text.
Pin --platform and --device on iOS, Android and Vega. On Chromium, pass --platform chromium and omit --device. Then the runner boots the declared app path with a reproducible window size. On a Mac app, pass --platform macos and omit --device. Then the runner starts the app that the launch step names.
The argent e2e list JSON document
argent e2e list --json writes one document on stdout. The document holds an e2eTests array. The array has the same order as the plain output. Each entry holds these fields:
| Field | Description |
|---|---|
path | The path that the plain output prints |
name | The name that argent e2e run <name> accepts. Only for a file directly in .argent/e2e or .argent/flows |
legacy | true for an e2e test in .argent/flows |
shadowedBy | The path of an earlier entry with the same name. argent e2e run <name> runs that entry instead. Only when such an entry exists |
warning | The problem in the file, as one of the codes below. Only when the file is empty or has a YAML syntax error |
warning code | Meaning |
|---|---|
empty | The file is empty or holds only whitespace |
syntax-error | The file has a YAML syntax error |
{
"e2eTests": [
{ "path": ".argent/e2e/login.yaml", "name": "login", "legacy": false },
{ "path": ".argent/e2e/suite/pay.yaml", "legacy": false },
{
"path": ".argent/flows/login.yaml",
"name": "login",
"legacy": true,
"shadowedBy": ".argent/e2e/login.yaml"
}
]
}
When Argent finds no e2e test, the array is empty and the exit code is 0. When neither .argent/e2e nor .argent/flows exists, the document also holds a top-level warning field with the code no-e2e-directory. The top-level warning field is not the warning field of an entry. With --json, Argent writes no notes on stderr. With --json, a usage error writes one JSON object with event and error on stderr and gives exit code 2.
argent e2e list <dir> --json gives each entry only the path field, and the warning field when it applies. With --json, Argent writes each <dir> error as the JSON error object on stderr.
E2E tests over a link
argent link and the ARGENT_TOOLS_URL variable route the CLI to a tool-server. In this section, both are a link. Over a link, argent e2e run uploads these files with the call:
- the e2e test file
- each
run:fragment and each nested e2e test that the e2e test reaches. A nested e2e test is an e2e test that atool: e2e-executestep runs - the snapshot baselines of each run
- the file arguments of the
tool:steps
The other files stay on the client.
A file argument is an argument that a tool: step reads as one file, named by an absolute path that ends in .png or .yaml. The case of the letters does not matter. An example is baselinePath of screenshot-diff.
| E2E test content | Over a link |
|---|---|
Directives that drive the device, echo, when | Runs |
run:, snapshot:, a tool: step with a file argument | Runs. When the CLI or the tool-server is an older version, the tool-server rejects the e2e test before the first step |
script: | The tool-server rejects the e2e test before the first step |
A tool: step that names a file by a relative path, or a file that does not end in .png or .yaml | The tool-server rejects the e2e test before the first step. Use an absolute path to a .png or .yaml file |
A tool: step that takes a directory, an app or an output directory, for example react-profiler-component-source, reinstall-app or screenshot-diff with outputDir | The tool-server rejects the e2e test before the first step |
A tool: step that builds a file path from two or more arguments, for example e2e-read-prerequisite with name | The tool-server rejects the e2e test before the first step |
tool: e2e-execute with name and an absolute project_root without a .. segment | Runs. The CLI also uploads the e2e test that the step names, with its own files. The same rules apply to that e2e test. When the CLI or the tool-server is an older version, the tool-server rejects the e2e test before the first step |
tool: e2e-execute with e2e_test_path, or with name and a project_root that is relative or has a .. segment | The tool-server rejects the e2e test before the first step |
A tool: step that records an e2e test, for example e2e-add-step | The tool-server rejects the e2e test before the first step |
A relative launch: { chromium: <path> } | The launch fails. Use an absolute path on the tool-server |
The rejections in the table apply to the e2e test file and to each fragment and nested e2e test that the CLI uploads. For each rejection, the error lists the steps that caused it. When an older CLI sends an e2e test with run: steps, snapshot: steps, nested e2e tests or file arguments, the error also tells you to update the argent CLI. For a tool: step that names a file by a relative path, the error tells you to use an absolute path. For a file that does not end in .png or .yaml, the error says that a tool: step can name only such a file.
The CLI uploads each fragment that a run: step names, and each nested e2e test that a tool: e2e-execute step names with name. This includes the steps in fragments, in nested e2e tests and in when: blocks. The CLI uploads a fragment or a nested e2e test only when its real location is under one of these directories:
- the project root, which is the working directory of the CLI
- the project of an e2e test saved under
<project>/.argent/e2e/or<project>/.argent/flows/ - the
.argent/e2eand.argent/flowsdirectories of the project root - the directory of the e2e test file, and the directory of its real file when the e2e test file is a symlink
The CLI does not upload these fragments and nested e2e tests:
- a file outside these directories
- a
.yamlname that links to a file that is not a YAML file - a directory, or a file that the CLI cannot read
- a file larger than 32 MiB
The tool-server then rejects the e2e test before the first step. The error gives the reason for each such file.
A fragment or a nested e2e test that does not exist in these directories does not stop the e2e test before the first step. Its step reports error when the step runs. A when: block that does not run does not need its files.
For an e2e test with snapshot: steps, the CLI also uploads baselines from the __baselines__/<test>/ directory that a run without a link uses. A snapshot: step in a fragment uses the baselines of the e2e test file, as in a run without a link. A nested e2e test is a run of its own, with the baselines beside its own real file. It updates its baselines as Snapshots describes. The upload depends on the run:
| Run | The CLI uploads |
|---|---|
| A run that compares baselines | The baselines of the snapshot: steps in the e2e test and its fragments, also with cropOn. With --platform, or a Chromium device in --device, only the baselines of that platform |
| A run that updates baselines | The name of each baseline, without its content |
When a run that compares and a run that updates use the same directory, the CLI uploads the content of each baseline that the run that compares reads. It uploads the name of each other baseline.
When a run updates baselines, the tool-server returns each new baseline in the result, also the baselines of nested runs. The CLI writes each one into the directory of its run after the run, and into no other directory. When the CLI cannot write a baseline, it prints the path and the reason. The run then fails with exit code 1. When the connection closes before the result, the CLI writes no baseline.
The CLI does not upload a baseline that links out of the directories above, or a .png name that links to a file that is not a PNG file. A snapshot: step that reads or writes such a baseline reports error. When the __baselines__/<test>/ directory itself links out of these directories, the CLI uploads no baseline and writes none.
The CLI also uploads each file argument of the tool: steps in the e2e test, its fragments and its nested e2e tests, in each when: block. Only an argument that the tool declares as a file counts: an absolute .png path that a keyboard step types is not uploaded. When a step reads a baseline that an earlier step of the same call wrote, the step gets the new baseline. This is also true in a nested run.
The CLI does not upload these file arguments:
- a file argument whose real location is outside the directories above
- a name that links to a file that does not end in
.pngor.yaml - a file argument that is not on the client
Such a file argument does not stop the e2e test before the first step. Its tool: step reports error when it runs. The tool-server never uses a file at the same path on its own computer.
The CLI puts fragments, nested e2e tests, baselines and file arguments in the call until they are 256 KiB in total. The call carries the e2e test file and these files as base64 text, which is about 4/3 of their size. The CLI sends each other file in a separate request to POST /upload. Thus a reverse proxy in front of the tool-server must accept these request bodies:
- on
POST /tools/e2e-execute, the e2e test file and about 350 KiB more - on
POST /upload, one file at a time
When the proxy refuses a file with the status 413, the e2e test does not run. The error gives the body size that the proxy must accept on POST /upload. When the proxy refuses the call with the status 413, the error gives only the status. The Tool-server API reference documents the upload.
Set ARGENT_E2E_FILES_LOG=1 to see what the CLI does with each fragment, nested e2e test, baseline and file argument. The CLI then prints one [e2e-files] line on stderr for each file before the call, and one line for each baseline that it writes. With --json, each line is a warning object: { "event": "warning", "warning": <text> }.
The tool-server runs the uploaded copy even when it has a file at the same path. This is also true over a link to 127.0.0.1. Only the copy from the client is certain to be the latest. Without a link, the tool-server reads the file in place, and all step kinds run.
A rejected e2e test gives exit code 1. With --json, the CLI writes one JSON object for each rejected e2e test on stderr: { "event": "error", "error": <message>, "error_code": "FLOW_FILE_INVALID", "error_kind": "validation" }. With --json-stream, the same object is the last line on stdout. For a file with a version that is higher than the version this Argent reads, the object holds "error_code": "FLOW_FILE_NEWER_VERSION". A directory run continues with the next e2e test and keeps the rejected e2e test in its e2eTests array.
When the upload of a fragment to POST /upload fails, the e2e test does not run. The CLI prints the reason. The exit code is 1. The JSON object holds "error_code": "FILE_INPUT_UPLOAD_FAILED" and "error_kind": "validation". A directory run continues with the next e2e test.
When the CLI cannot connect to the tool-server that a link names, the e2e test does not run. The CLI prints the address of that tool-server, the setting that names it and the reason, for example connection refused. The exit code is 2. The JSON object holds "error_code": "TOOL_SERVER_UNREACHABLE" and "error_kind": "network". A directory run stops at that e2e test and marks the e2e tests after it as skipped.
Remote runs
The e2e-execute tool takes one e2e test source: name for a saved e2e test, or e2e_test_path for any file. Without a link, the tool-server reads the file in place, and all step kinds run.
Over a link, both sources send these files with the call, and the tool-server runs those copies:
- the e2e test file
- each
run:fragment that the e2e test reaches - each nested e2e test, that is an e2e test that a
tool: e2e-executestep names withnameand an absoluteproject_root - the snapshot baselines of each run
- the file arguments of the
tool:steps
The client sends these files with the rules of E2E tests over a link. The project_root argument is the project root. When a run of the call updates baselines, the client writes each new baseline into the __baselines__/<test>/ directory of its e2e test when the run ends. When the client cannot write a baseline, the report shows baseline not written with the path and the reason. When the call sets updateBaselines, a tool: e2e-execute step in the e2e test also updates the baselines of its e2e test, unless the step sets updateBaselines itself.
Before the first step, Argent checks the e2e test and each fragment and nested e2e test that the client sends, at any depth. Argent rejects these:
- each step that the table in E2E tests over a link rejects
- a fragment or a nested e2e test that the client does not send, except one that does not exist
The error lists each such step. A fragment, a nested e2e test or a file argument that does not exist fails its step when that step runs. See E2E tests over a link for the argent e2e run command.
YAML safety
Quote a string that contains #, : or quotes. Quote a number, true or false in a text slot. Use single quotes for a regex with backslashes. Parsing rejects an unknown directive, an invalid selector, an invalid regex, else, an unsupported option, and a standalone e2e test that declares executionPrerequisite.