> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.qa.army/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.qa.army/_mcp/server. # QA.army CLI > Install the CLI, authorize access, save a Web Test, run it, and inspect the result from your terminal. Use `qa-army` to manage saved Tests and request Web Runs from your terminal or coding agent. Runs execute in QA.army and use the same Workspace permissions, saved Tests, results, and evidence as the app. `qa` is a shorter command alias. ## Install and discover commands Install Node.js 22 or newer, then install the versioned package from the [official CLI releases](https://github.com/QA-army/cli/releases): ```sh npm install --global https://github.com/QA-army/cli/releases/download/v0.2.11/qa-army-0.2.11.tgz npm list --global qa-army --depth=0 qa-army --help qa-army capabilities --json ``` These examples use release 0.2.11. The package is distributed through GitHub releases. Use `capabilities` to check the commands supported by your installation. It works without signing in and distinguishes supported commands from capability requests. A capability request does not perform the requested operation. ## Authorize access Interactive agent registration needs your operating system's credential store: macOS Keychain, Windows Credential Manager, or Secret Service and the desktop portal on Linux. See the [authentication guide](https://qa.army/auth.md) for platform requirements. ```sh qa-army auth agent-register --email you@example.com qa-army status --json ``` Complete sign-in in the browser opened by the CLI, then enter the short-lived code at the CLI's hidden prompt. Keep that code and all credentials out of chat, Test files, and shell arguments. If the credential store is unavailable, fix it before continuing; do not save credentials in a plain-text file. `status` reports local credential metadata; it does not test an authenticated API request. For scripts, `status` exits `0` when a credential is available and intentionally exits `1` when the result is unauthenticated. After you complete the claim, reuse the same registered agent identity to discover Workspaces and Projects, manage Test Groups, create and update saved Tests, and create, start, inspect, or cancel Runs. A second API key is not required for these operations. Workspace membership is checked on every request. Registration does not authorize every command listed by `capabilities`. Test deletion, Test Account metadata and credentials, Live View, account administration, and integration consent remain separate user operations. Signing in to QA.army also does not sign the test browser into your application; use a governed Test Account for that. Workspace Owners can also use the same claimed identity to select Test Groups for an existing connected GitHub deployment watch. Use `prs watch` to read its current selection and `prs watch-update` to replace the complete selection, then read it back. See [regression watch configuration](/github-pull-request-testing#select-regression-test-groups-with-an-agent) for the request body and permission limits. This does not connect an integration or grant consent. ## Select an existing Project Replace the example identifiers below with complete IDs returned by QA.army: ```sh qa-army workspaces list qa-army projects list --workspace wsp_... qa-army tests list --project prj_... qa-army tests get --test tst_... ``` Confirm the Project's target before making a change. Supply the relevant Workspace or Project ID on each command when working across several Projects. An identifier never grants access to another Workspace. For a new Project, first prepare a Test file as described below. With your registered agent, use `setup` only when you also intend to start a Run: ```sh qa-army setup --workspace wsp_... --app-url https://your-app.example --project-name "Your project" --input "$(cat test.json)" ``` `setup` creates or reuses the Project and Test, starts **one Run**, prints its live URL, and waits for a result. Each explicit invocation requests a new Run. Use a target QA.army can reach; a local-only address is not a supported target. It exits `0` only when setup completes and the Run is `PASSED`. A valid terminal result of `FAILED`, `ERROR`, or `CANCELLED` exits `1`, as do setup, API, and waiting errors. Inspect the emitted receipts to distinguish the Test result from a command error. ## Save a Test without running it Save this example as `test.json` and adapt the instructions to your application: ```json { "name": "Signup entry point", "enabled": true, "allow_web_search": false, "deep_thinking": false, "steps": [ { "type": "act", "instruction": "Open the Project homepage", "enabled": true }, { "type": "assert", "instruction": "Verify the Get started link is visible", "enabled": true } ] } ``` The boolean settings and each step's `enabled` flag are required. Use dedicated Test Accounts for login steps; never put passwords or tokens in this file. ```sh qa-army tests create --project prj_... --input "$(cat test.json)" ``` This saves the definition and returns its Test ID and version. It does not start a Run. The shell examples use a POSIX shell; in PowerShell, pass the file with `--input (Get-Content -Raw test.json)`. There is no `--input-file` flag. Read [Write Reliable Test Steps](/write-reliable-test-steps) before adding more coverage. ## Organize and update saved coverage Use a Test Group for a family of journeys, such as Onboarding: ```sh qa-army groups list --project prj_... qa-army groups create --project prj_... --input '{"name":"Onboarding","description":"Signup and first-use journeys"}' qa-army tests get --test tst_... qa-army tests update --test tst_... --version 3 --input "$(cat test.json)" ``` Use the returned Group ID and the Test's current version. Send the complete editable Test definition with every step and setting preserved; use `group_id` to move it and `enabled: false` to keep a Test with missing prerequisites out of execution. A conflict means another editor changed the Test: reload and reconcile before saving. Updating a definition does not start a Run or alter earlier results. ## Run one saved Test Start execution only when intended; it can interact with your application and use your Workspace's Run allowance. ```sh qa-army tests run --test tst_... --wait --timeout 900000 --interval 2000 ``` `--timeout` and `--interval` are milliseconds. This command creates a Run, starts it, and waits for a terminal result. With `--wait`, exit code `0` means `PASSED`; `FAILED`, `ERROR`, and `CANCELLED` return `1`. Input, authentication, API, and waiting errors also return `1`, so inspect the receipt or error message. The completed response contains `command_status` and a nested `run` receipt; read that receipt's `status` and `run_url`. Without `--wait`, a successful response only confirms that the Run started. For explicit control of each operation: ```sh qa-army runs create --test tst_... qa-army runs start --run run_... qa-army runs get --run run_... ``` The first command creates a `READY` Run; it does not execute the Test. Use the returned Run ID in the following commands. ## Run selected Tests from a Group Read the Group and its Tests first. Save an explicit selection of 1–20 unique, enabled Tests and their current versions in `selection.json`: ```json { "selection": [ {"test_id": "tst_...", "version": 3} ] } ``` Replace the abbreviated ID with a complete Test ID, then run: ```sh qa-army groups run --project prj_... --group tgr_... --request-key onboarding-001 --input "$(cat selection.json)" ``` Every selected Test must still belong to that Project and Group, remain enabled, and match the supplied version. Each becomes a separate Run and can consume the normal Run allowance. The command checks all selected definitions before creating Runs, then verifies each created Run matches its selected Test version before starting any. A server that cannot supply that version proof is blocked. The receipt is `DISPATCHED` or `INCOMPLETE`, never a passing suite result. `DISPATCHED` returns exit code `0`; `INCOMPLETE` returns `1`. Inspect every returned Run's terminal result and evidence. An interruption may leave some Runs ready or started: reconcile their IDs before retrying, preserve the same selection and request key. On a preparation failure before dispatch, the client attempts to cancel known ready Runs; inspect each cleanup outcome and cancel any remaining unused ready Runs explicitly. Do not replace the selection with a fresh list on retry or start a child that is already beyond `READY`. ## Inspect progress and evidence ```sh qa-army runs get --run run_... qa-army runs watch --run run_... --after 0 qa-army runs wait --run run_... --timeout 900000 qa-army runs evidence --run run_... ``` `watch` reads one batch of events. Continue from the returned `next_after` with `--after`; it is not a continuous stream. `wait` polls an existing Run. Its exit code confirms the read succeeded, including when the final Run is `FAILED` or `ERROR`; inspect `status` before making a release decision. `runs evidence` lists finalized artifact metadata and stable evidence IDs. To download an authorized artifact, choose its evidence ID and a new local filename: ```sh qa-army evidence download --evidence evd_... --output screenshot.png ``` The command creates a private file without overwriting an existing file and verifies its size and SHA-256 checksum. Downloads are limited to 128 MiB. A failed download empties the file it created; it does not leave partial evidence presented as valid. The receipt omits the temporary access URL. Keep downloaded evidence private and do not share access capabilities. Open the returned `run_url` to review the Run in the app, including available screenshots and recordings. A successful download proves artifact integrity, not that the Test passed. Inspect the Run verdict and required screenshots. | Result | Meaning | | ----------- | --------------------------------------------------------- | | `PASSED` | Required expectations completed successfully. | | `FAILED` | A product assertion did not match the observed behavior. | | `ERROR` | QA.army could not establish a trustworthy product result. | | `CANCELLED` | The Run stopped before completion. | Stop an authorized Run with `qa-army runs cancel --run run_...`, then read its status again. Your claimed agent identity can request cancellation within its Workspace access. The cancellation request alone is not a terminal result. ## Use output in scripts and agents Successful command output is JSON by default; `--json` is accepted. Read the resource's status, not just the command's exit code. `setup` and authentication can emit several JSON receipts, while command errors are text on stderr. Do not assume every command produces one JSON object or a JSON error envelope. Preserve resource IDs and event cursors before continuing. If a wait times out, the remote Run may still be active: inspect it with `runs get` or `runs wait`. Do not repeat `setup` or `tests run` to reconnect; those commands request a new Run. If no Run ID was received, inspect `runs list --test tst_...` and the app before deciding whether another execution is needed. Run history returns a page of results. Use `runs list --test tst_... --limit 20`, then pass its `next_before` as `--before` for the next page, until it is null. A first page is not necessarily the complete history. The basic Test and Run commands above do not expose a caller-supplied `--request-key` flag. For explicit retry control, use the API's documented idempotency contract in [Coding Agents](/interfaces/coding-agents). ## Troubleshoot | Symptom | Next action | | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | | No credential or expired registration | Check `qa-army status --json`; complete registration again if requested. | | Credential store unavailable | Unlock or configure the native store using the authentication guide. | | Access denied | Confirm Workspace access and whether the operation permits your credential type. Repeating sign-in does not add permissions. | | Invalid Test input | Check required booleans, ordered steps, and complete IDs against the [API schema](https://api.qa.army/v1/openapi.json). | | Waiting timed out | Preserve the Run ID and inspect that Run before retrying execution. | | `REQUEST_CAPABILITY` and exit code `2` | The requested CLI operation is unavailable; inspect `capabilities` for supported alternatives. | Direct Web testing is available. Native Android, iOS, and Desktop execution are not generally available. See [CI and Deployments](/ci-and-deployments) for script boundaries and [Results and Evidence](/results-and-evidence) for result review.