QA.army CLI | QA.army | Documentation

Use qa-army to manage saved Tests and request Web Runs from your terminal or coding agent. Runs execute in QA.army and use the same Workspace permissions, saved Tests, results, and evidence as the app. qa is a shorter command alias.

Install and discover commands

Install Node.js 22 or newer, then install the versioned package from the official CLI releases:

npm install --global https://github.com/QA-army/cli/releases/download/v0.2.11/qa-army-0.2.11.tgz
npm list --global qa-army --depth=0
qa-army --help
qa-army capabilities --json

These examples use release 0.2.11. The package is distributed through GitHub releases. Use capabilities to check the commands supported by your installation. It works without signing in and distinguishes supported commands from capability requests. A capability request does not perform the requested operation.

Authorize access

Interactive agent registration needs your operating system’s credential store: macOS Keychain, Windows Credential Manager, or Secret Service and the desktop portal on Linux. See the authentication guide for platform requirements.

qa-army auth agent-register --email you@example.com
qa-army status --json

Complete sign-in in the browser opened by the CLI, then enter the short-lived code at the CLI’s hidden prompt. Keep that code and all credentials out of chat, Test files, and shell arguments. If the credential store is unavailable, fix it before continuing; do not save credentials in a plain-text file. status reports local credential metadata; it does not test an authenticated API request. For scripts, status exits 0 when a credential is available and intentionally exits 1 when the result is unauthenticated.

After you complete the claim, reuse the same registered agent identity to discover Workspaces and Projects, manage Test Groups, create and update saved Tests, and create, start, inspect, or cancel Runs. A second API key is not required for these operations. Workspace membership is checked on every request.

Registration does not authorize every command listed by capabilities. Test deletion, Test Account metadata and credentials, Live View, account administration, and integration consent remain separate user operations. Signing in to QA.army also does not sign the test browser into your application; use a governed Test Account for that.

Workspace Owners can also use the same claimed identity to select Test Groups for an existing connected GitHub deployment watch. Use prs watch to read its current selection and prs watch-update to replace the complete selection, then read it back. See regression watch configuration for the request body and permission limits. This does not connect an integration or grant consent.

Select an existing Project

Replace the example identifiers below with complete IDs returned by QA.army:

qa-army workspaces list
qa-army projects list --workspace wsp_...
qa-army tests list --project prj_...
qa-army tests get --test tst_...

Confirm the Project’s target before making a change. Supply the relevant Workspace or Project ID on each command when working across several Projects. An identifier never grants access to another Workspace.

For a new Project, first prepare a Test file as described below. With your registered agent, use setup only when you also intend to start a Run:

qa-army setup --workspace wsp_... --app-url https://your-app.example --project-name "Your project" --input "$(cat test.json)"

setup creates or reuses the Project and Test, starts one Run, prints its live URL, and waits for a result. Each explicit invocation requests a new Run. Use a target QA.army can reach; a local-only address is not a supported target. It exits 0 only when setup completes and the Run is PASSED. A valid terminal result of FAILED, ERROR, or CANCELLED exits 1, as do setup, API, and waiting errors. Inspect the emitted receipts to distinguish the Test result from a command error.

Save a Test without running it

Save this example as test.json and adapt the instructions to your application:

{
  "name": "Signup entry point",
  "enabled": true,
  "allow_web_search": false,
  "deep_thinking": false,
  "steps": [
    {
      "type": "act",
      "instruction": "Open the Project homepage",
      "enabled": true
    },
    {
      "type": "assert",
      "instruction": "Verify the Get started link is visible",
      "enabled": true
    }
  ]
}

The boolean settings and each step’s enabled flag are required. Use dedicated Test Accounts for login steps; never put passwords or tokens in this file.

qa-army tests create --project prj_... --input "$(cat test.json)"

This saves the definition and returns its Test ID and version. It does not start a Run. The shell examples use a POSIX shell; in PowerShell, pass the file with --input (Get-Content -Raw test.json). There is no --input-file flag. Read Write Reliable Test Steps before adding more coverage.

Organize and update saved coverage

Use a Test Group for a family of journeys, such as Onboarding:

qa-army groups list --project prj_...
qa-army groups create --project prj_... --input '{"name":"Onboarding","description":"Signup and first-use journeys"}'
qa-army tests get --test tst_...
qa-army tests update --test tst_... --version 3 --input "$(cat test.json)"

Use the returned Group ID and the Test’s current version. Send the complete editable Test definition with every step and setting preserved; use group_id to move it and enabled: false to keep a Test with missing prerequisites out of execution. A conflict means another editor changed the Test: reload and reconcile before saving. Updating a definition does not start a Run or alter earlier results.

Run one saved Test

Start execution only when intended; it can interact with your application and use your Workspace’s Run allowance.

qa-army tests run --test tst_... --wait --timeout 900000 --interval 2000

--timeout and --interval are milliseconds. This command creates a Run, starts it, and waits for a terminal result. With --wait, exit code 0 means PASSED; FAILED, ERROR, and CANCELLED return 1. Input, authentication, API, and waiting errors also return 1, so inspect the receipt or error message. The completed response contains command_status and a nested run receipt; read that receipt’s status and run_url.

Without --wait, a successful response only confirms that the Run started. For explicit control of each operation:

qa-army runs create --test tst_...
qa-army runs start --run run_...
qa-army runs get --run run_...

The first command creates a READY Run; it does not execute the Test. Use the returned Run ID in the following commands.

Run selected Tests from a Group

Read the Group and its Tests first. Save an explicit selection of 1–20 unique, enabled Tests and their current versions in selection.json:

{
  "selection": [
    {"test_id": "tst_...", "version": 3}
  ]
}

Replace the abbreviated ID with a complete Test ID, then run:

qa-army groups run --project prj_... --group tgr_... --request-key onboarding-001 --input "$(cat selection.json)"

Every selected Test must still belong to that Project and Group, remain enabled, and match the supplied version. Each becomes a separate Run and can consume the normal Run allowance. The command checks all selected definitions before creating Runs, then verifies each created Run matches its selected Test version before starting any. A server that cannot supply that version proof is blocked.

The receipt is DISPATCHED or INCOMPLETE, never a passing suite result. DISPATCHED returns exit code 0; INCOMPLETE returns 1. Inspect every returned Run’s terminal result and evidence. An interruption may leave some Runs ready or started: reconcile their IDs before retrying, preserve the same selection and request key. On a preparation failure before dispatch, the client attempts to cancel known ready Runs; inspect each cleanup outcome and cancel any remaining unused ready Runs explicitly. Do not replace the selection with a fresh list on retry or start a child that is already beyond READY.

Inspect progress and evidence

qa-army runs get --run run_...
qa-army runs watch --run run_... --after 0
qa-army runs wait --run run_... --timeout 900000
qa-army runs evidence --run run_...

watch reads one batch of events. Continue from the returned next_after with --after; it is not a continuous stream. wait polls an existing Run. Its exit code confirms the read succeeded, including when the final Run is FAILED or ERROR; inspect status before making a release decision.

runs evidence lists finalized artifact metadata and stable evidence IDs. To download an authorized artifact, choose its evidence ID and a new local filename:

qa-army evidence download --evidence evd_... --output screenshot.png

The command creates a private file without overwriting an existing file and verifies its size and SHA-256 checksum. Downloads are limited to 128 MiB. A failed download empties the file it created; it does not leave partial evidence presented as valid. The receipt omits the temporary access URL. Keep downloaded evidence private and do not share access capabilities.

Open the returned run_url to review the Run in the app, including available screenshots and recordings. A successful download proves artifact integrity, not that the Test passed. Inspect the Run verdict and required screenshots.

Result	Meaning
`PASSED`	Required expectations completed successfully.
`FAILED`	A product assertion did not match the observed behavior.
`ERROR`	QA.army could not establish a trustworthy product result.
`CANCELLED`	The Run stopped before completion.

Stop an authorized Run with qa-army runs cancel --run run_..., then read its status again. Your claimed agent identity can request cancellation within its Workspace access. The cancellation request alone is not a terminal result.

Use output in scripts and agents

Successful command output is JSON by default; --json is accepted. Read the resource’s status, not just the command’s exit code. setup and authentication can emit several JSON receipts, while command errors are text on stderr. Do not assume every command produces one JSON object or a JSON error envelope.

Preserve resource IDs and event cursors before continuing. If a wait times out, the remote Run may still be active: inspect it with runs get or runs wait. Do not repeat setup or tests run to reconnect; those commands request a new Run. If no Run ID was received, inspect runs list --test tst_... and the app before deciding whether another execution is needed.

Run history returns a page of results. Use runs list --test tst_... --limit 20, then pass its next_before as --before for the next page, until it is null. A first page is not necessarily the complete history.

The basic Test and Run commands above do not expose a caller-supplied --request-key flag. For explicit retry control, use the API’s documented idempotency contract in Coding Agents.

Troubleshoot

Symptom	Next action
No credential or expired registration	Check `qa-army status --json`; complete registration again if requested.
Credential store unavailable	Unlock or configure the native store using the authentication guide.
Access denied	Confirm Workspace access and whether the operation permits your credential type. Repeating sign-in does not add permissions.
Invalid Test input	Check required booleans, ordered steps, and complete IDs against the API schema.
Waiting timed out	Preserve the Run ID and inspect that Run before retrying execution.
`REQUEST_CAPABILITY` and exit code `2`	The requested CLI operation is unavailable; inspect `capabilities` for supported alternatives.

Direct Web testing is available. Native Android, iOS, and Desktop execution are not generally available. See CI and Deployments for script boundaries and Results and Evidence for result review.