Frequently Asked Questions

What can I use today?

You can sign in to the QA.army SaaS app, create or reuse a Workspace and Project, save plain-language Web Tests, start an explicit Web Run, and review its result and available evidence. Native Android, iOS, and Desktop execution are not generally available yet.

How many Runs are free?

Every new Workspace includes 10 initial free Runs with no credit card required. The allowance does not reset monthly. Creating or editing Projects, Test Groups, or Tests does not by itself authorize a Run; execution remains explicit.

Which journeys should I add first?

Start with flows whose failure would directly affect customers or block a release: sign-in, onboarding, checkout, billing, account recovery, and important cross-application journeys.

Choose clear owners and write observable assertions for each Test.

How should I organize coverage?

Use one Project for each independently testable product surface. Add environments for its approved targets, then organize related Tests into groups such as Authentication, Checkout, or Account Management.

What is the difference between FAILED and ERROR?

FAILED means the product journey ran and an assertion failed. ERROR means QA.army could not determine a trustworthy product result because setup, access, execution, or required evidence was incomplete.

This distinction keeps test problems from being reported as product bugs.

What should I do when a Run fails?

Review the failed assertion and available evidence. Confirm the problem is reproducible, capture the affected environment and account role, then create a clear bug report with the observed and expected behavior.

What if a Test is temporarily unreliable?

Do not weaken its assertions to force a pass. Check the environment, account access, test data, and product state. Run the saved Test again only after the likely cause is understood.

How do I keep credentials safe?

Use dedicated test accounts with the minimum permissions required. Never place passwords, access tokens, recovery codes, or real customer data in Test steps, screenshots, fixtures, or notes.

Who can see Tests and evidence?

Only authorized Workspace members can access its Projects, Tests, Runs, and evidence. Review membership regularly and remove people who no longer need access.

Which interface should I use?

• Use the SaaS app for visual setup, direct Web Runs, and evidence review.
• Use the CLI for terminal setup, saved Tests, and supported Web Run operations, or the API for custom integrations.
• Use the published local MCP server with a supported client and the same claimed identity used by the CLI. Hosted MCP is unavailable.
• The SaaS chatbot is currently unavailable.

Every available interface uses the same Workspace permissions and result meanings.

Does a successful CLI command mean my Test passed?

qa-army tests run --test tst_... --wait uses exit code 0 to report a PASSED Test. A successful create, start, get, or wait operation can return a Run that has not passed. Inspect the Run’s status and evidence. See CI and Deployments for script guidance.

Can a registered agent edit Tests or cancel Runs?

Yes. After the human claim is complete, the same registered agent identity can create and update saved Tests, manage Test Groups, and request Run cancellation within its Workspace access. No second API key is required. Read the Run again to confirm a cancellation request reached a terminal result.

Test deletion, Test Account metadata and credentials, Live View, membership changes, billing, and integration consent remain separate user operations. A command appearing in CLI help does not grant permission to use it.

Should I run the command again after a timeout?

Inspect the existing Run first. A client timeout does not stop remote execution, and another tests run or setup command requests a new Run. Preserve the Run ID, or inspect the saved Test’s Run history if the original response was lost.

Does QA.army support Web, Mobile, and Desktop?

QA.army’s product scope includes Web, Android, iOS, and Desktop. Direct Web execution is available. Native Android, iOS, and Desktop execution are not generally available yet.

Are schedules and pull-request tests available?

Scheduled monitoring is unavailable. Dynamic pull-request testing is an assisted pilot and is not generally available. An authorized script can request a saved Web Test through the CLI; that alone does not set up schedules or pull-request checks. Read GitHub Pull Request Testing for pilot requirements.