An autonomous QA loop: test, fix, re-test, until green

The recording above is the real CLI on a real session pair — round 1 with two evidence-backed fails and a fix pass, the re-test round, and the verdict flipping to green. The rendered result: the live example report.

Set the project up

npx sluglist init --agents-md

That installs the four bundled skills into .claude/skills/, creates .sluglist/checklists/, writes the .gitignore rules, and drops a .sluglist/PROJECT.md to fill in. Fill it in — it is where the loop learns your base branch, how to start the app, how to sign in (referenced, never stored), the actions it must never complete, and how far it may go on its own:

max rounds: 3
fix without asking: no
commits: leave the changes uncommitted, one summary at the end

Ask for the whole thing

The sluglist-loop skill takes it from there:

1. Checklist — the branch diff becomes a client-voice list at .sluglist/checklists/.
2. QA run — a browser walks every item and writes verdicts and issues through the headless writer, sluglist/node.
3. Report — npx sluglist report renders the round as one HTML file.
4. Status — npx sluglist status --json says what is still failing and whether it is worth another round.
5. Fix — the failing issues are patched; each outcome is recorded in fixes.yaml.
6. Re-test — a checklist of only the fixed items, ids preserved and provenance attached, then back to step 4.

See the output for yourself: a live example report — one self-contained HTML file from a real QA round on a demo app: evidence-backed passes with the observed facts, a fail linked to its issue, an item honestly reported as not tested, and the fix pass answering in fixes.yaml.

The decision point

npx sluglist status

release-2026-08 · branch · 3 items
  1  session-2026-08-15-tw1w  1 pass · 1 fail · 1 not tested  ·  1 fixed
  2  session-2026-08-15-jtyf  0 pass · 1 fail · 0 not tested  ·  no fix pass yet

  still failing (1)
    csv-columns — for the next fix pass · failed in 2 rounds · issue 01

verdict: stalled — 1 item failed in 2 or more rounds — a fix pass has already been tried

It is derived entirely from the artifacts — the verdicts in session.yaml, the resolutions in fixes.yaml, and the retest_of chain that links round 2 back to round 1. That matters: the one thing an agent should not be asked is whether its own work is done.

The guarantees that make it usable

• No fail without a screenshot; no pass without performing the check. An unreachable item is not tested, with the reason.
• A screenshot proves the screen looked right, never that the action worked. For downloads, submissions and background jobs the verdict carries the observed fact — the file name and size, the toast text, the row count.
• The loop may not manufacture green. It cannot edit, narrow or delete a check so it stops failing, and it cannot write wontfix to end a round; those are proposals surfaced to you.
• Hard limits are enforced. Live payments, real emails, external submissions: the run stops at the last safe step and records not tested with the reason.

Drive it yourself

The skills are convenience, not dependency — the writer is a public API:

import { createSession, LocalConnector } from "sluglist/node";

const session = await createSession({
  connectors: [new LocalConnector({ dir: ".sluglist" })],
  project: "acme",
  baseUrl: "http://localhost:3000",
  checklist: ".sluglist/checklists/release-2026-08.json",
  reporter: { name: "qa-agent", kind: "agent" },
});

await session.setVerdict("csv-columns", "pass", {
  evidence: {
    screenshots: [png],
    note: "Exported reports-2026-08.csv — 4.1 KB, 57 rows, all 9 columns present",
  },
});

// An item it could not check: no verdict, and the reason with it.
await session.setVerdict("email-receipt", null, {
  evidence: { note: "could not test: no mailbox reachable from this environment" },
});

Any language that can write files can produce the same artifacts: the format is documented field by field.