Agentic Scrum, Part 4 of 5: Tests Are the New Code Review

This is part 4 of 5 in the Agentic Scrum series. Part 3 covered the specs and the coding loop. This part is about the QA agent and why tests carry so much weight once humans stop reading code. Part 5 closes the series with security, compliance and the final gate.

In Part 1 I made a trade: Humans no longer review generated code. The tests are what make that trade survivable, because if they are weak, nothing else stands between a plausible diff and production. That makes the quality of the test specification the critical path of the whole pipeline.

 

Where the QA agent gets its test cases

The QA agent reads agent-spec.md and requirements.md first, because they say what the feature should do. It reads the code second, only to learn names, interfaces and where behavior lives. The order matters. Tests derived from code confirm what the code does, bugs included. Tests derived from the spec confirm what we asked for. A suite written only from the code is a diary, and you want a suite that reads like a contract.

The test specification

Before any test code exists, the agent writes test-spec.md. Each case has an ID, the requirement it traces to, a type, a starting state, an action and an expected result. Here is a short excerpt for the invoice export from the earlier parts.

# Invoice export: test specification
--
TC-01 traces to R1
Type: integration
Given: a customer with 3 invoices in March
When: the user exports March in format A
Then: the file contains exactly those 3 invoices

TC-02 traces to R2
Type: integration
Given: two customers, each with invoices in March
When: customer 1 exports March
Then: no invoice of customer 2 appears in the file

TC-03 traces to F1
Type: integration
Given: a customer with no invoices in April
When: the user exports April
Then: the response says no invoices were found and no file is created

TC-04 traces to N1
Type: performance
Given: a customer with 40000 invoices in one period
When: the user exports that period
Then: the export finishes within 5 minutes

TC-05 traces to F3
Type: integration
Given: an export in progress
When: the connection drops halfway
Then: no partial file remains on the server
--

The traceability rule is simple. Every requirement ID in requirements.md has at least one case, and every case traces to a requirement or a failure mode. A small script can check this and list the gaps. It does not need an agent, and it is dull and extremely effective. A requirement with no test case is a promise nobody is checking.

Who validates the test specification

QA engineers and developers review test-spec.md in a pull request before the QA agent writes any test code. Reviewing a test spec is cheaper than reviewing test code, because the question is whether we test the right behavior, not whether the syntax is pretty. Reviewers hunt for a short list of gaps. Missing negative cases. Boundaries such as zero, one, many, the limit and one over the limit. Data variety, like names with apostrophes, time zones and empty fields. Permission cases, such as the wrong user and the right user in the wrong role. Repeated actions, like the double click. And two kinds of bad cases: those that merely restate the implementation, and those that cannot fail.

Writing the tests

After approval, the QA agent writes the tests, runs them and records the results in handover 03. The rules live in guardrails/qa.md. The sample below is short on purpose.

# QA rules
--
Use synthetic test data only. Never copy production data.
Every test carries the ID of its test case in its name.
Do not change production code. Report failures in the handover.
Quarantine a flaky test after three inconsistent runs and note it in the handover.
Never delete or weaken a test to make a run pass.
--

The separation runs both ways. The QA agent may not touch production code, and the coding agent may not touch the tests. A failing test goes back into the coding loop with a handover that names the failing case ID, so the coding agent knows exactly which promise it broke. The three-run flaky rule is an example number, so pick your own, but write it down. A flaky test left to rot teaches the whole team to ignore red.

Do the tests actually test anything

Passing tests prove little if they cannot fail. One way to check is mutation testing. A tool changes the code in small ways, for example flipping a comparison, and sees whether any test notices. Tools exist for many languages, such as PIT for Java and Stryker for JavaScript and .NET. It costs run time, so I would run it only on the changed code and treat each surviving mutant as a question for the QA engineers. I have not measured the effect, so treat it as an option and not a rule.

What humans still test

Agents test what we specified. Humans find what we did not think of. Keep exploratory testing time for the QA engineers, especially on new features.

The strange order of clicks, the double submit, the customer who uses the export for something else entirely. Every finding becomes a new case in test-spec.md, which is how the suite grows from reality instead of imagination.

 

Previous: Part 3, Write Specs a Coding Agent Can Actually Follow. Next: Part 5, Security, Compliance and the Final Gate.

© Copyright 2026 Sorin Mustaca, All rights Reserved. Written For: Sorin Mustaca: Security-AI-Software

---

Want to work with me on this topic?
Check Endpoint Cybersecurity to see the consulting services we offer.