Green tests don't mean the spec is covered.
SpecTrace reads your requirements document, traces every numbered requirement to the code and tests behind it, writes the tests that are missing, and tells you what's covered, what's broken and what was never built, with evidence an auditor can check.
Traceability is still done by hand, in a spreadsheet.
Banks, insurers and healthcare teams must prove every requirement is implemented and tested before each release. Today someone reads the spec, searches the code for each requirement, checks for tests, writes the missing ones and updates a spreadsheet, then does it all again when the spec changes.
per requirement to locate the code, find the tests, write what's missing and update the matrix. About 13.5 hours for a 27-requirement spec (our estimate).
A passing suite only proves the existing tests pass. It says nothing about the requirements nobody tested.
When the spec changes and the code doesn't, nothing fails. The mismatch ships.
One command in Bob: /trace docs/requirements.pdf
Start the clock
Records the start time and pins a git baseline, so the tests Bob writes can never be mistaken for existing ones.
Read the spec
Document understanding pulls every requirement and acceptance criterion from the PDF, tables included, word for word.
Trace in parallel
One subagent per module finds the code, checks existing tests, writes missing JUnit tests, runs them and diagnoses failures.
accountstransfersloanssecurityVerify
A clean full test run, then a script rebuilds every status from the real results, not from what the agents said.
Report
A traceability matrix, a Markdown copy for PRs and a brief: bugs with
file:line, gaps, tests added.
Then /fix patches the bugs and re-checks until the rows turn green, and /release-notes writes a release-readiness report. When the spec changes, /trace re-checks only the modules whose requirements changed.
27 requirements in, three real bugs out.
The demo target is a Spring Boot banking API with accounts, transfers, loans and a JWT security layer. Figures below are from a live Bob run: four parallel subagents, 39 tests written, 26.2 minutes end to end.
| Requirement | The spec says | The code does | Where |
|---|---|---|---|
| REQ-SEC-05 Account ownership Not planted |
Another customer's account is refused with HTTP 403 | HTTP 200. Any logged-in customer can read, or send money from, anyone's account | TransferController.java:35 AccountController.java:44 |
| REQ-TRF-03 Daily transfer limit |
SGD 5,000.00 limit is inclusive | A day total of exactly 5,000.00 is rejected (>= instead of >) |
TransferService.java:50 |
| REQ-LN-03 Monthly instalment |
Rounded half-up: SGD 754.90 | Truncated: SGD 754.89 | LoanService.java:60 |
Two of the bugs were planted for the demo. The ownership bug came in with our own login code, and SpecTrace caught it by testing the rule through the real HTTP and Spring Security stack. It is Broken Object Level Authorization, number one on the OWASP API Security Top 10. Bob also found a gap in a requirement that looked covered: no existing test checked that a SGD 0.00 opening deposit is allowed, so it wrote one. See every requirement in the full dashboard.
The AI doesn't grade its own homework.
Agents can claim "all covered". SpecTrace doesn't take their word for anything that ends up in the report.
Statuses come from real test results
Every verdict is rebuilt from the Surefire XML the test run produced, and from the @Tag("REQ-…") annotations in the test source.
Disagreements are flagged
If a subagent says "covered" but its test fails, the report says so. The same goes for tests that are untagged or never ran.
Line numbers are checked
Every file:line an agent cites must exist and sit inside the method it names. Stale or invented locations are flagged.
Production code is off limits
Tracing modes can only write tests and reports. Only the separate Fixer mode may touch production code, and it may never weaken a test.
Every step runs on Bob.
- Document understanding
- Extracts requirements and acceptance criteria from the PDF spec
- Custom modes
- Orchestrator, per-module Tracer and Fixer, each limited to the files it may edit
- Parallel subagents
- Four modules traced at once, each in its own build folder
- Agent mode
- Writes, compiles and runs tests, and fixes its own test mistakes
- Rules
- The team's traceability conventions, applied in every mode
- Slash commands
/trace,/fix,/release-notes
Hours of checking, done in one run.
| By hand | SpecTrace | |
|---|---|---|
| Trace 27 requirements | ~13.5 h (estimate) | 26.2 min, measured on a live Bob run |
| Missing tests | Often skipped | 39 tagged tests written |
| Bugs before release | Found in UAT, or by attackers | 3 found, with file:line |
| Spec change | Redo the spreadsheet | Re-check changed modules only |
| Audit evidence | A spreadsheet you trust | Built from real test results |
Run it
git clone https://github.com/decentkiwi/spectrace.git cd spectrace && pip install -r tools/requirements.txt
You need JDK 21+, Python 3 and IBM Bob with the spectrace folder open as the workspace.
# baseline: the app's own tests, all green cd bank-app && ./mvnw test # in Bob /trace docs/requirements.pdf # trace all 27 requirements /fix # patch the bugs, re-check, rows turn green /release-notes v1.0.0 # release-readiness report /trace docs/requirements-v2.pdf # spec change: only Loans is re-checked
Results land in spectrace-out/matrix.html.