Skip to main content
AtwalLabs

Manual QA knowledge needs an execution engine

Run the regression pack on every release, and compare it to the last one

The business-as-usual suite is the one that never gets automated, because it is long and boring and it changes. This runs it in a real browser, across the data rows it needs, and tells you what differs from the previous run — not just how many passed.

01

One journey, many rows

Upload a sheet and the same journey runs once per row, each with its own full evidence. Accepting and rejecting rows are generated together, so a limit is exercised on both sides of its edge rather than only where it works.

RowDataExpects
Standard, in policyAmount '250000' · Roof year '2015'to reach the end
Over the limitAmount '2500000' · Roof year '2015'to be stopped, with a message
Roof too oldAmount '250000' · Roof year '1974'to be stopped, with a message
Boundary, on the limitAmount '1000000' · Roof year '=today-25y'to reach the end

=today-25y is stored as an expression rather than a date, so a row that sits on a boundary today still sits on it next month. A literal date would quietly stop testing the boundary.

02

Cases that continue where the last one stopped

A real regression pack is not a list of independent cases: it signs in once, then walks through a policy, a quote, a document. Later cases keep the same browser session, so the pack reads the way a tester works. Step 1 below is that, in a real run.

A chained case, continuing an earlier sessionrun-f999d57a · 23 September 2026 · 6 steps · 21.5s
StepActionResultWhat the engine recorded
1noteSKIPPEDContinue from previous step on same browser window
2clickPASSclick -> href-first goto -> /dashboardstrategy click.href-first · 668ms
3readvarPASSreadvar -> page3 = "PG-01" (definition-list)strategy readvar.definition-list · 2ms
4readvarPASSreadvar -> session3 = "45b3285a" (definition-list)strategy readvar.definition-list · 0ms
5assertvaluePASSassertValue -> "PG-01" equals "PG-01" (text comparison)
6assertTitlePASSassertTitle -> "Dashboard | Fixture"

Steps 3 and 4 read values out of the page into variables, and step 5 asserts one against what was recorded earlier — which is how a suite checks continuity rather than just absence of errors. Both are runs against our own test fixtures, so the URLs are local and the titles say “Fixture”. The statuses, strategies and wording are exactly as the engine wrote them.

03

What differs from the last run

A count of passes tells you almost nothing on a regression pack — it was passing last time too. These are the verdicts the screen renders, in its own words.

METreached the end of the flow

the row did what was expected of it

METstopped where the story said it would

a reject row that was rejected — the application agreed with the requirement

DIFFERSexpected to reach the end, but stopped at step 4

a regression: something that used to complete no longer does

DIFFERSexpected to stop, but the flow carried on

a rule that is no longer enforced — the quieter and more expensive kind

The last one is why this page exists. A rule that has stopped being enforced does not fail a test — the flow completes, and a pass/fail count calls it green. It is only visible if something knew the row was supposed to be stopped.

What makes two runs comparable at all

  • The suite carries a content hash, so a change in the tests is distinguishable from a change in the verdict.
  • Each step compiles the same way every time, so a difference between runs is a difference in the application.
  • Every resolved element records the strategy that found it, so a step that started resolving differently is visible.

Bring the pack in from the spreadsheet it lives in, or see what it takes to run this behind SSO.

The same product, from another angle