Keeping Regression Suites Fast Without Losing Coverage

Regression suites grow quietly. Every feature adds a few checks, nobody removes the stale ones, and after a year the full run takes long enough that engineers start skipping it before merging. The fix is rarely more machines — it is deciding what the suite is actually for.

We see the same pattern in very different products. A healthcare booking platform, an iGaming operator running more than twenty sites from one codebase, a banking product with compliance reporting: the suites looked nothing alike, but in each case the team trusted a green build less than they should have been able to.

Why suites get slow in the first place

Nobody sets out to build a slow suite. It happens through small, reasonable decisions:

• A bug reaches production, so someone adds a UI test for it — even when an API check would have caught it in a fraction of the time.

• Tests are written against screens rather than behaviour, so a redesign breaks dozens of them at once and they get patched, not rethought.

• Flaky tests are re-run instead of fixed, and a second run quietly becomes part of the pipeline.

• Nothing is ever deleted, because nobody remembers what a given test protects.

Each of these is minor. Together they turn a safety net into a queue.

Start with what actually breaks production

Before touching the suite, look at incident history and map defects back to the code paths that caused them. In most products a small set of user and revenue flows carries most of the risk, while a long tail of checks protects screens nobody ever escalates about. That map decides what runs on every commit and what can wait for the night.

On the iGaming platform this was easy to see: deposits, withdrawals and player balance were where a defect cost real money, on every brand at once. On the healthcare product it was booking and document upload — the two things patients and clinicians would notice within minutes. Everything else still mattered, just not enough to block a merge.

Rebuild the pyramid around risk

From there the suite splits into layers:

• On every commit: unit, contract and API checks for the critical flows. These should finish while the developer is still looking at the diff.

• On every merge: a short end-to-end UI run across the critical user paths.

• Nightly: the long tail — edge cases, older features, cross-browser and cross-device runs.

• Before a release: the full suite plus agreed exit criteria, so “ready” means the same thing to QA, developers and product.

Teams stop skipping tests when the fast path finishes quickly. The slow checks still run — they just no longer stand between an engineer and a merge.

Push checks down a level whenever you can

A lot of UI tests exist only because the UI was the easiest place to write them. When a check is really about data — a transfer amount, a calculated return, a permission rule — it belongs at the API or service level. On the banking project, moving transfer and reporting checks below the UI and adding contract tests between services made the suite faster and more precise at the same time: when something failed, the report named the service and the contract, not just “the page looked wrong”.

Treat flaky tests as defects

A test that fails one run in ten teaches the team to ignore red builds. We move flaky tests out of the blocking path straight away, give each one an owner, and either fix or delete it within the sprint. It sounds strict, but it is the only way a failing build keeps meaning something.

One suite, many configurations

When a product runs in several variants — brands, regions, white-label clients — the temptation is to copy the suite for each one. That multiplies maintenance with every copy. For the iGaming platform we built a single shared suite with per-brand configuration, running from one pipeline. A fix to a test is made once and applies everywhere, and launching a new site means adding configuration, not another suite to look after.

Keep the suite honest over time

Restructuring once is not enough; suites drift back unless someone looks after them. A few habits help:

• Revisit the risk map whenever incidents or the roadmap change.

• Watch run time in the pipeline the same way you watch build health.

• When a test is added, write down what it protects. When nobody can answer that for an old test, it is a candidate for removal.

The goal is not the highest coverage number. It is a suite the team runs without thinking about it, and a green build they actually believe.