147 specs in one run, 70 false failures
Seventy specs went red in one run and passed when run alone. The session expired partway through, so every spec after that point measured a login screen and reported the difference as a failure.
A single run of 147 end-to-end specs produced 70 failures, and none of them were real. Run individually, the same specs passed. The cause was not flakiness in any of them: the browser session dropped roughly halfway through the run, and every spec after that point loaded a login screen, found none of the elements it was asserting on, and reported the difference as a defect. Seventy red specs, one fault, and the fault was in the run rather than in the code under test.
A dropped session in an end-to-end suite fails loudly in every spec that follows it and quietly in none. Each remaining spec navigates to a page it expects to be signed in for, gets redirected to the sign-in form, and then asserts against a document that has no heading, no table and no button it was looking for. The failure message names the missing element, which is true and completely misleading: the element is missing because the page is the wrong page.
The shape to look for is a contiguous block of failures rather than a scatter. Real defects are distributed across the suite, because they follow the code. A run where the first sixty specs pass and everything after them fails is describing something that happened at spec sixty-one, and no amount of reading individual assertions will find it.
Running a failing spec on its own is what separates a test defect from a run defect, and it is worth doing before reading any stack trace. A spec that passes alone and fails in the suite is telling you that something shared between specs changed underneath it: a session, a database row another spec deleted, a server that restarted, a build directory two processes were writing to. None of those are visible inside the spec that reported them.
The working arrangement here was to split the suite into seven chunks and shut the servers down between them, which keeps any shared state from surviving long enough to poison a later chunk. That is a workaround rather than a repair — the real repair is a suite where each spec establishes its own session — but it produces a run whose red specs are all real, which is the property that matters when you are trying to ship.
We split one long numbered prompt into two. Every step after the cut changed number, and the sentences that navigate by those numbers went on pointing at whatever now wore them.
Market4 turns one release note into a changelog page, a blog post, a mail-out and a week of social posts — and then tells you which of them brought anyone back.
| What the run reported | What it looked like | What it actually was |
|---|---|---|
| 70 specs failing from one point onward | Widespread breakage in the application | The session dropped mid-run; every later spec measured a login screen |
| Every route returning 404 on two different ports | The application completely broken | Two development servers sharing one build directory and corrupting the route manifest |
| A spec timing out on its first navigation | A flaky test worth retrying | Cold compilation taking longer than the default 30-second ceiling |
| Fixtures missing and transactions refused | A data-layer defect | The process pointing at a different database from the one holding the fixture |
The second row is worth its own warning. A Next.js development server and a Playwright web server configured to run one both write to the same build directory when they are pointed at the same project. Started at the same time, they corrupt each other's route manifest and both then return 404 for every route, which reads as total application failure rather than as two processes fighting. The rule that follows is dull and effective: stop one before starting the other.
The third row is the cheapest to fix and the easiest to misread. A first navigation that compiles a route from cold can take longer than a default per-test timeout allows, so the first spec in a run fails while the same spec passes on a warm server. Raising the timeout to two minutes for the suite made that class of failure disappear entirely; retrying the spec would have hidden it instead.
The wider lesson from a run like this is that a test suite reports on the environment as much as on the code, and it cannot tell you which of the two it just measured. Seventy failures felt like a catastrophe for about ten minutes. The catastrophe was an expired session, and the useful output of the whole episode was a script that runs the suite in pieces and a note about which port not to leave running.
Because something shared between specs changes during the run: an expired session, a record another spec deleted, a restarted server, or a build directory two processes are writing to. The individual run is the diagnostic — a spec that passes alone is not broken, so the fault is in what the suite does between specs rather than in the spec that reported it.
Look at the distribution. Real defects follow the code and appear scattered across the suite; environment faults appear as a contiguous block from a single point onward. Then read the captured page rather than the assertion message. A failure that says a heading is missing, on a page that turns out to be a login form, is describing the wrong page rather than a missing heading.
It is a workaround, and a useful one while the real fix is outstanding. Splitting the run and restarting the servers between chunks stops shared state surviving long enough to affect later specs, so every red spec in the output is real. The repair it defers is making each spec establish its own session and its own data rather than inheriting either.
For cold-start failures, yes. A first navigation that has to compile a route can exceed a default timeout on a cold server while passing on a warm one. Raising the timeout absorbs that cost and still fails when something is genuinely wrong. A retry passes on the second attempt, records the spec as flaky, and leaves the reason unrecorded.
Publishing a post sends two independent announcements. One IndexNow request reaches Bing, Yandex, Seznam, Naver and Yep. Google takes no part in IndexNow, so it is told by re-submitting the sitemap.
The link measured 297 by 44 pixels and reported a pass. Tapping its centre hit the paragraph behind it. An inline box that wraps is not one rectangle, and the pseudo-element meant to enlarge it was positioned against the same broken geometry.