Blog · Quality

Unclear requirements: ask, don't guess

When a requirement does not say what should happen, testing does not stop. It guesses, silently, and reports a pass. That guess is the most expensive thing in a release, because nobody can see it. We think the honest answer is sometimes a question, and that a question deserves to appear in the report next to the passes and the failures.

Published · ShipperAG team · Free to republish (CC BY 4.0)

Every requirement has an edge it does not describe

The brief says the discount applies to orders over £50. It does not say whether that is before or after the delivery charge, or what happens at exactly £50, or whether a refund that drops the order below £50 claws the discount back. Nobody was careless. Requirements are written by people who know the product, for people who know the product, and the shared assumptions never make it onto the page.

The standards community has been clear about this for a long time. ISO/IEC/IEEE 29148, the requirements engineering standard, exists precisely to define what makes a requirement a quality requirement and to set out the attributes one should have. Real briefs, written under deadline, meet that bar in the middle and drift at the edges. The edges are where the bugs that reach customers live.

A guess and a pass look identical

Here is the part that should worry a release owner. When a tester — a person, a script or a model — meets an edge the requirement does not cover, they make a decision. They pick the interpretation that seems sensible, check that the product does that, and move on. The result reads: passed.

Nothing in that report distinguishes "this behaviour was specified and the product does it" from "nobody specified this, I assumed, and the product matched my assumption". Both are a green tick. The first is evidence. The second is a coin toss that happened to land the same way twice, and it will keep reading green until a customer finds the edge.

This is the failure mode we care most about, and it is not solved by running more checks. More checks against an unclear requirement produce more confident-looking guesses.

The honest answer is sometimes a question

We think a quality report needs a fourth state alongside passed, failed and not covered: needs input. It means the checker reached a point where the requirements were unclear or contradictory, and stopped rather than invented an answer.

That state is uncomfortable to design for. It makes a report look less finished. A run that says "18 verified, 3 issues, 2 questions for you" is less satisfying than one that says everything passed. But it is the only version that tells the truth, and it puts the decision in front of the person who can actually make it — the product owner — at the moment it is cheap to make, rather than after release when it is not.

It also changes what a sign-off means. If you are recording a release decision on a UAT sign-off, the open questions belong on it. They are not admin. They are the list of things your team has agreed to ship without deciding.

Why this matters more now

Code is increasingly written by tools that are very good at producing something plausible for an underspecified request. In Stack Overflow's 2025 developer survey, the single biggest frustration developers reported with AI tools was "AI solutions that are almost right, but not quite", named by 66% of respondents; 45.2% said debugging AI-generated code is more time-consuming. In the same survey more developers distrust the accuracy of AI tools (45.7%) than trust it (32.7%).

"Almost right" is what you get when a generator fills an unspecified edge with a reasonable guess. If the checking layer fills the same gap with its own reasonable guess, the two guesses agree and the gap closes over without anyone seeing it. That is why we think a checker must be able to say it does not know — and why the same discipline applies whether a person, a script or a model wrote the code. There is more on the practical side of this in our guide to testing AI-generated code.

What we're building

ShipperAG checks a release against what it is supposed to do, re-checks what it finds, and reports what it could not settle instead of guessing. Open questions are a first-class result, not a footnote. The product is in a private pilot through 2026; you can see how it works at a glance, or score your next release yourself with our free release readiness checklist.

Sources, checked 23 September 2026: ISO/IEC/IEEE 29148-2018, Systems and software engineering — Life cycle processes — Requirements engineering (IEEE Standards Association); Stack Overflow Developer Survey 2025, AI section.

Free to republish. This article is licensed under Creative Commons Attribution 4.0 (CC BY 4.0). Copy it, share it or adapt it, as long as you credit ShipperAG and link to this page.

Free 45-day trial · waitlist open

Ship with evidence

ShipperAG is in a private pilot. Join the waitlist to become a product partner and get evidence you can sign for a real release.

  • Free 45-day trial, no card
  • Up to 20 release checks on your product
  • Direct line to the founders

Work email only. We keep your email, team size, plan choice and the page you joined from, only to contact you about the ShipperAG pilot. No spam.