← BackShare your ideasSubscribe

Published

Manifesto for testing in the age of AI

Vu Lam · Cristiano Caetano · Alex Martins · Mush Honda

AI is changing how software is created - and how it must be tested.

Code can now be generated in moments. Tests can be created, executed, analyzed, and maintained with increasing autonomy. But more code, more tests, and faster delivery do not automatically produce better software.

AI accelerates testing. People lead quality.

This is what we commit to.

01

We will earn confidence - not manufacture it

Speed does not equal quality. Test volume does not equal coverage. A passing test does not prove that a system is ready.

Confidence must be earned by examining what was tested, what was assumed, what was omitted, and whether the evidence addresses the business risks that matter.

We will measure assurance by the strength and relevance of the evidence - not by the number of tests generated or checkmarks turned green.

02

We will keep people accountable for quality

Machines can generate code, tests, analysis, and evidence. They cannot be accountable for the consequences of what ships.

People must continue to define what quality means, determine which risks matter, challenge assumptions, and decide when the evidence is sufficient.

Quality belongs to the entire software organization. But making testing everyone’s responsibility does not make everyone an experienced tester. Human judgment, skepticism, critical thinking, and domain experience remain essential - especially as machines act with greater autonomy.

03

We will evolve the craft - not preserve its mechanics

AI will absorb more of testing’s repetitive mechanics: test creation, execution, analysis, and maintenance. Some testing activities - and some roles built around them - will disappear.

The mission of testing will not.

It will expand toward understanding intent, exploring uncertainty, modeling risk, challenging systems, and protecting the end-user experience. Practitioners must combine testing judgment, domain expertise, critical thinking, and the ability to direct and evaluate AI.

Manual exploration, deterministic automation, and autonomous testing will coexist. We will use each according to its purpose and the risks involved - not according to fashion.

04

We will give AI the context - and scrutiny - it requires

AI is only as trustworthy as the context that guides it.

Clear requirements, source code, test cases, test results, production behavior, architecture, system boundaries, constraints, and human domain knowledge are becoming the new testing infrastructure. As implementation accelerates, structure and shared understanding become more important - not less.

AI also introduces new failure modes: hallucination, inconsistency, bias, omission, and false confidence. We must test the intelligence, not only the software it helps produce.

05

We will pursue application success - not testing activity

The purpose of testing is not to produce more tests. It is to help create software that succeeds for the people and organizations that depend on it.

Functional correctness is only part of that responsibility. Quality also means that software is usable, accessible, secure, resilient, performant, and worthy of trust.

We will judge testing by the risks it reveals, the decisions it improves, and the outcomes it protects. And we will shape its future through open experimentation, practitioner experience, and measurable evidence - not autonomous-testing hype.

Our call to the testing community

The future of testing should not be dictated by AI vendors, defended by those protecting the past, or left entirely to machines.
It should be shaped by the practitioners who understand what quality demands - and who are prepared to evolve how it is achieved.

We are not here to preserve testing as it was.
We are here to lead quality into what comes next.