51 posts tagged with "AI-Agentic Test Automation"

Playwright is one of the best browser automation frameworks ever built. It is fast, reliable, developer-friendly, and powerful. If your team knows exactly what to test, how the UI is structured, which locators are stable, and who will maintain the tests forever, Playwright is an excellent choice.

But that is also the problem.

Most product teams do not struggle because Playwright is weak. They struggle because writing and maintaining browser tests is still manual engineering work:

• Every test needs scripts.
• Every script needs locators.
• Every locator can break.
• Every UI change needs test maintenance.
• Every failure needs someone to investigate whether the app is broken or the test is broken.

At some point, the test suite becomes another codebase to maintain.

Agent-based testing changes this model. Instead of writing scripts that tell the browser exactly what to click, type, and assert, you give the agent an objective. The agent understands the application, interacts with the UI, adapts to changes, validates behavior, and reports issues.

That is why developers should start moving away from writing Playwright tests manually.

Every team that adopted vibe coding eventually hits the same wall: code ships faster, but validation does not keep up. This post is the implementation manual for closing that gap on every pull request.

Cursor changed the default for how features get built. What has not changed is that untested code still breaks production — it just gets there faster.

For twenty years, automated testing meant writing more code. That era is ending — and most teams haven't noticed yet.

The first automated test I ever wrote was in Playwright. It was 2012. It launched a browser, filled in a login form, and checked that the dashboard loaded. It passed. I felt like a wizard.

More than a decade later, the fundamental contract hasn't changed. To test software, you write more software. You describe, in code, what your code is supposed to do. Then you maintain that second codebase forever.

We've built entire careers, conferences, certifications, and consultancies on this premise. Selenium. Cypress. Playwright. Test pyramids. BDD. Page object models. The whole apparatus rests on a single assumption: humans must specify, in writing, what to test.

That assumption is quietly dying.

A CTO told me last month, very pleased with himself:

"We're already paying $200/month per dev for Claude. Testing is basically free now — we just ask Claude to also write the tests."

I asked him to pull up his Anthropic bill. The number was 14x what he'd budgeted at the start of the quarter. And his team still hadn't shipped the regression suite.

This is the most expensive trap in the AI tooling stack right now, and it's expensive precisely because it looks free. If you've already bought a coding agent, asking it to do double duty as a testing agent feels like the obvious move. One subscription, one workflow, one bill.

Except there is no "one bill." There are six.

Last week, on a customer call, a CTO asked me the question I now get every single week:

"I'm already using Claude to write my code. Why can't I just point the same agent at the code and have it test itself?"

It's a fair question. If one AI can write a React component, surely it can write the test for that component too. The economics look seductive — one tool, one workflow, one bill.

But here's an insight:

Testing your own PR is like proofreading your own essay. You'll read it 10 times. You'll miss the same typo 10 times. Because your brain autocorrects what it wrote.

That insight is what this blog is about. And it explains why a coding agent, no matter how capable, is structurally the wrong tool to verify its own work.

Modern web applications are becoming harder to automate with traditional scripts alone. Dynamic UIs, asynchronous loading, popups, custom components, visual states, and complex user journeys make browser interaction more than a simple sequence of clicks and text inputs.

Modern software delivery has fundamentally changed. What once followed a slow, linear release cycle: design → development → testing → release—has now evolved into a continuous, fast-moving pipeline powered by Agile, DevOps, and CI/CD practices. In this environment, traditional test automation is no longer enough.

Chatbots have changed the way users interact with software. Instead of clicking through rigid workflows, users now type questions, give instructions, ask for summaries, request charts, upload files, and expect intelligent responses. This shift has created a new testing problem that traditional automation tools were never designed to solve.

For years, teams have relied on tools like Selenium and Playwright to test web applications. These tools work well when the application behaves in a mostly deterministic way. A button click should open a modal. A form submission should show a success message. A table should contain a certain row. In these cases, testers can depend on locators, assertions, and exact text matching.

But chatbot testing is different.