38 posts tagged with "CI/CD"

The argument goes like this.

Coding agents write better code than they did a year ago. Bug density is falling. When something does break, a monitoring agent flags it, another agent reads the trace, writes the fix, opens the PR, and ships. Minutes, not sprints.

So why pay for a test suite? Why staff QA? Why maintain a regression pack that takes two hours to run and breaks every time someone renames a div?

For a lot of small teams, this isn't a hypothetical. It's already how they ship. And I don't think they're being reckless.

I think they're running a specific bet, and the bet only holds under specific conditions.

"Is staging green?" is a Slack question. Until today, answering it meant leaving Slack: open the dashboard, hunt for the last run, or ping whoever kicked it off.

O2 now runs from Slack. Type /devassure, pick a job, and the result card lands in the same conversation.

Every team that gets serious about end-to-end testing eventually builds the same thing: a pipeline that takes a pull request, spins up browsers somewhere, runs the suite, and reports back. In 2026 the default stack for this is Playwright on GitHub Actions - no Selenium Grid hub/node topology to babysit, because Playwright's own worker model plus CI-level sharding is the grid.

This guide builds that setup end to end:

Then we do the part most tutorials skip: what happens at 1,000+ tests, what the infrastructure actually costs in dollars and engineer-hours, and why flakiness is an environment problem before it's a test problem.

There's a number nobody puts in their testing ROI deck: the cost of keeping the test infrastructure alive.

Not writing tests. Running them. The Selenium grid that needs a Chrome bump every six weeks. The CI runners that queue for twenty minutes at 5 PM. The Docker image someone built in 2023 that nobody dares touch. Ask any engineering team where their automation effort actually goes, and "maintaining the thing that runs the tests" is quietly near the top.

Today we're shipping the fix: O2 Cloud Agent. Your tests execute in real browsers on infrastructure we run — nothing to install, nothing to configure, nothing to maintain.

For most of software history, writing code was the bottleneck and testing was the thing squeezed around it. That balance has flipped. AI coding agents — tools like Claude Code, GitHub Copilot's agent mode, and Cursor — can now scaffold a feature, write the surrounding logic, and open a pull request in the time it used to take an engineer to read the ticket. The code isn't the constraint anymore. Verifying that the code actually does what it's supposed to is.

That's the real argument for AI in testing, and it's worth being precise about it: it isn't "AI is trendy, so apply it everywhere." It's a capacity problem. If code generation gets 5–10x faster and test creation doesn't, testing becomes the new bottleneck — or worse, it gets skipped, and the speed gains show up in production incidents instead of shipped features.

An AI coding agent does not know its code works. It infers this from signals — the same way a junior engineer infers a change is safe to merge because the test suite went green, not because they proved it mathematically. The quality gap between a coding agent that "looks done" and one that is actually done comes almost entirely from how well its architecture is built to gather, trust, and act on those signals. That architecture — not the underlying language model — is where most of the meaningful differentiation between coding agents currently sits, and it's worth understanding in some detail when building with these tools or evaluating which one to trust with production code.

Manual QA doesn't scale with a growing codebase. Every new feature adds another set of paths someone has to click through by hand, and every release means re-checking that nothing old broke. Test automation exists to break that tradeoff — but only if it's built the right way. Here's a practical, in-depth look at what test automation actually involves, where teams typically go wrong, how to think about tooling choices, and what's changed recently with AI-driven approaches.

If a build breaks in an obvious way — the login page won't load, the checkout flow throws a 500, the dashboard is a blank white screen — you want to know in minutes, not after a full regression suite finishes hours later. That's the job of a smoke test.

Test-Driven Development (TDD) is one of those practices everyone agrees with in principle and skips under deadline pressure. The reason is usually mechanical, not philosophical: writing the test first means maintaining test code before the feature exists, and that upfront cost is what teams cut when a sprint runs long.

This guide walks through a version of that workflow that removes most of the mechanical cost. Instead of writing test code, you write test cases as plain-English steps — in a spreadsheet or in TestRail — and DevAssure's O2 agent executes them against a real browser. You still get the core discipline of TDD (write the test first, watch it fail, build until it passes, refactor with a safety net), but the "test code" is a row in Excel or a case in TestRail instead of a Playwright script.

One clarification up front, in the interest of not overselling this: classic TDD, as Kent Beck described it, is a unit-level practice — a developer writes a small, fast, in-process test for a function or class before writing the function itself, and the red-green-refactor loop happens in seconds. What this guide describes is closer to acceptance TDD (ATDD) — writing the acceptance criteria for a user-facing flow before the flow is built, then using O2 to validate the whole feature end-to-end in a browser. It's a genuinely useful discipline, and it fits naturally with tools like TestRail that already store acceptance criteria in natural language. It is not a replacement for unit tests on business logic, and this guide isn't claiming it is — the two operate at different layers and different speeds, and most teams doing this well run both.