Skip to main content

45 posts tagged with "Quality Assurance"

View All Tags

Your AI Coding Agent Might Be Gaming Its Own Tests

Divya Manohar
Co-Founder and CEO, DevAssure

TL;DR

RepoRescue (arXiv:2607.01213) shows coding agents can chase a green checkmark — including by editing tests — instead of fixing the real problem. For QA, that means scrutinizing test-and-fixture diffs, building production eval harnesses from your own backlog, and keeping an independent testing agent off the coding agent's incentive loop.

A theme has been building in AI-coding-agent research over the past couple of weeks, and it's one that should matter a great deal to anyone doing software testing or QA: coding agents are getting good enough to satisfy the check you wrote rather than the request you meant. That distinction — between passing a test and actually doing the job — is turning into one of the more consequential debates in agentic software engineering right now, and it has direct implications for how testing and QA teams should be evaluating AI-generated code and AI-generated tests alike.

QA Playbook for Autonomous Coding Agents

Divya Manohar
Co-Founder and CEO, DevAssure

TL;DR

2026 is the year agentic testing moved from experiment to baseline expectation. Coding agents generate code and tests faster than humans can review them — shifting the QA bottleneck from writing coverage to validating confidence. The teams adapting fastest aren't replacing testers; they're pairing dedicated testing agents with human quality strategists.

If you've shipped code in the last year, you've probably noticed the shift: AI coding agents aren't just autocompleting functions anymore — they're running in execution loops, writing tests, reading failures, and fixing their own bugs before a human ever opens the pull request. Anthropic's 2026 Agentic Coding Trends Report frames the change plainly: software development is shifting from writing code to orchestrating agents that write code.

For testing teams, that changes the job description as much as the tooling.

Here's what's actually happening in agentic testing and AI-assisted QA right now, and what it means if you're building (or buying) test automation.

Test Suite Passes, Users Still Find Bugs

Divya Manohar
Co-Founder and CEO, DevAssure

Last month I had a conversation with a CTO that stuck with me.

Their team has 3,400 tests. 94% coverage. A CI pipeline that runs on every PR. Tests pass reliably — less than 2% flaky rate. By every industry metric, this is a well-tested codebase.

They also had 6 production bugs in the past 30 days. All reported by users. All missed by the test suite.

I asked him to send me the bugs. Here's what they were:

  1. A modal didn't close when clicking outside it. Users had to refresh the page to dismiss a confirmation dialog.
  2. A price displayed as $1,299 in the cart but charged $12.99. Decimal formatting inconsistency between the display component and the payment API.
  3. The "Export to CSV" button worked on Chrome, broke on Safari. Downloaded an empty file.
  4. A newly added field was editable for admins but displayed as read-only for regular users — the opposite of what it should have been. Permission logic was inverted.
  5. A search that returned 0 results showed the previous results instead of an empty state. Stale state from a React component not resetting.
  6. The onboarding flow skipped step 3 entirely when the user's timezone was UTC+0. A conditional that checked for a truthy timezone value — and 0 is falsy in JavaScript.

None of these are exotic edge cases. Every one of them is something a human using the app would hit within 5 minutes.

And none of them were caught by 3,400 tests at 94% coverage.

Why?

Salesforce Testing: Apex vs Selenium vs O2

Divya Manohar
Co-Founder and CEO, DevAssure

Short answer

Apex validates server logic fast but cannot see the UI. Selenium covers user flows but breaks on Shadow DOM, MFA, and every release. DevAssure O2 runs plain-English tests in a real browser — same UI coverage as Selenium without locator maintenance. Use Apex + O2, not Apex + brittle scripts.

Every Salesforce team tests. The question is how — and what slips through the cracks depending on which approach you use.

In this post, I'm going to take a real Salesforce application — Dreamhouse, Salesforce's official sample app — and test the same functionality three different ways:

  1. Apex unit tests (the traditional Salesforce-native approach)
  2. Selenium WebDriver (the industry-standard UI automation approach)
  3. DevAssure O2 Agent (autonomous browser-based testing)

For each approach, I'll show you the actual code, explain what it catches, what it misses, and the ongoing maintenance cost. By the end, you'll have a clear picture of where each approach fits — and where it falls short.

Why We Built DevAssure O2 for PRs

Badri Varadarajan
Co-Founder and COO, DevAssure

Short answer

Engineering velocity is rarely limited by how fast people write code. It is limited by testing friction on pull requests — flaky suites, run-everything CI, and release gates that force a choice between speed and quality. DevAssure O2 was built to validate every PR from intent and impact inside the developer workflow, without maintaining brittle test scripts.

For years, I led teams focused on engineering productivity and release management across startups and enterprises. My primary job was not just shipping features — it was ensuring release trains moved predictably, developers remained productive, and quality gates did not become velocity killers.

One thing became very clear over time:

Engineering velocity was rarely limited by coding speed. It was limited by testing friction.

Shift Left Failed. Autonomous Testing Is What Comes Next.

Divya Manohar
Co-Founder and CEO, DevAssure

TL;DR

For a decade, shift left meant developers write more tests earlier. That overloaded engineers, bloated suites, and barely moved the bug needle. Autonomous testing keeps the timing - tests at the pull request - but changes the mechanism: an agent reads the diff, generates scoped tests, runs them, and leaves nothing to maintain. DevAssure calls this shift smart: AI handles execution; humans handle judgment.

For a decade, the testing industry rallied behind a simple mantra: shift left.

Find bugs earlier. Test sooner. Put quality in the hands of developers.

The theory was sound. A bug caught in development costs roughly 10× less than one found in production. Move testing to the left of the timeline, and you save money, ship faster, and improve quality.

But here is what actually happened:

Why Personas Matter in Testing

Santhosh Selladurai
Co-Founder and CTO, DevAssure

Software testing traditionally focuses on one core question:

Does the application work?

But there is a more important question that often gets ignored:

Manual QA Engineer Cost vs AI Testing

Anush Chandra Shekar
SEO & Digital Marketing, DevAssure

“Hunt those pesky bugs before they haunt your customers.”

One of the most important challenges that every IT company in the United States faces at some point is how to kick-start QA engineer hiring and establish the best QA team that can compete globally. Testing the respective web application and detecting bugs early is one of the most important things that every IT company should ensure. This is where the STLC (Software Testing Life Cycle) jumps in!

How US Startups Cut QA Costs 50% with AI

Anush Chandra Shekar
SEO & Digital Marketing, DevAssure

Software Testing has evolved rapidly over the last decade. From test automation frameworks to AI-powered testing tools, the industry has witnessed a rapid change in how software testing is performed.