Skip to main content
Hashnodenebulablogsnebulablogs

Command Palette

Search for a command to run...

From Finding Bugs to Fixing Them: How Far Has AI Vulnerability Remediation Come?

Comparing reports from frontier AI labs, AI startups and enterprises as of Aug 2026

Updated
•13 min read•View as Markdown
From Finding Bugs to Fixing Them: How Far Has AI Vulnerability Remediation Come?
N
Nivethitha Kumaravelu

By August 2026, frontier labs (Google DeepMind, OpenAI, Anthropic) and AppSec startups have all shipped some version of autonomous vulnerability remediation, and every one of them stops at the same gate.

None of the inspectable systems take ownership of the merge. That is the current state.

This post compares what the labs built against what the startups shipped: which loops each closes, where they sit on an autonomy scale, and what would have to change before that gate comes off.

"Autonomous" is five jobs, not one word

A scanner that opens a pull request is not the same object as an agent that discovers a bug nobody handed it, proves it, writes a root-cause patch, and rolls that patch into production. Both get called autonomous remediation.

The public record is clearer if the work is scored in stages:

  1. Discovery. Identify a vulnerability that was not supplied as a CVE, scanner alert, or crash.

  2. Patch generation. Change source, dependencies, or configuration. Advice without a diff does not count.

  3. Autonomous validation. Rebuild, rescan, replay a proof, run tests, fuzz, or otherwise produce evidence that is not the model grading its own homework.

  4. Production deployment. Merge or apply the change in a real environment. A draft PR is not this stage.

  5. Closed-loop remediation. Intake → patch → independent validation → deploy → watch → roll back, running continuously under policy.

Two collapses show up in almost every pitch. Supplied-finding repair is not autonomous discovery. GitHub, Snyk, and Semgrep usually start from CodeQL, Dependabot, Snyk, or Semgrep alerts. Retesting is not patching. A system can prove an engineer's mitigation worked without ever writing the fix.

The rest of this map is what frontier labs and startups have actually demonstrated against those stages, through August 2026.

Finding: What the frontier labs built

CodeMender:

Google DeepMind introduced CodeMender on 6 October 2025. Gemini reasoning sits behind a debugger, source browser, static and dynamic analysis, differential testing, fuzzing, and SMT-style checks, plus a critique agent that diffs original and modified code. That is closer to a security engineer with a very fast intern than a chatbot with a "fix" button.

The published result: 72 security fixes upstreamed in six months, including repositories up to 4.5 million lines. Every patch was reviewed by human researchers before submission. There is no public attempt count, rejection rate, or false-fix rate. "Upstreamed" is stronger evidence than "generated." It is still not autonomous deployment.

In July 2026, DeepMind introduced Gemini 3.5 Flash Cyber to find, validate, and patch through CodeMender. Google says it already runs on internal Chrome, Android, Cloud, Ads, and YouTube code. Access is governments and trusted partners. No merge rate or unattended-production claim is published.

Codex Security:

OpenAI announced Aardvark on 30 October 2025 and shipped it as Codex Security in research preview on 6 March 2026. The pipeline is threat-model the repo, scan commits against that model, try to trigger findings in an isolated sandbox, then attach a Codex-generated patch for human review. It does not lean on classic fuzzing or SCA. It reasons over the code the way a researcher would, then asks a sandbox whether the bug is real.

The numbers that travelled:

  • 92% identification on an unpublished set of "golden" repositories

  • 1.2 million+ commits scanned in a 30-day beta window

  • 792 critical and 10,561 high-severity findings in that window

  • 14 assigned CVEs

Read those as volume and as a vendor-controlled recall figure, not as a patch-success rate. OpenAI's own docs are explicit: fixes are proposed for review. There is no published patch precision, regression rate, or unattended merge. The later Daybreak materials still describe a human-led process that deduplicates findings before maintainers see them.

Claude Security:

Anthropic launched Claude Code Security in limited research preview in February 2026. It reasons over the codebase rather than matching known patterns, re-examines findings in multiple stages, assigns confidence and severity, and suggests targeted patches. The product line is blunt: nothing is applied without human approval.

The volume around it is larger than the product page. Project Glasswing reported that Claude Mythos Preview and about 50 partners found more than 10,000 high- or critical-severity issues. Of 530 high/critical bugs disclosed to open-source maintainers, 75 had been patched at publication. Separately, Claude Security was used to patch more than 2,100 vulnerabilities in three weeks. Anthropic notes the speed came because enterprises were fixing their own code. A Government of Alberta case study is the most concrete production-adjacent writeup: about 50 Claude Code agents scanned 466 million lines in 20 hours and often generated, tested, and built fixes. Engineers approved every patch before it shipped.

Glasswing's useful finding is not the 10,000. It is the bottleneck. Several maintainers asked Anthropic to slow disclosures. Average time to patch a high/critical Mythos finding was about two weeks. Discovery is no longer the scarce resource. Verification, disclosure, and a trusted fix are.

Microsoft sits next to this category, not inside it

Codename MDASH is a multi-model scanner for zero-days and logic flaws. Microsoft's FAQ is unusually explicit: it is read-only. It does not modify environments, open PRs, or push remediation. A separate developer-invoked Copilot flow can propose a function-level fix for review. MDASH is a discovery system. It is not repair.

Finding: the startups implemented the same gate in the developer workflow

Frontier labs proved the agent can produce an accepted patch. AppSec vendors put that pattern on the alert queue teams already have.

System Starts from What it automates How it checks the patch The gate
GitHub agentic autofix CodeQL or third-party code-scanning alerts Explore files, propose a fix, iterate, open a draft PR (typically 2-4 minutes) Reruns the original analysis Human review / merge
Snyk Remediation Agent Snyk SCA or SAST finding Scan → plan → fix → verify, including CLI --auto-approve for local edits Rescan plus existing application tests Human merge; documented goal is mergeable PRs, not a demonstrated default
ZeroPath Autofix ZeroPath's own scanner Generate a fix branch; approve patch opens the PR Claims compile, tests, and rescan; public denominators are thin Explicit generate vs approve; human merges
Semgrep Autofix Semgrep finding Branch, apply AI changes, open a draft PR/MR No general exploit or regression oracle documented Human review / merge

GitHub also lets Dependabot alerts be assigned to Copilot, Claude, or Codex. The agent can update calling code around a dependency bump. Review is still required.

Two details matter more than the vendor column.

Scanner disappearance is a weak oracle. GitHub rerunning CodeQL, or Snyk rescanning until the alert goes away, proves the original detector is quiet. It does not prove a variant of the bug is gone, or that a retry policy, a rate limit, or an auth boundary still behaves the way production needs. ZeroPath is the vendor that most clearly claims compile + tests + rescan together. It has not published a reproducible attempted/passed/merged denominator.

--auto-approve is not production autonomy. Snyk's CLI flag removes per-change prompts on a local run. That is a workflow convenience for an engineer who is already at the keyboard. It is not an unattended merge into main.

Other types of autonomous security systems

The repair loop is find or intake → generate a patch → validate the patch. That is CodeMender, Codex Security, Claude Security, GitHub agentic autofix, Snyk, ZeroPath, and Semgrep Autofix. In every inspectable case, a human still accepts the change.

The offensive-validation loop is attack → report → retest after someone else fixed it.

  • XBOW runs autonomous black-box testing against live applications, posts validated findings, and retests after remediation. Its Microsoft integration is described as a continuous offense/defense feedback loop. XBOW itself says workflows that feed findings into automated remediation are still evolving. There is no public general source-patch generator.

  • RunSybil continuously models and attacks applications and infrastructure, chains weaknesses, and checks exploitability. Public evidence covers discovery, attack-path validation, guidance, and retesting, not autonomous software repair.

  • ProjectDiscovery Neo is one of the clearest documented validation loops: confirm an issue in an isolated sandbox, optionally open a PR or ticket, and after a human merge, replay the original exploit and keep that check on later deploys. That closes tracking and regression. It does not establish that Neo generally synthesizes, merges, and deploys source patches.

Calling any of those "closed loop" is accurate for testing. It is not closed-loop software repair.

The operations patch loop is inventory known CVEs → deploy a vendor-supplied update → verify → roll back. Products such as Ivanti Neurons can do this in production, with ring rollouts. That is genuine closed-loop remediation for a narrower task: the patch already exists. It is not AI-generated source repair, and it is not zero-day discovery.

Agentic SOC platforms sit in a fourth loop (investigation, detection engineering, scoped containment). They are not evidence for source-vulnerability patching, even when the marketing word is "remediation."

Current state: an autonomy ladder, and where 2026 products sit

The useful scale is evidence quality and blast radius.

  1. Advice only (a suggested fix with no diff).

  2. Generated diff.

  3. Scanner-verified diff.

  4. Executably validated draft PR.

  5. Policy auto-merge in low-risk scopes.

  6. Ring deployment with health checks and rollback.

  7. Continuous novel-vulnerability closed loop.

Most source-remediation products are at levels 3-4. Known-patch endpoint management can reach level 6 for vendor updates. Level 7 is not publicly established for novel source flaws.

That is also the honest picture of what is usable now. The architecture that matches the evidence is an evidence-producing agent inside a governed delivery system: deterministic discovery and tests where they already work, agentic root-cause work where scanners miss, independent verification, and a human or policy gate sized to blast radius.

Mapped onto the products:

  • Supplied-finding repair on a backlog the scanner already owns. GitHub agentic autofix, Snyk, and Semgrep Autofix are the shortest path from an existing CodeQL, SCA, or SAST queue to a reviewable PR. The fit is low-breakability dependency bumps and isolated services.

  • Lab agents for the bugs scanners are bad at. Codex Security and Claude Security are built for repo-specific logic and "this only breaks under these conditions" findings. The sandbox trigger or multi-stage verification is why the diff is worth review time. The diff still gets reviewed.

  • Offensive retest after merge. XBOW, RunSybil, or Neo can prove the deployed fix actually closed the hole, including variants. That is the half of the loop the repair products mostly do not own.

  • CodeMender-class hardening stays in the trusted-partner bucket unless the organisation is already in that programme. Upstreamed patches and internal Chrome/Android/Cloud use are real. They are not a self-service merge button.

  • Novel source patches still sit behind a gate. The labs with the most compute independently kept it there. A confirmation dialog that only shows a tool name is not a gate. A PR with a diff, a rationale, and the commands that were run is.

What remains unproven, and what the next step actually is

The unsolved problem is not code generation. It is confidence under incomplete specifications and production consequences.

General patch correctness. Passing the original scanner or proof can overfit the symptom. Few vendors publish variant-attack, fuzzing, differential, and full regression results together.

Independent denominators. "72 upstreamed," "10,561 findings," and "2,100 patched" are success counts. Attempted-patch totals, rejection rates, escaped regressions, and time-to-rollback are still mostly unpublished.

Unattended acceptance and rollout. The human gate is policy, not a missing feature. Draft PRs are widely demonstrated. Autonomous canary rollout, causal regression detection, and automatic rollback for AI-authored source diffs are not.

Adversarial robustness. These agents ingest untrusted repositories, issues, build logs, and tool output. Public product evidence rarely quantifies prompt-injection or poisoned-test resistance. A system that finds vulnerabilities in untrusted code is, structurally, a system that reads untrusted code.

The volume problem Glasswing named. Models can now find more high-severity issues than maintainers can verify and patch. A remediation agent that dumps unvalidated diffs onto the same queue makes that worse. The near-term win is not more findings. It is fewer, better-evidenced patches.

The plausible next rung is not level 7. It is level 5 in narrow scopes: policy auto-merge for low-blast-radius changes that already have an executable oracle. A dependency bump with a clean rescan and green tests. A linter-equivalent SAST fix. A config hardening with a deterministic check. Ring deployment (level 6) is the adjacent operations problem, and known-patch tools already know how to do rings. Nobody has published the combination for novel, agent-written patches.

If a vendor claims they have, the question is which of discovery, proof, patch, verification, and deployment happens without a person, and what independent evidence sits on the PR.

Putting it together

Frontier labs showed that discovery, patch generation, and non-trivial validation can produce accepted real-world fixes at useful scale. Startups put that loop on the scanner queue. Offensive platforms closed a different loop: prove it, then prove the fix held. Known-patch management already closes production rollout, for patches a vendor already wrote.

The shared decision across all of the source-repair products is the finding. A reviewed, evidence-backed diff is operationally useful now. An agent that owns production risk is not what shipped.

What shipped takes mechanical work off the queue. The production decision is still a person's, just smaller, faster, and backed by something that is not the model talking to itself.

References

#cybersecurity

Comments

Join the discussion

No comments yet. Be the first to comment.

More from this blog

N

nebulablogs

50 posts

Technical deep dives into cloud security, DevSecOps pipelines, threat modeling and AI security.