AI-generated code security: how to audit what your agent wrote

AI-generated code is not reliably safe: in Veracode's 2025 test, models chose the insecure option in 45% of coding tasks. You can audit it with your own coding agent if the audit is structured. Cloudflare's open-source security-audit skill runs six phases and has a second agent try to disprove every finding. A code audit tells you where to look. An attack on the live app tells you what a stranger can do.

The team behind Keybrum's attacksUpdated 9 min read

In short

  • Generated code usually fails by omission: the route works, and the check that the record belongs to the caller was never written. Tests pass and nothing looks wrong.
  • Asking your agent to "review this for security" gives a list of maybes. A structured audit separates the agent that hunts from the agent that validates.
  • Cloudflare's security-audit skill is free and MIT licensed. It runs in your own coding agent, on your machine, in six phases, and writes its findings as records you can check.
  • Cloudflare says a single run found roughly half of the vulnerabilities that repeated runs found in total. Run it more than once.
  • A code audit reads the repository. It cannot see a database policy changed in a dashboard or a key added in the host's settings. Proof comes from the deployed app.
On this page

Is AI-generated code secure?

Not by default. For its 2025 GenAI Code Security Report, Veracode gave more than 100 language models 80 coding tasks that could each be completed in a safe or an unsafe way. The models chose the unsafe one in 45% of the tasks, and newer models were no better at it than older ones, even as they wrote code that compiled more often.

That gap is the whole problem of AI-generated code security. The code runs, the screen looks right, and the flaw is in something the code does not do. The bug we see most often in apps built with coding agents is a route or a policy that checks the caller is signed in and never checks that the record belongs to them. No test fails. The vibe coding security risks guide covers the wider picture.

Why code written by an agent needs a different review

Code review was designed for a colleague who writes two hundred lines and can explain each one. An agent changes the terms in three ways.

  • Volume. An afternoon with Cursor or Claude Code produces more code than a person reads carefully in a week. A review that samples will miss the one handler without a check.
  • Confidence. Generated code is tidy, commented and consistent. It reads as finished. The missing ownership check does not look like a mistake, because nothing on the page is wrong.
  • Defaults. The agent reaches for whatever makes the feature work: a service key to get past a permission error, a policy that allows every signed-in user, an environment variable with a public prefix. Each one works at once, and each one is a hole.
  • No memory of intent. The agent that wrote the admin page last week does not remember that the export route was meant for admins only. Nobody wrote that rule down.

So the review has to be systematic, cover everything, and hunt for absences. That is tedious work for a person and reasonable work for an agent, as long as the agent is not asked to mark its own homework.

What to look for first in generated code

WhereWhat the agent often leaves outRead more
Every API route and Server ActionThe session check inside the handler, then the ownership check on the recordAPI penetration testing
Database migrationsRow level security on a new table, or a policy that is just trueSupabase RLS
Client code and env filesA secret key read through a VITE_ or NEXT_PUBLIC_ variable, which ships to every visitorAPI keys exposed in the frontend
Edge and cloud functionsThe service role key used for a query that should run as the callerSupabase service role key
Webhook handlersThe signature check on the incoming eventStripe webhook security
AI routesA session, a quota and a budget on the endpoint that calls the modelUnprotected AI endpoints
Update handlersA list of the fields a user may change, so role or plan cannot be set from the clientIDOR and broken access control

Cloudflare's security-audit skill: what it is

In 2026 Cloudflare published the skill it started from when it built its internal vulnerability discovery system. It is a set of Markdown instructions and two small validators that turn a coding agent into a security auditor. It is open source under the MIT license, at github.com/cloudflare/security-audit-skill, and Cloudflare's engineering blog explains how the larger system grew out of it.

What makes it more than a prompt is its discipline. The skill does not ask one agent for a list of problems. It splits the work between isolated agents and forces every claim through a second reader. Its six phases:

  1. 1

    Reconnaissance

    Agents map the architecture, the trust boundaries and the input surfaces, and write them to architecture.md and a coverage ledger.

  2. 2

    Coverage-led hunting

    Separate hunters are assigned parts of the ledger and one class of attack each. Coverage critics look for what nobody examined.

  3. 3

    Candidate validation

    Every candidate goes to a fresh verifier whose job is to disprove it.

  4. 4

    Structured output

    Results are written to findings.json and checked against a schema by a validator script.

  5. 5

    Independent record verification

    Fresh agents check the final claims against the source once more.

  6. 6

    Reporting

    The reports are derived from the verified records, including a separate file for what still needs validation.

Each record ends with one of three verdicts, and the difference between them is the most useful part of the design.

VerdictWhat it meansWhat you do
confirmedA complete trace through the source, with a bounded observed resultFix it
needs_validationA lead grounded in the source with one exact fact unresolved, and no severityResolve the fact, often by testing the running app
rejectedA candidate that a verifier disprovedNothing. It is kept so the next run does not raise it again
The agent that checks a finding is never the agent that found it.
Design principles, security-audit skill, Cloudflare

How to run a security audit with your own agent

The skill needs a coding agent whose model supports tool use and parallel sub-agents, and Node.js for the validators. Install it with the Skills CLI, then ask for an audit in plain words.

bash
# Install the skill for your coding agent
npx skills add https://github.com/cloudflare/security-audit-skill \
  --skill security-audit

# Then, in your agent, inside the repository:
#   security audit this codebase
By default the reports are written outside the repository, under a security-audit-skill folder in your home directory, one sub-folder per run.
  • Run it more than once. Runs are additive: the skill reads earlier ledgers and findings and aims at the gaps. Cloudflare reports that a single run found roughly half of the vulnerabilities that repeated runs found in total.
  • Expect it to refuse to execute your code without a sandbox. If the agent has no isolated environment with networking disabled, the skill keeps a lead as needs_validation instead of running the target. That is the right behaviour on a laptop that holds your keys.
  • Read `NEEDS-VALIDATION.md` as carefully as the confirmed list. On a web app, many of the interesting leads end there, because the answer depends on how the deployed database is configured.
  • Budget for it. Six phases with parallel agents use far more tokens than one review prompt. Start with the directory that handles auth, data access and payments.

Code audit or live attack: why you want both

A code audit and an attack on the deployed app answer different questions. Neither one covers the other.

Code auditLive attack
ReadsThe repositoryThe deployed app, from outside
FindsA missing check, an unsafe query, a secret in a file, a flaw in logic that no request revealsWhat a stranger or a second account can actually read, change or call
MissesAnything configured outside the code: a policy edited in a dashboard, a bucket made public, a key set at the hostCode paths it cannot reach, and flaws that need knowledge of the source
A finding isA reasoned claim about the sourceA request that worked, with the response
Typical false alarmA check exists in a layer the agent did not connectFew, when findings require proof
RunsOn your machine, when you askOn every deploy

The two meet at needs_validation. The audit says: this table is created without row level security in the migration, and I cannot tell whether a policy was added later. The live attack asks the database with the public key and gets rows, or does not. One request settles it. The white box pentest guide explains why testers with source access still attack the running app.

Running the audit through Keybrum

Cloudflare's skill is part of Keybrum's code audit, with full credit to its authors. Keybrum's MCP server gives your coding agent two tools for it. start_source_audit hands the agent the skill's files and the instructions to run it on your machine. submit_source_findings sends the resulting findings.json back when the last phase is done and the validator passes.

Your code never leaves your machine. Only the final records are sent. Keybrum then compares each record with what its attack proved on the deployed app: a record about a table, a route or a bucket is matched to the live finding on the same object, and the next attack run tries the records that still need validation. A code claim with a live request behind it stops being a maybe.

Keybrum only attacks an app after its owner proves ownership. A one-time report costs €49. Plans, from €29 a month excluding VAT, attack every deploy and keep retests open.

The limits of auditing AI code with AI

  • It can be wrong in both directions. The validation phases cut false alarms. They do not make a confirmed record a proof, and a clean report does not mean the code has no flaw.
  • It shares blind spots with the author. A model that tends to forget an ownership check when writing may also overlook it when reading. Separate agents help. A different model for validation helps more, and Cloudflare makes the same point about its own system.
  • It does not know your business rules. Whether a refund may exceed the payment, or a trial may restart, is not in the code unless you wrote it there.
  • It reads one repository. Flaws that cross services, or sit in infrastructure settings, are outside its view.
  • It is not a pentest report. If a customer or an auditor asks for one, see pentest for startups for what they usually accept.

Questions

Can I use AI to audit AI-generated code?

Yes, with structure. A single prompt asking for a security review returns a list of possible issues with no way to tell which are real. A structured audit uses separate agents to hunt and to disprove each finding, and labels what it could not settle. Then test the running app, because some answers are not in the code.

What is the Cloudflare security-audit skill?

It is an open-source skill, MIT licensed, that turns a coding agent into a security auditor. It runs six phases, from reconnaissance to reporting, has fresh agents try to disprove every candidate, and writes each result as confirmed, needs validation or rejected. Cloudflare says its internal vulnerability discovery system grew out of it.

How do I run a security audit with Claude Code?

Claude Code has a built-in /security-review command for a review of your checkout. For a deeper audit, install Cloudflare's security-audit skill with npx skills add and ask the agent to security audit the codebase. Run it more than once: Cloudflare reports that one run finds roughly half of what repeated runs find.

Does an AI code security review replace a pentest?

No. A code review reads the repository and reasons about it. A pentest attacks the running app and reports what worked. Settings made outside the code, such as a database policy changed in a dashboard, only show up in the second. See AI penetration testing.

How often does AI-generated code contain vulnerabilities?

Often enough to review all of it. In Veracode's 2025 test of more than 100 models on 80 coding tasks, the models chose the insecure option 45% of the time. Those were short isolated tasks, not whole apps, and in a real app the larger risk is that nobody reviews the result before it ships.

Is my source code sent anywhere during the audit?

The skill runs inside your own coding agent, so your code goes where that agent already sends it, to your model provider. When you run it through Keybrum's MCP server, only the final findings records are sent to Keybrum. The repository stays on your machine.

Sources

  1. 1.security-audit-skill (README and license), Cloudflare
  2. 2.Build your own vulnerability harness, Cloudflare Blog
  3. 3.AI-Generated Code Poses Major Security Risks in Nearly Half of All Development Tasks, Veracode
  4. 4.Data Security, Next.js Docs
  5. 5.Securing your API, Supabase Docs
  6. 6.Commands, Claude Code Docs

Your app is public. So are its mistakes.

Check what any attacker can already see in it, free, in ten seconds. The full attack, with proof and fixes: €49 once, or €29 a month on a plan.

or continue with Google or GitHub