AI-generated code often ships with security holes — how to catch them before launch

AI is genuinely good at writing code that works. It is not, by default, good at writing code that's safe, and those are different skills. When you describe a feature, the model optimises for "does the task," not "can't be abused," so the result usually runs fine and quietly carries a few holes. Study after study has found that a large share of AI-generated code, often cited around 45%, ships with some kind of security weakness.

That sounds alarming, and I'd rather it be reassuring instead, because the fix is mostly a short, checkable list. You don't need to become a security engineer. You need to look in the few places where the damage actually happens. Let me show you where.

Why AI skips security

It's not that the model is careless. It's that security is usually invisible in the prompt. You ask for "a form that saves a message," and it gives you exactly that, a form that saves a message. You didn't ask it to reject hostile input, or to check that the saver is allowed to save, so it didn't, because that wasn't the task.

So the gaps aren't random. They're the predictable result of security being unstated. Which is good news: predictable gaps are exactly the kind you can check for with a list.

The big four

Most real breaches come from a small number of issues. Cover these four and you've handled the large majority of the risk.

Exposed secrets. API keys, passwords, and tokens that ended up in the code or a commit. This is the most common and one of the most damaging. Keep them in environment variables, never in the source, and if one has ever been committed, rotate it, because the history keeps it forever.

Unvalidated input. Anything a user can type is something an attacker can abuse. AI-generated code often trusts input that should be checked. Validate and sanitise what comes from users, especially anything that reaches a database or gets shown back on the page.

Missing access checks. The classic AI gap: the code fetches "the document with this id" without checking that the person asking is allowed to see it. Make sure users can only reach their own data. Test it by logging in as one user and trying to open another's thing.

Outdated dependencies. AI often pulls in libraries, and some carry known vulnerabilities. Run your package manager's audit command, update what it flags, and do it again before launch. It's a one-line check that closes a surprising number of doors.

You don't have to do it alone

The reassuring part is that almost all of these are obvious once someone looks. They survive because nobody looked, not because they're cleverly hidden. So the single most effective thing you can do is have a second set of eyes go over the app before you launch, ideally someone who'll poke at it rather than admire it.

That's part of why honest review matters so much for AI-built apps specifically. A reviewer who tries the empty form, the other user's id, and the giant paste will surface in ten minutes what the model quietly left open. Shipyard is built for exactly that, real builders reviewing your work before the wider internet does, which is a much friendlier place to find a hole. We're opening the first 50 founding spots now. If you've shipped AI-written code and want a second set of eyes first, come and get them.