Software Securityhome
Fix a findingSearch
Phase 3 · Development

AI coding assistants

An AI coding assistant suggests or writes code in your editor. A coding agent also reads issues and files, runs shell commands, installs packages and opens pull requests, usually with the developer's own credentials. Using them safely means treating what they write and read as untrusted and limiting what they can run.

Assistants add risks that code review never sees. Models invent package names: Spracklen et al. counted 205,474 unique made-up names in 576,000 code samples from 16 models, and an attacker can register one with a malicious install script. An agent that follows an instruction planted in an issue can run commands or send data out with your credentials before any diff exists.

Who
Every developer who uses an assistant; a platform or AppSec owner maintains the approved tools, managed settings and CI gates
When
Whenever an assistant writes code or an agent runs commands, and before adopting a new assistant, agent mode or MCP server
How often
On every AI-assisted change; approved tools, settings and the MCP allowlist reviewed when tools or models change
You end up with
A written tool and data policy, enforced agent settings, CI gates for new packages, and a named human approver on every agent change

Do this

  1. Decide which tools and data are allowed

    List the approved assistants and account types, and which repositories and data classes may go to which provider (see Data classification). Check each provider's retention and training terms and what context the tool sends: many send open files, terminal output and tool results, not only the current file. For regulated or highly sensitive code, the OWASP Secure Coding with AI Cheat Sheet suggests self-hosted or air-gapped tools.

  2. Review generated code as an outside contribution

    Read every file in the diff, not the agent's summary: agents change files nobody asked for. Check authorization on new routes, SQL and shell commands built from strings, weakened controls such as verify=False or wildcard CORS, and deleted or weakened tests. Give each AI-written change a human owner who approves it, and write security-critical tests for authentication, authorization and input validation yourself. Security code review has the review questions.

  3. Gate every new dependency

    Before installing a package an assistant suggests, check that it exists, is the project you meant and is not newly registered. Run the same check in CI on every manifest change, keep an internal allowlist of approved packages, and add a release-age cooldown, such as npm's min-release-age (in days) or the equivalent settings in pnpm and uv, so a version published hours ago is not installed. Dependency selection covers the rest of package intake.

  4. Keep secrets out of the workspace

    .gitignore only affects Git; an assistant can still open .env and send it as context. Keep production secrets off developer machines, inject scoped non-production credentials at run time from a secret manager, and do not run commands that print secrets, such as env or kubectl get secret -o yaml, in an agent session. If a secret appears in a prompt, transcript or provider log, rotate it as Secrets out of code describes.

  5. Sandbox agents and give them their own credentials

    Run agents that execute commands in an OS-enforced sandbox, a dev container or a disposable VM, with only the repository mounted and egress limited to the model API and your package proxy. Give each agent its own short-lived, repository-scoped token: a GitHub App installation token, for example, can be limited to named repositories and permissions and expires after one hour. Keep bypass modes such as --dangerously-skip-permissions for disposable environments that hold no credentials.

  6. Treat repository text and MCP servers as untrusted

    Issues, pull request comments, READMEs and tool output can carry instructions. Do not let one agent session read untrusted text, hold broad credentials and reach the network at the same time. Install MCP servers like code: keep an allowlist, pin each to an exact version or image digest, review tool descriptions, and run local servers in a sandbox with minimal privileges, because they otherwise run with the same privileges as the client.

  7. Enforce the rules in settings, owners and CI

    A wiki page does not stop an agent. Use managed settings to disable bypass and auto-approve modes. Require code-owner review for agent instruction files (AGENTS.md, CLAUDE.md, .github/copilot-instructions.md), MCP configs and CI workflows, and do not let the person who asked an agent for a pull request approve it, a rule GitHub's own cloud agent enforces. Run secret scanning, SAST and the new-package check on every agent change (CI security checks).

Done when

Generated code goes through the same review and checks, and agents run with scoped permissions.

Mark it in the gap check

Common mistakes

  • Treating an instruction file or ignore file as access control. A line in AGENTS.md saying never to read .env is a request to the model, and the agent can still open the file.
  • Approving an agent's pull request from its description. A deleted assertion, a new postinstall script or an edited workflow file ships without anyone having read it.
  • Accepting the tests an agent wrote for its own code as proof. They can assert what the code does rather than what it should do, so a broken authorization check passes.
  • Running an agent in auto-approve mode on a laptop that holds SSH keys and cloud tokens. One injected instruction then acts with all of them.
  • Giving an agent a personal access token for every repository. One malicious issue can then reach code and settings across the whole organization.
  • Starting MCP servers with an unpinned npx or uvx command, which downloads and runs a package at launch with your privileges, so a malicious new release runs unreviewed.

What an assistant reads, writes, runs and sends

Each channel fails in its own way, so each needs a control that still holds when the model gets it wrong.

What an assistant reads, writes, runs and sends: table
ChannelTypical failureControl that holds
ReadsIssue text tells the agent to copy private code into a public pull requestSessions that read untrusted text get no cross-repository token and no open egress
WritesA generated handler builds a file path from user inputFull-diff review and tests for the case that must be refused
RunsThe agent installs an invented package whose install script steals tokensExistence and age check, release cooldown, sandbox without host credentials
SendsA .env file open in the editor is sent to the provider as contextSecrets kept out of the workspace and denied at the operating-system level

Tools and guides

Standards

NIST SSDF 1.1
PO.5 Implement and Maintain Secure Environments for Software DevelopmentPW.5 Create Source Code by Adhering to Secure Coding PracticesPW.7 Analyze Human-Readable Code to Identify Vulnerabilities and Verify Compliance with Security Requirements

Sources

  1. OWASP Secure Coding with AI Cheat Sheet
  2. Spracklen et al., We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs. USENIX Security 2025.
  3. GitHub Docs: Risks and mitigations for Copilot cloud agent
  4. GitHub Docs: Generating an installation access token for a GitHub App
  5. Model Context Protocol: Security Best Practices (local MCP server compromise). Revision 2026-07-28.
  6. npm config: min-release-age
  7. OWASP SAMM: Secure Build
  8. NIST SP 800-218: Secure Software Development Framework (SSDF) Version 1.1. February 2022.

Find your next answer

Type to search. Search runs in your browser.

Searches run in your browser, but the host can infer them from the index files they load. The address shows your search so you can bookmark or share it.