AI agents and tools

Worked threat model: an LLM agent with tools

A complete worked threat model for an LLM agent with tools: the data-flow diagram, the threats from prompt injection, excessive agency, data exfiltration, confused-deputy and memory poisoning, and the mitigations, mapped to the OWASP Top 10 for LLM Applications 2026 and the OWASP Top 10 for Agentic Applications 2026.

Design · AppSec engineers

Secure the developer workstation: extensions, package managers, secrets and AI agents

Attackers reach the software supply chain through developers' editors, package installs and AI agents. Allow-list extensions, stop install scripts and brand-new versions, keep long-lived secrets off laptops, and lock down repository files that run code when a project opens.

Design, Development, Release · Developers, AppSec engineers, Platform engineers and SREs

Secrets at runtime: Vault, Kubernetes Secrets, CSI driver, External Secrets and SOPS

How a running workload should get its credentials: encrypt Kubernetes Secrets with a KMS v2 provider and lock down RBAC, mount from an external store with the Secrets Store CSI Driver, sync with External Secrets Operator, issue dynamic credentials with Vault and its Agent Injector, keep Git copies encrypted with SOPS, and make rotation reach the process.

Design, Development, Maintenance · Developers, Platform engineers and SREs

Contain prompt injection at the application boundary

Assume retrieved content can influence model output; enforce data access and consequential actions in components the model cannot authorize for itself.

Design, Development · Developers, AppSec engineers, Platform engineers and SREs

Authorize agent tools outside the model

Bind each proposed action to the user, resource and approved arguments before allowing a tool to cause a side effect.

Design, Development · Developers, AppSec engineers, Platform engineers and SREs

Red-team LLM applications and agents by their effects

A red-teaming method for LLM features and agents: scope the whole system, map attacks to OWASP and MITRE ATLAS identifiers, score trials from observed effects, report rates with confidence intervals and retest on every model or provider change.

Design, Testing, Security program · Developers, AppSec engineers, Platform engineers and SREs

Bound agent spending, memory and recovery

Enforce budgets and stop conditions outside the model, and make retained context and side effects recoverable.

Design, Development · Developers, AppSec engineers, Platform engineers and SREs

Constrain MCP and multi-agent trust boundaries

Validate each connection and delegated action without treating a token for another service, a tool description, another agent or a state handle as authority.

Design, Development · Developers

Contain cascading failures and rogue agents

Stop a looping, compromised or drifting agent from spreading harm: kill switches, budget and rate limits, circuit breakers, behavior monitoring, blast-radius limits, quarantine, rollback and incident response, all enforced outside the agent.

Design, Development, Maintenance · Platform engineers and SREs

Secure AI-assisted development with coding assistants and agents

Treat generated code, suggested packages, repository text and MCP servers as untrusted inputs, and confine coding agents with OS-level isolation and short-lived, narrowly scoped credentials.

Design, Development · Developers, AppSec engineers

Sandbox model-generated code and computer-use agents

Run the code an agent writes, and agents that drive a browser or desktop, inside a boundary built for hostile code: a disposable runtime with no secrets and no default network, hard resource limits, and a person in front of every side effect.

Design, Development · Developers, Platform engineers and SREs

Keep secrets and controls out of hidden context (OWASP LLM08:2026)

OWASP LLM08:2026 Hidden Context Exposure, which replaced System Prompt Leakage: why system prompts, tool schemas and retrieved policy text get extracted, what must never be in them, and how to enforce rules where a leak changes nothing.

Design, Development · Developers, AppSec engineers

Defend agent memory and context against poisoning (OWASP ASI06)

OWASP ASI06 Memory & Context Poisoning: how an instruction or false fact saved once in agent memory, conversation summaries or a retrieval store steers later sessions, users and agents, and the write, scope, provenance, expiry and recovery controls that contain it.

Design, Development · Developers, AppSec engineers

Case study: the Nx s1ngularity compromise and its AI-assisted payload (2025)

How a pull-request-title injection in a GitHub Actions workflow on an outdated branch gave an attacker the Nx npm token in August 2025, how the malicious versions tried to turn developers' own AI command-line tools into file-search agents, and the workflow, publishing and agent controls that stop the same chain.

Design, Development, Release, Maintenance · Developers, Platform engineers and SREs

Case study: the Amazon Q Developer extension compromise (2025)

How an over-scoped GitHub token, extracted from an AWS CodeBuild pull request build, let a threat actor commit code to the Amazon Q Developer extension for VS Code that shipped in version 1.84.0 in July 2025, why the injected code aimed at the Q Developer CLI failed, and the CI and AI-agent controls the incident argues for.

Design, Development, Testing, Release, Maintenance · Developers, Platform engineers and SREs

Excessive agency lab: Bind an agent’s action to the approved arguments

An assistant receives approval to export one report, then substitutes a different report and destination. Repair the tool gateway that authorizes the final action.