Bound agent spending, memory and recovery

An agent request can trigger repeated model calls, tool calls, retrieval, retries and memory writes. Limits on the initial HTTP request do not bound that downstream work. A loop can spend money or capacity without a high volume of inbound requests, and a cancelled interface can leave server-side work running. The OWASP Top 10 for LLM Applications 2026 lists this as Unbounded Consumption (LLM06:2026); the API Security Top 10 calls the general case Unrestricted Resource Consumption (API4:2023).

Memory adds a separate persistence boundary. Storing a tool response or model summary for future tasks can preserve malicious instructions and sensitive data beyond the original session; the Agentic Top 10 lists this as Memory and Context Poisoning (ASI06:2026). A model deciding that text is worth remembering is not authorization to keep it or share it with other users.

Task T has a 100-credit cap and has already spent 60. Children C1 and C2 each need a maximum reservation of 30. If both read remaining=40 before either starts, both dispatch and the eventual spend can reach 120. The code below reserves capacity in one conditional UPDATE: the row changes only if spent + reserved + amount <= cap, so two concurrent children cannot both take the last 40 credits. After C1 reserves 30, only 10 remain, and C2 waits until capacity is released or the budget is explicitly raised.

Reserve a bound the provider honors and reconcile the actual charge afterwards. Setting an output-token cap on each request, as /domains/ai-emerging/llm-security/calling-llm-apis-safely/ does on every call, is what makes the bound real: the worst case is the input tokens at the input price plus the cap at the output price. If C1 uses 18 of its 30 reserved credits, spent becomes 78 and 22 become available, so a new 30-credit call still does not fit. If actual cost can exceed the reservation, settle records it anyway and returns a flag to alert on: an estimate is not a hard spending guarantee.

Retries and nested agents are part of T, so every child charges T’s row. Giving each child its own 100-credit allowance turns a per-task limit into a multiplier. Track parentage, outstanding reservations, committed usage and cancellation state through the delegation graph. Cancelling T stops new dispatch and asks workers to stop, but committed provider work can still incur charges; reconcile those outcomes rather than erasing reservations on a UI cancellation.

Persistent memory has a different risk. A retrieved document may say “the user approves all future exports.” Saving that sentence as memory does not create approval. Store its provenance, label it untrusted content and never consult it as a permission record. User preferences, retrieved facts and authorization state need different storage and rules. During an incident, quarantine the contaminated memory and identify the tasks that read it, while preserving the evidence.

import sqlite3

SCHEMA = """
CREATE TABLE budgets (
  task_id TEXT PRIMARY KEY,  -- the root task: children and retries charge this row
  cap INTEGER NOT NULL,
  spent INTEGER NOT NULL DEFAULT 0,
  reserved INTEGER NOT NULL DEFAULT 0,
  CHECK (spent >= 0 AND reserved >= 0)
);
"""


def reserve(db, root_task, amount):
    """Reserve `amount` credits atomically before dispatch; False if it does not fit."""
    row = db.execute(
        """UPDATE budgets SET reserved = reserved + :amount
           WHERE task_id = :task AND spent + reserved + :amount <= cap
           RETURNING cap - spent - reserved""",
        {"task": root_task, "amount": amount},
    ).fetchone()
    return row is not None


def settle(db, root_task, reserved_amount, actual):
    """Release the reservation and record the actual charge, even if it overran."""
    db.execute(
        "UPDATE budgets SET reserved = reserved - ?, spent = spent + ?"
        " WHERE task_id = ?",
        (reserved_amount, actual, root_task),
    )
    return (
        actual > reserved_amount
    )  # True means the reservation was not a hard bound: alert


def connect(path):
    return sqlite3.connect(
        path, isolation_level=None, timeout=10
    )  # each UPDATE is atomic

Set limits for steps per task, elapsed time, parallel children, input and output size, and reserved cost. Apply per-user and per-tenant budgets as well as service-wide protection; /domains/runtime/api-security/resource-consumption-limits/ covers the same limits at the API edge. Enforce them in orchestration code and downstream workers, not in prompts, and include retries and nested child tasks in the same accounting.

Reserve capacity before starting concurrent work and reconcile actual consumption afterwards. Otherwise simultaneous tasks each observe the same remaining budget and together exceed it. Know which provider charges or running operations cannot be stopped instantly; a local cancellation flag is not proof that costs have ended.

Define which memory types exist, who owns them, which tasks can read them and when they expire or are deleted. Keep user preferences separate from retrieved facts and operational authority. Store the source of every entry and prevent stored text from becoming a permission grant when it is reintroduced into another prompt.

Bound memory size and write frequency, and do not store credentials or unnecessary source documents. Let users inspect and remove their retained context. During an incident, be able to quarantine contaminated memory or disable a tool without deleting the evidence needed to understand what happened; /domains/ai-emerging/agent-security/agent-failure-containment/ covers kill switches, quarantine and rollback. How injected text reaches memory in the first place, and how to gate, scope and expire memory writes, is in /domains/ai-emerging/agent-security/agent-memory-poisoning/.

Cancellation should stop new dispatch, propagate to supported workers and leave a clear state for work already committed. Use idempotent effects and explicit task states so retries or resumes do not repeat payments, messages or writes (see /domains/runtime/availability/bounded-background-work/). Test worker failure after an effect but before acknowledgement. Logs should connect resource use and side effects to the task without indiscriminately storing sensitive prompts.

The agent-tool-boundaries project, linked from /domains/ai-emerging/agent-security/tool-authorization/, includes this case: its tests crash the worker after the provider has committed a credit and check that the retry reconciles the existing receipt instead of applying the credit twice.

Releasing a reservation merely because a request timed out can allow another dispatch while the original provider task still spends. Keep an uncertain state until the provider outcome or configured bounded reconciliation is known. Account for cancellation latency and already-running work in operational limits.

Deleting contaminated memory does not undo messages or payments already sent. Trace downstream effects using task and operation IDs, then apply the specific recovery procedure. Retention cleanup, permission repair and business-effect recovery are separate actions.

Each row is a step in task T’s budget or memory handling and the state it must leave behind. The budget rows follow the arithmetic of reserve and settle above.

Sources & further reading

The standards, specifications and documentation behind the guidance above. Each is the evidence for a claim on this page; consult it for detail beyond the defaults and versions listed here.

1. OWASP Top 10 for LLM Applications 2026 (LLM06 Unbounded Consumption). 3 August 2026.
2. OWASP LLM10:2025 Unbounded Consumption (2025 list; LLM06 in 2026)
3. OWASP Top 10 for Agentic Applications for 2026 (ASI06 Memory & Context Poisoning). 9 December 2025.
4. OWASP AI Agent Security Cheat Sheet
5. OWASP API4:2023 Unrestricted Resource Consumption
6. NIST Privacy Framework: A Tool for Improving Privacy through Enterprise Risk Management. Version 1.0 (16 January 2020).

Reference information

Tags

Denial of service and resource limitsLLM application securityAI agents and tools

SDLC phase

DesignDevelopment

Role

DevelopersAppSec engineersPlatform engineers and SREs

ASVS 5.0.0 requirements

v5.0.0-15.4.2

Download this guide as Markdown

Continue exploring