Bound API work, not just request counts

List endpoints that parse large inputs, search broadly, generate documents, resize images, perform password verification, or invoke paid services. Estimate the work one request can trigger, including nested queries and downstream fan-out. A low request rate can still exhaust resources when one accepted operation is unbounded; that is how API abuse becomes an application-layer denial of service (DoS) without a flood of traffic.

Define a useful service budget: acceptable input size, execution time, memory, result size, concurrency, and downstream cost. Choose values from representative legitimate workloads and deployment capacity; there is no universal safe rate for every endpoint.

A report request contains account IDs, a page size and requested date range. Each account causes a database query, and each returned row causes one formatting operation. For this example, estimated work units are accounts × page size. That estimate is a deliberately simple admission model, not a promise that all database rows cost the same. The dangerous failure is that a per-minute HTTP limit counts a request costing 10 units and one costing 20,000 units equally.

The service contract permits at most 10 accounts, 100 rows per account, and 500 estimated units per request. These controls overlap usefully: the account cap bounds fan-out, the page cap bounds each query, and the combined cap rejects 10 × 100 despite both fields passing individually. Validate integers and ranges before computing or allocating work. Bound request bytes before parsing, then limit parsed arrays and strings; parsing an enormous object to discover its excessive length is too late to protect parser memory.

Capacity adds a second dimension. Each tenant has two running slots and five waiting slots. The third otherwise-valid request may queue, but the eighth is rejected with the documented retry behavior. There is also a global limit, because a thousand tenants cannot each reserve two workers on a twenty-worker service. Reserve and release slots atomically; a read-then-increment counter can admit multiple requests over the limit during a race.

Give each accepted operation a two-second deadline in this fictional service. At the deadline, returning 504 alone protects the client’s patience, not the database pool. The handler must signal cancellation, ensure the database driver observes it, and release the reserved slot in a finally path. If a downstream effect cannot be cancelled, record its operation identity and uncertain status instead of pretending it never happened. Distinguish overload rejection before work starts from failure after a billable effect.

The Express 5 module below implements this contract. Validation and the combined cost check run before any reservation; acquire and release update the counters without awaiting in between, which makes them atomic within one Node.js process; and AbortSignal.timeout gives the report two seconds before the database work is cancelled. With several instances, keep the counters in one place, for example a Redis Lua script or a single SQL statement that reserves only when below the limit. The tests in the verification section exercise every row of the acceptance matrix.

// Admission control for POST /reports (Express 5): bounded bytes and fields, a combined cost
// cap, per-tenant and global concurrency with a bounded queue, and a deadline that aborts work.
import express from 'express';

const LIMIT = {accounts: 10, pageSize: 100, units: 500, running: 2, waiting: 5, global: 20, ms: 2000};
const tenants = new Map(); // tenant -> {running, waiting}; one process, so check-and-update is atomic
let runningGlobal = 0;

async function acquire(tenant) {
  const t = tenants.get(tenant) ?? {running: 0, waiting: []};
  tenants.set(tenant, t);
  if (runningGlobal >= LIMIT.global) return 503;
  if (t.running < LIMIT.running) {
    t.running++;
    runningGlobal++;
    return 0;
  }
  if (t.waiting.length >= LIMIT.waiting) return 429;
  await new Promise(resolve => t.waiting.push(resolve)); // release() hands over its slot
  return 0;
}

function release(tenant) {
  const t = tenants.get(tenant);
  const next = t.waiting.shift();
  if (next) return next();
  t.running--;
  runningGlobal--;
}

export function createApp(runReport) {
  const app = express();
  app.post('/reports', express.json({limit: '16kb'}), async (req, res) => {
    const {accounts, pageSize} = req.body ?? {};
    const valid = Array.isArray(accounts) && accounts.length >= 1 && accounts.length <= LIMIT.accounts
      && new Set(accounts).size === accounts.length
      && Number.isInteger(pageSize) && pageSize >= 1 && pageSize <= LIMIT.pageSize;
    if (!valid) return res.status(400).json({error: 'invalid_request'});
    if (accounts.length * pageSize > LIMIT.units) return res.status(400).json({error: 'too_expensive'});

    const tenant = req.tenant; // set by authentication middleware from the verified identity
    const refused = await acquire(tenant);
    if (refused) return res.status(refused).set('Retry-After', '5').json({error: 'busy'});
    const deadline = AbortSignal.timeout(LIMIT.ms);
    try {
      res.json(await runReport(accounts, pageSize, deadline)); // the driver must honor the signal
    } catch (err) {
      if (!deadline.aborted) throw err;
      res.status(504).json({error: 'deadline_exceeded'});
    } finally {
      release(tenant);
    }
  });
  return app;
}

export const usage = () => ({runningGlobal, tenants: Object.fromEntries(tenants)});

• Reject oversized bodies, strings, arrays, uploads, page sizes, and excessive batch/query complexity before expensive processing where possible.
• Apply limits by relevant identity, tenant, operation, and resource. An IP-only rule may penalize shared networks while missing distributed misuse.
• Bound running jobs and queue depth, as the bounded background work guide at /domains/runtime/availability/bounded-background-work/ shows. Propagate cancellation and deadlines to expensive downstream work, not just the HTTP response.
• Set provider spending controls where supported and alert on unusual cost growth. An alert alone does not stop spending.
• Define backpressure and graceful degradation so an overloaded feature does not consume every shared worker or database connection.

Rate limiting at an edge proxy can reject floods and oversized bodies before they reach the application, but it cannot see the cost of a report or the tenant behind it, so it complements the application limits rather than replacing them; the layer 7 DDoS guide at /domains/runtime/availability/l7-ddos-resilience/ covers slow clients and floods. In the nginx 1.30 configuration below, 40 simultaneous requests from one address produced 21 successes, the first plus a burst of 20, and 19 responses of 429 with Retry-After: 1; a 17 kB body received 413, and requests succeeded again three seconds later. Key limits and per-tenant quotas by authenticated identity where the edge knows it: an address-based key penalizes users behind shared NAT and misses distributed abuse.

Use the status codes clients and libraries understand. 429 Too Many Requests (RFC 6585) means this client exceeded a limit; 503 Service Unavailable means the server cannot take the work now; both can carry Retry-After, in seconds or as a date (RFC 9110). The proposed RateLimit and RateLimit-Policy fields, such as RateLimit: "default";r=50;t=30, let clients pace themselves, but they are still an Internet-Draft, so treat their format as subject to change.

# Coarse edge limits in front of the application (nginx 1.30); the application still
# enforces cost, tenant concurrency and deadlines.
limit_req_zone $binary_remote_addr zone=reports:10m rate=10r/s;

server {
    listen 8080;

    location = /reports {
        client_max_body_size 16k;                   # larger bodies get 413 before proxying
        limit_req zone=reports burst=20 nodelay;    # 10 requests/s per address, bursts of 20
        limit_req_status 429;                       # the default is 503
        error_page 429 = @too_many;
        proxy_pass http://app:3000;
    }

    location @too_many {
        add_header Retry-After 1 always;
        return 429 '{"error":"rate_limited"}\n';
    }
}

In an authorized nonproduction environment, exercise limits just below, at, and above their thresholds. Include a single expensive request, concurrent requests, repeated retries, a stalled dependency, and a full queue. Observe CPU, memory, connection pools, backlog, and third-party calls.

Verify that rejected or timed-out work stops consuming resources, other tenants retain the intended service, and legitimate work recovers after pressure ends. Make retry guidance consistent with the failure behavior. Retain capacity assumptions and results alongside configuration. These controls address application resource abuse; volumetric network attacks may additionally require upstream network protection.

The tests below start the Express app on a loopback port with a report function that holds until released or aborted. They pass on Node.js 26.3.0 with Express 5.2.1.

import test from 'node:test';
import assert from 'node:assert/strict';
import express from 'express';
import {createApp, usage} from './report-admission.mjs';

const sleep = ms => new Promise(r => setTimeout(r, ms));
const held = []; // resolvers for reports that are running
const heldReport = (accounts, pageSize, signal) =>
  new Promise((resolve, reject) => {
    held.push(() => resolve({rows: accounts.length * pageSize}));
    signal.addEventListener('abort', () => reject(signal.reason)); // a cancellable "query"
  });

async function server(runReport) {
  const app = express();
  app.use((req, res, next) => ((req.tenant = req.get('x-test-tenant')), next()));
  app.use(createApp(runReport));
  return new Promise(resolve => {
    const s = app.listen(0, '127.0.0.1', () => resolve(s));
  });
}
const post = (s, body, tenant = 'north') =>
  fetch(`http://127.0.0.1:${s.address().port}/reports`, {
    method: 'POST',
    headers: {'content-type': 'application/json', 'x-test-tenant': tenant},
    body: typeof body === 'string' ? body : JSON.stringify(body),
  });
const ids = n => Array.from({length: n}, (_, i) => `acct-${i}`);

test('field and cost limits reject before any work', async () => {
  const s = await server(async (a, p) => ({rows: a.length * p}));
  assert.equal((await post(s, {accounts: ids(5), pageSize: 100})).status, 200); // 500 units
  assert.equal((await post(s, {accounts: ids(6), pageSize: 100})).status, 400); // 600 units
  assert.equal((await post(s, {accounts: ids(1), pageSize: 101})).status, 400);
  assert.equal((await post(s, {accounts: ['a', 'a'], pageSize: 1})).status, 400);
  const big = JSON.stringify({accounts: ids(1), pageSize: 1, pad: 'x'.repeat(17_000)});
  assert.equal((await post(s, big)).status, 413);
  s.close();
});

test('two run, five wait, the eighth gets 429 with Retry-After; other tenants still run', async () => {
  const s = await server(heldReport);
  const queued = Array.from({length: 7}, () => post(s, {accounts: ids(1), pageSize: 1}));
  await sleep(200);
  const eighth = await post(s, {accounts: ids(1), pageSize: 1});
  assert.equal(eighth.status, 429);
  assert.equal(eighth.headers.get('retry-after'), '5');
  assert.equal(held.length, 2);
  const south = post(s, {accounts: ids(1), pageSize: 1}, 'south');
  await sleep(100);
  assert.equal(held.length, 3); // south started despite north's full queue
  while (usage().runningGlobal > 0) {
    held.shift()?.();
    await sleep(20);
  }
  assert.deepEqual((await Promise.all([...queued, south])).map(r => r.status), Array(8).fill(200));
  s.close();
});

test('the deadline aborts the work and frees the slot', async () => {
  const s = await server(heldReport);
  const started = Date.now();
  const res = await post(s, {accounts: ids(1), pageSize: 1}, 'east');
  assert.equal(res.status, 504);
  assert.ok(Date.now() - started >= 1900);
  held.length = 0; // the aborted report can no longer resolve
  assert.equal(usage().tenants.east.running, 0);
  assert.equal(usage().runningGlobal, 0);
  s.close();
});

A complexity score can be gamed if an expensive resolver is assigned a cheap weight. Compare estimated units to observed duration, bytes and provider calls, then revise the model when query plans or product features change. Keep a hard downstream deadline and concurrency limit even when the estimator is accurate on ordinary traffic.

Avoid teaching users to retry every denial immediately. Repeated retries can turn transient overload into permanent saturation. Return stable error categories and suitable retry guidance, cap server-side retries, and preserve an operation ID for accepted asynchronous work. Tenant fairness, global capacity and endpoint budgets answer different questions; none replaces the other two.

These outcomes belong to the worked example, and the tests above check each row against the reference implementation.

Sources & further reading

The standards, specifications and documentation behind the guidance above. Each is the evidence for a claim on this page; consult it for detail beyond the defaults and versions listed here.

1. OWASP API Security: unrestricted resource consumption. API4:2023.
2. OWASP Denial of Service
3. RFC 6585: additional HTTP status codes (429). April 2012.
4. RFC 9110 §10.2.3: Retry-After. RFC 9110 (June 2022).
5. IETF draft: RateLimit header fields for HTTP. Internet-Draft, revision 11.
6. Express: express.json()
7. nginx: limit_req module

Reference information

Tags

Denial of service and resource limitsAPI security

SDLC phase

DevelopmentMaintenance

Role

DevelopersPlatform engineers and SREs

ASVS 5.0.0 requirements

v5.0.0-15.1.3 · v5.0.0-15.2.2

Download this guide as Markdown

Continue exploring