Design a safer file-upload pipeline

An upload is a sequence of trust decisions, not a single extension check. The client controls the filename, claimed media type, bytes and transmission pace. A storage service can faithfully store those bytes while a later previewer, browser or document parser interprets them differently. The security boundary must cover receipt, inspection, processing, publication, retrieval and eventual deletion.

Consider a profile-photo feature that needs only JPEG and PNG images, at most 2 MiB compressed, at most 12 megapixels decoded, and one published image per user. It does not need SVG, HTML, archives or the original filename as a filesystem path. Those decisions remove entire parser capabilities. A small compressed image may still require excessive decoded memory, so a byte cap and a pixel cap address different resources. An extension and a Content-Type header are useful consistency signals but are controlled by the sender and cannot establish the actual format.

Accept the stream into a private quarantine object with a server-generated id. Enforce the incoming byte budget while reading, before exhausting storage. Store owner, received length and a digest beside it. A worker with bounded CPU, memory and time decodes an allowed format, checks dimensions, strips unnecessary metadata and writes a newly encoded image. Publish the newly produced object only after all required checks succeed. A timeout or missing verdict leaves the upload unpublished. Framework-specific fixes are at /fix/file-upload/.

Bind inspection to the exact object version or digest that will be published. Otherwise a client can upload safe bytes, obtain approval and replace them with different bytes before release. Separate receipt credentials from publication credentials. The public read path must check the object's approved state and any required ownership policy, including thumbnails and temporary URLs. A signed URL to quarantine still bypasses quarantine if anyone possessing it can fetch the bytes. Do not put originals in the public bucket and hope an asynchronous scanner removes them quickly enough.

The Python program below implements receipt and processing for the photo contract with Pillow. receive reads at most one byte past the cap, so a 2,097,153-byte stream is rejected without being stored, and names the object with a random id. process opens the bytes with formats=("PNG", "JPEG"), rejects more than 12,000,000 pixels from the header before img.load() decodes the pixel data, and writes a new image that does not copy the original metadata. Install Pillow, save the file as upload_pipeline.py and run python3 upload_pipeline.py; it prints PASS: format, pixel, byte and metadata checks. The lab at /labs/path-traversal-python/ practices serving stored uploads without path traversal.

• States: RECEIVING → QUARANTINED → PROCESSING → PUBLISHED, or REJECTED at any step.
• A processing timeout returns the object to QUARANTINED for a bounded retry, or moves it to REJECTED.
• A public GET serves only the generated output of a PUBLISHED object.
• Approval of digest d1 for q17@v1 cannot publish q17@v2 with digest d2.

# Requires Pillow (pip install Pillow). Run: python3 upload_pipeline.py
import hashlib
import io
import secrets

from PIL import Image

MAX_BYTES = 2 * 1024 * 1024  # 2 MiB = 2,097,152 bytes
MAX_PIXELS = 12_000_000
FORMATS = ("PNG", "JPEG")


class Rejected(Exception):
    pass


def receive(stream):
    """Read into private quarantine; never read more than MAX_BYTES + 1 bytes."""
    data = stream.read(MAX_BYTES + 1)
    if len(data) > MAX_BYTES:
        raise Rejected("too_large")
    return {"id": secrets.token_urlsafe(16),  # never the client's filename
            "sha256": hashlib.sha256(data).hexdigest(), "data": data}


def process(item):
    """Decode an allowed format, check dimensions before decoding pixels, re-encode."""
    try:
        with Image.open(io.BytesIO(item["data"]), formats=FORMATS) as img:
            if img.width * img.height > MAX_PIXELS:
                raise Rejected("too_many_pixels")
            img.load()
            out = io.BytesIO()
            img.save(out, format=img.format)  # new file; metadata is not copied
    except (Image.UnidentifiedImageError, Image.DecompressionBombError, OSError):
        raise Rejected("not_an_allowed_image") from None
    return {"source": item["sha256"], "format": img.format, "data": out.getvalue()}


def png(width, height, **info):
    from PIL import PngImagePlugin
    meta = PngImagePlugin.PngInfo()
    for key, value in info.items():
        meta.add_text(key, value)
    out = io.BytesIO()
    Image.new("RGB", (width, height)).save(out, format="PNG", pnginfo=meta)
    return out.getvalue()


def rejects(data, reason):
    try:
        process(receive(io.BytesIO(data)))
    except Rejected as e:
        assert str(e) == reason, e
    else:
        raise AssertionError(f"accepted, expected {reason}")


ok = process(receive(io.BytesIO(png(800, 600, Comment="gps=51.5,-0.1"))))
assert ok["format"] == "PNG" and b"gps=" not in ok["data"]
assert process(receive(io.BytesIO(png(4000, 3000))))["format"] == "PNG"  # exactly 12,000,000
rejects(png(4001, 3000), "too_many_pixels")
rejects(b"<html><img src=x onerror=alert(1)></html>", "not_an_allowed_image")
gif = io.BytesIO()
Image.new("RGB", (8, 8)).save(gif, format="GIF")
rejects(gif.getvalue(), "not_an_allowed_image")
assert len(receive(io.BytesIO(b"\0" * MAX_BYTES))["data"]) == 2_097_152
try:
    receive(io.BytesIO(b"\0" * (MAX_BYTES + 1)))
    raise AssertionError("accepted 2,097,153 bytes")
except Rejected as e:
    assert str(e) == "too_large"
print("PASS: format, pixel, byte and metadata checks")

In this example a 2 MiB limit means 2,097,152 bytes and the image dimension rule means width × height must not exceed 12,000,000 pixels. Test exactly those boundaries rather than only a convenient large file. The valid case must produce a usable image; the rejected cases must create no public object. A response saying upload accepted may mean only that private receipt completed, so distinguish accepted-for-processing from published.

Instrument the decoder process and storage transitions in a staging system using synthetic files. Confirm cancellation kills abandoned processing, temporary objects expire and repeated failures cannot grow an unbounded retry queue. For the paper exercise below, supplied decoder results stand in for file bytes so you can evaluate the state machine without manufacturing image files. They establish the design decision only; a deployed decoder also requires package maintenance and integration checks against actual formats.

Re-encoding can remove unsupported active content and metadata, but the decoder itself processes attacker-controlled input. Isolating the worker limits the consequences of a parser flaw. Antivirus may identify known malicious content; a clean verdict is not proof of harmlessness. Keep the narrow allowlist and publication gate even when scanning is enabled. Download disposition, a separate content origin and nosniff can reduce browser interpretation risks, but cannot authorize who may read a private file.

Serve uploaded content so the browser cannot treat it as part of your application. Use a separate origin for user files, send Content-Disposition: attachment for downloads, X-Content-Type-Options: nosniff with the verified Content-Type, and Content-Security-Policy: sandbox for anything displayed inline. SVG is XML that can carry script, so rasterize it, sanitize it with an SVG-aware allowlist or serve it only as an attachment. A polyglot file is valid as two formats at once and passes magic-byte checks, which is why this pipeline decodes and re-encodes instead of trusting signatures; content disarm and reconstruction applies the same idea to office documents and PDFs, and /domains/appsec/secure-features/untrusted-document-conversion/ shows how to convert those in an isolated parser. If the feature accepts archives, apply /domains/appsec/secure-features/archive-path-safety/; the archive-extraction project linked below tests those rules.

Operational failures often appear as stuck uploads or mismatched versions. Expose a stable processing status to the owner, record the failed stage and retry count, and keep sensitive original bytes out of logs. A quarantine-retention policy needs enough time for investigation without keeping rejected personal documents indefinitely. Backpressure should reject new work predictably when capacity is exhausted; silently skipping inspection to reduce queue length converts an availability incident into a security failure.

CVE-2021-21972 was an unauthenticated file upload in the VMware vCenter Server vSphere Client plugin that let an attacker with network access to port 443 write a file and execute commands with unrestricted privileges on the host operating system. The missing controls were validating the upload and storing it where the web server will never execute it.

The pipeline can be correctly sandboxed yet still publish the wrong person's object. Suppose Alice receives upload id U1, then submits finishUpload(U2) where U2 belongs to Bob. A finish endpoint that merely finds a clean verdict for U2 may attach Bob's image to Alice's profile. Require the authenticated actor to own the upload session, verify the immutable content version, and bind the resulting published object to the intended feature. A scanner's clean verdict answers content-policy questions; it does not answer who may publish or retrieve the object.

Direct-to-storage uploads require the same discipline. Limit the issued upload capability to its object key, method, size constraints where supported and lifetime; an Amazon S3 presigned POST can carry the size limit in its policy as ["content-length-range", 1, 2097152]. The completion endpoint should verify actual stored properties and ownership rather than trusting client-reported length or digest. If a client can replace the same key after scanning, publication should use an immutable version or copied generated output. Changing the object key to a random string helps prevent collisions but does not establish ownership or immutability.

Consider concurrency too. If Alice uploads U1 and U2 and the older worker finishes last, an unconditional profile update can replace her newer choice. Bind publication to the current selected upload or compare a profile revision before committing. The old worker should mark its own processing complete without overriding a newer user decision. This is a correctness issue with privacy consequences when objects have different visibility or retention expectations.

Repair the code

Path traversal lab: Serve an uploaded file without path traversal

File upload lab: Accept avatars without publishing pages

Sources & further reading

The standards, specifications and documentation behind the guidance above. Each is the evidence for a claim on this page; consult it for detail beyond the defaults and versions listed here.

1. OWASP File Upload
2. OWASP Input Validation
3. OWASP ASVS 5.0.0: V5 File Handling. Version 5.0.0.
4. Pillow: Image.open and decompression bomb protection. Pillow 12.3.0.
5. Amazon S3: POST policy (content-length-range)
6. MITRE CWE-434: unrestricted upload of file with dangerous type. CWE 4.20.
7. NVD: CVE-2021-21972, file upload to RCE in VMware vCenter

Reference information

Tags

Input validation and parsingFile uploads and archives

SDLC phase

DesignDevelopment

Role

DevelopersAppSec engineers

ASVS 5.0.0 requirements

v5.0.0-5.2.1 · v5.0.0-5.2.2 · v5.0.0-5.2.6 · v5.0.0-5.3.2 · v5.0.0-5.4.1

OWASP WSTG v4.2 tests

WSTG-BUSL-08 · WSTG-BUSL-09

Download this guide as Markdown

Continue exploring