Parse bounded JSON and XML into an explicit application schema

A JSON or XML parser accepts a language and turns it into values. The application still has to decide which values and structures belong to its business vocabulary. Syntax success does not establish that an invoice contains only allowed fields, that cents is an integer, or that the caller may submit an invoice for the chosen account.

Arbitrary object deserialization adds a more dangerous capability: the representation can select types or behavior rather than just data. Python pickle is designed to reconstruct Python objects and must not be used for untrusted input; the fix cards at /fix/deserialization/ show data-only replacements. Replacing such a format with JSON narrows capabilities, but the following schema and resource checks remain necessary. This guide uses actual JSON and XML parsing and never demonstrates loading an untrusted executable object.

The project accepts an invoice with exactly currency and items. Currency is USD or EUR. There must be 1–100 items, each containing exactly cents, an integer from 0 through 10,000,000. The aggregate must be at most 100,000,000. A document is at most 8,192 encoded bytes; JSON nesting is at most four containers. These values are explicit fixture requirements, not universal invoice limits.

The XML vocabulary expresses the same data as an invoice element with a currency attribute and item children with cents attributes. It permits XML 1.0 in UTF-8, whitespace and comments. It excludes DTDs, declared entities, processing instructions, namespaces, nested item children and non-whitespace text. XML cents must be 1–8 ASCII decimal digits before conversion. Decide on these rules before selecting a parser configuration.

JSON: {"currency":"USD","items":[{"cents":150},{"cents":225}]}
XML: <invoice currency="USD"><item cents="150"/><item cents="225"/></invoice>
Normalized result: {currency: USD, items: [150,225], total: 375}
Boundary order: bytes -> encoding/syntax/capabilities -> exact structure -> numeric limits -> normalized value
Authorization and database commit occur after this boundary and are not provided by it.

A parser that retains only the last occurrence of a duplicate JSON name has already thrown away evidence before a normal dictionary validator runs. The reference uses Python’s object_pairs_hook to inspect member pairs and reject repeated names. parse_constant refuses non-finite constants accepted by Python’s permissive default JSON decoder; an integer-token bound limits conversion work.

Exact numeric type matters. In Python, bool is a subclass of int and int(True) becomes 1. int(1.5) also loses a fractional part. The reference therefore checks type(value) is int before accepting cents, then applies the stated range and aggregate bounds. Conversion should not silently invent an allowed business value from a forbidden one.

The depth precheck is a small state machine tracking structural braces and brackets outside quoted strings. It recognizes escaped quotes so string content does not inflate depth. It is only a resource guard; json.loads still validates JSON syntax. The byte cap comes first, limiting the amount of input the scanner and parser must examine. An HTTP endpoint must independently bound incoming and decompressed bytes before calling this function.

The reference uses Python’s Expat event interface for this narrow document vocabulary. It rejects a DOCTYPE declaration as soon as the parser reports it, disables parameter-entity parsing, and rejects external-entity callbacks. Event handlers construct only the allowed root and item structure; unexpected elements, attributes or text fail while parsing.

This distinction matters even for a harmless entity that expands to USD. The fixture rejects the capability to declare entities, independently of whether a particular expansion looks benign. A schema validator applied after expansion would be too late to decide whether expansion was allowed. A case-insensitive substring search for a few declaration tokens is not a replacement for an XML parser.

Check the underlying library version, not only the Python interpreter version. The project requires Expat 2.6.0 or newer and enables reparse deferral where supported. That is a capability floor, not a safe version: later releases fixed CVE-2024-45490 to CVE-2024-45492 (2.6.3), CVE-2024-50602 (2.6.4), CVE-2024-8176 (2.7.0), CVE-2025-59375 (2.7.2) and more than 25 CVEs in the 2026 releases 2.7.4 to 2.9.0. Run the current release, 2.9.0 as of 10 October 2026, and check what each runtime actually links with pyexpat.EXPAT_VERSION: on the machine used to verify this guide, Python 3.14.7 reported expat_2.7.4 while the python:3.11-alpine image linked the newer system expat_2.8.5, because the interpreter build decides whether it bundles its own copy. Keep the interpreter and linked libraries maintained. For broader XML processing, choose a maintained hardened parser and configure each required capability explicitly; the XXE fix cards at /fix/xxe/ show that configuration for Java, .NET and Python.

Download the bounded data parsing project linked with this guide. It requires only Python’s standard library and opens no sockets or files containing user data. Run python3 -m unittest -v test_parsing.py. The starter passes five of twenty-four cases and fails nineteen: its syntax parser works, but it coerces amounts, drops unknown fields and accepts unsupported XML features.

Repair parse_invoice in exercise.py against the written contract. Keep byte bounds, duplicate detection, parser capability choices and schema checks as distinct decisions. The reference command is EXERCISE_MODULE=reference python3 -m unittest -v test_parsing.py. All twenty-four cases pass. On PowerShell set the environment variable before running the same Python command; clear it before retesting your own code.

The suite keeps ordinary JSON and XML, zero cents and exactly one hundred items as positive controls. It also supplies a tiny internal entity expanding to a three-character currency and an unused external-DTD declaration. The tests do not read a local file, fetch a network entity or generate an entity-expansion flood. Their purpose is to establish the configured acceptance boundary safely.

Dropping unknown fields can be appropriate for some versioned protocols, but it violates this exact-field invoice contract. A client-supplied approved field must not silently become a later authority flag. Conversely, changing an established extensible format to reject every unknown field is a compatibility decision requiring a documented policy; do not generalize this narrow example blindly.

Checking each line item independently misses aggregate effects. Eleven values of 10,000,000 sum to 110,000,000, above the document cap despite valid individual ranges. Rejecting only negative numbers misses floats and booleans. Rejecting DTDs does not remove the need for byte and structure limits. Each check has a distinct counterexample.

After parsing, pass the normalized result to separate authorization and transactional business logic. Do not use parsing success as an approval signal. Likewise, do not log an entire rejected document by default: length, content type, policy reason and a request identifier often provide enough diagnostic evidence without creating a second sensitive-data store.

The outcomes below belong to this deliberately small vocabulary. Each rejection should have a useful internal reason while external error details follow the application’s disclosure policy.

This project does not implement XML signatures, schema languages, namespace processing, XPath, multipart upload handling, decompression or an invoice storage service. It is a bounded example of translating two simple data syntaxes into one business object. A production integration must add service-level time/memory budgets and evaluate its actual parser features, document sizes and supported versions. Preserve the regression fixture whenever that vocabulary changes.

Repair the code

XXE lab: Parse an uploaded invoice without resolving external entities

Pickle deserialization lab: Load a session token without pickle

Sources & further reading

The standards, specifications and documentation behind the guidance above. Each is the evidence for a claim on this page; consult it for detail beyond the defaults and versions listed here.

1. Python JSON decoder and compliance behavior. Python 3.15.
2. Python XML security considerations. Python 3.15.
3. Python Expat parsing callbacks and version behavior. Python 3.15.
4. Python pickle: trusted data requirement. Python 3.15.
5. libexpat changelog. Release 2.9.0, 5 October 2026.
6. OWASP XML External Entity Prevention Cheat Sheet
7. MITRE CWE-776: XML entity expansion. CWE 4.20.

Reference information

Python 3.13+ with Expat 2.6.0 or later; run the current Expat release. The project checks its parser runtime.

Tags

Input validation and parsingUnsafe deserializationDenial of service and resource limits

SDLC phase

Development

Role

DevelopersAppSec engineers

ASVS 5.0.0 requirements

v5.0.0-1.5.1 · v5.0.0-1.5.2 · v5.0.0-1.5.3 · v5.0.0-2.2.1

Download this guide as Markdown

Continue exploring