--- key: "bounded-json-xml-parsing" title: "Parse bounded JSON and XML into an explicit application schema" url: "https://softwaresecurity.com/domains/appsec/input-handling/bounded-json-xml-parsing/" api: "https://softwaresecurity.com/api/v1/articles/bounded-json-xml-parsing.json" domain: "appsec" topic: "Application security" type: "Guide" identifiers: ["CWE-20","CWE-502","CWE-611","CWE-776","CWE-770","A05:2025","A08:2025","A02:2025","API4:2023","V1.5.1","V1.5.2","V1.5.3","V2.2.1"] tags: ["Input validation and parsing","Unsafe deserialization","Denial of service and resource limits"] stages: ["build"] roles: ["developer","appsec-engineer"] license: "CC-BY-4.0" license_url: "https://creativecommons.org/licenses/by/4.0/" code_license: "MIT" --- # Parse bounded JSON and XML into an explicit application schema Separate parser capabilities from schema rules, reject ambiguous and coercible inputs, and implement the boundary in a complete Python project. Canonical URL: https://softwaresecurity.com/domains/appsec/input-handling/bounded-json-xml-parsing/ ## In short Cap the bytes before parsing, then turn the document into an exact schema: reject duplicate keys, unknown fields, booleans or floats where integers belong, and anything beyond the depth, item and total limits. For XML, refuse DTDs and entities instead of expanding them, and run a current parser library; Expat 2.6.0 provides the needed API, but later releases fixed many vulnerabilities. Parsing proves only syntax: authorization and business rules still follow. - Bound bytes first, then depth, item counts and totals; per-item checks miss aggregate limits. - Inspect JSON member pairs so a repeated key is rejected instead of silently keeping the last value. - Check exact types: in Python `True` is an `int` and `int(1.5)` truncates, so coercion invents allowed values. - Reject a DOCTYPE as soon as the parser reports it; a harmless-looking entity is still an excluded capability. - The linked Expat version decides XML safety: check `pyexpat.EXPAT_VERSION`, not only the Python version. - Never use `pickle` or other object deserialization for untrusted input. ## Recommended defaults | Setting | Value | Note | Source | | --- | --- | --- | --- | | Document size | 8,192 encoded bytes in this example, checked before parsing | — | — | | JSON nesting | At most four containers in this example | — | — | | Items and amounts | 1 to 100 items; integer cents from 0 to 10,000,000; total at most 100,000,000 | — | — | | Python JSON decoding | `json.loads` with `object_pairs_hook` rejecting duplicates and `parse_constant` rejecting `NaN` and `Infinity` | — | Python json documentation | | Expat | The current release: 2.9.0, published 5 October 2026 | 2.6.0 is the minimum for reparse deferral, not a safe version: later releases fixed CVE-2024-45490, CVE-2025-59375 and more than 25 CVEs in 2026. | libexpat changelog | | XML features | No DTD, entity declarations, processing instructions or namespaces for this vocabulary | — | OWASP ASVS v5.0.0 V1.5.1 | ## Verify it 1. Run the project: `EXERCISE_MODULE=reference python3 -m unittest -v test_parsing.py` passes all 24 cases. 2. Send a document one byte over the cap, one level too deep and with 101 items; each must fail before business logic runs. 3. Send `{"currency":"EUR","currency":"USD"}` and `{"cents":true}` and confirm both are rejected with distinct reasons. 4. Send XML with an internal entity and with an external DTD reference and confirm both fail without any file or network access. 5. Print `python3 -c "import pyexpat; print(pyexpat.EXPAT_VERSION)"` on every runtime and compare it with the current Expat release. ## What you will learn - Define byte, depth, item and numeric limits for one document vocabulary - Reject duplicate fields and coercion before business use - Disable XML capabilities outside the vocabulary and test real parser behavior Audience: Developers implementing structured data imports and security engineers reviewing parser boundaries Before you start: - Read Python dictionaries, lists and exception handling - Distinguish syntax validation from business authorization - For the project: Python 3.13+ linked to Expat 2.6.0 or later, ideally the current release Time: About 29 minutes: 7 to read and 22 for the exercise, plus 20 to 45 for the local project ## Parsing produces meaning, not permission A JSON or XML parser accepts a language and turns it into values. The application still has to decide which values and structures belong to its business vocabulary. Syntax success does not establish that an invoice contains only allowed fields, that `cents` is an integer, or that the caller may submit an invoice for the chosen account. Arbitrary object deserialization adds a more dangerous capability: the representation can select types or behavior rather than just data. Python `pickle` is designed to reconstruct Python objects and must not be used for untrusted input; the fix cards at /fix/deserialization/ show data-only replacements. Replacing such a format with JSON narrows capabilities, but the following schema and resource checks remain necessary. This guide uses actual JSON and XML parsing and never demonstrates loading an untrusted executable object. ## Define a small vocabulary with quantitative bounds The project accepts an invoice with exactly `currency` and `items`. Currency is `USD` or `EUR`. There must be 1–100 items, each containing exactly `cents`, an integer from 0 through 10,000,000. The aggregate must be at most 100,000,000. A document is at most 8,192 encoded bytes; JSON nesting is at most four containers. These values are explicit fixture requirements, not universal invoice limits. The XML vocabulary expresses the same data as an `invoice` element with a `currency` attribute and `item` children with `cents` attributes. It permits XML 1.0 in UTF-8, whitespace and comments. It excludes DTDs, declared entities, processing instructions, namespaces, nested item children and non-whitespace text. XML `cents` must be 1–8 ASCII decimal digits before conversion. Decide on these rules before selecting a parser configuration. ```text JSON: {"currency":"USD","items":[{"cents":150},{"cents":225}]} XML: Normalized result: {currency: USD, items: [150,225], total: 375} Boundary order: bytes -> encoding/syntax/capabilities -> exact structure -> numeric limits -> normalized value Authorization and database commit occur after this boundary and are not provided by it. ``` ## Preserve ambiguity long enough to reject it A parser that retains only the last occurrence of a duplicate JSON name has already thrown away evidence before a normal dictionary validator runs. The reference uses Python’s `object_pairs_hook` to inspect member pairs and reject repeated names. `parse_constant` refuses non-finite constants accepted by Python’s permissive default JSON decoder; an integer-token bound limits conversion work. Exact numeric type matters. In Python, `bool` is a subclass of `int` and `int(True)` becomes 1. `int(1.5)` also loses a fractional part. The reference therefore checks `type(value) is int` before accepting cents, then applies the stated range and aggregate bounds. Conversion should not silently invent an allowed business value from a forbidden one. The depth precheck is a small state machine tracking structural braces and brackets outside quoted strings. It recognizes escaped quotes so string content does not inflate depth. It is only a resource guard; `json.loads` still validates JSON syntax. The byte cap comes first, limiting the amount of input the scanner and parser must examine. An HTTP endpoint must independently bound incoming and decompressed bytes before calling this function. ## Exclude unused parser capabilities at the parser boundary The reference uses Python’s Expat event interface for this narrow document vocabulary. It rejects a DOCTYPE declaration as soon as the parser reports it, disables parameter-entity parsing, and rejects external-entity callbacks. Event handlers construct only the allowed root and item structure; unexpected elements, attributes or text fail while parsing. This distinction matters even for a harmless entity that expands to `USD`. The fixture rejects the capability to declare entities, independently of whether a particular expansion looks benign. A schema validator applied after expansion would be too late to decide whether expansion was allowed. A case-insensitive substring search for a few declaration tokens is not a replacement for an XML parser. Check the underlying library version, not only the Python interpreter version. The project requires Expat 2.6.0 or newer and enables reparse deferral where supported. That is a capability floor, not a safe version: later releases fixed CVE-2024-45490 to CVE-2024-45492 (2.6.3), CVE-2024-50602 (2.6.4), CVE-2024-8176 (2.7.0), CVE-2025-59375 (2.7.2) and more than 25 CVEs in the 2026 releases 2.7.4 to 2.9.0. Run the current release, 2.9.0 as of 10 October 2026, and check what each runtime actually links with `pyexpat.EXPAT_VERSION`: on the machine used to verify this guide, Python 3.14.7 reported `expat_2.7.4` while the `python:3.11-alpine` image linked the newer system `expat_2.8.5`, because the interpreter build decides whether it bundles its own copy. Keep the interpreter and linked libraries maintained. For broader XML processing, choose a maintained hardened parser and configure each required capability explicitly; the XXE fix cards at /fix/xxe/ show that configuration for Java, .NET and Python. ## Run and repair the complete parser project Download the bounded data parsing project linked with this guide. It requires only Python’s standard library and opens no sockets or files containing user data. Run `python3 -m unittest -v test_parsing.py`. The starter passes five of twenty-four cases and fails nineteen: its syntax parser works, but it coerces amounts, drops unknown fields and accepts unsupported XML features. Repair `parse_invoice` in `exercise.py` against the written contract. Keep byte bounds, duplicate detection, parser capability choices and schema checks as distinct decisions. The reference command is `EXERCISE_MODULE=reference python3 -m unittest -v test_parsing.py`. All twenty-four cases pass. On PowerShell set the environment variable before running the same Python command; clear it before retesting your own code. The suite keeps ordinary JSON and XML, zero cents and exactly one hundred items as positive controls. It also supplies a tiny internal entity expanding to a three-character currency and an unused external-DTD declaration. The tests do not read a local file, fetch a network entity or generate an entity-expansion flood. Their purpose is to establish the configured acceptance boundary safely. ## Do not confuse a partial fix with the contract Dropping unknown fields can be appropriate for some versioned protocols, but it violates this exact-field invoice contract. A client-supplied `approved` field must not silently become a later authority flag. Conversely, changing an established extensible format to reject every unknown field is a compatibility decision requiring a documented policy; do not generalize this narrow example blindly. Checking each line item independently misses aggregate effects. Eleven values of 10,000,000 sum to 110,000,000, above the document cap despite valid individual ranges. Rejecting only negative numbers misses floats and booleans. Rejecting DTDs does not remove the need for byte and structure limits. Each check has a distinct counterexample. After parsing, pass the normalized result to separate authorization and transactional business logic. Do not use parsing success as an approval signal. Likewise, do not log an entire rejected document by default: length, content type, policy reason and a request identifier often provide enough diagnostic evidence without creating a second sensitive-data store. ## Use reason-specific results The outcomes below belong to this deliberately small vocabulary. Each rejection should have a useful internal reason while external error details follow the application’s disclosure policy. | Input | Expected result | Why | | --- | --- | --- | | 150 and 225 integer cents | Accept total 375 | Ordinary positive case | | `true` or `1.5` cents | Reject | No implicit numeric coercion | | Repeated `currency` name | Reject | Ambiguous member selection | | 101 items | Reject | Cardinality bound | | 11 × 10,000,000 cents | Reject | Total 110,000,000 exceeds 100,000,000 | | DTD declaring entity `USD` | Reject | Capability excluded even if payload is harmless | | Unknown XML attribute `approved` | Reject | Not in exact schema | ## Know where this implementation stops This project does not implement XML signatures, schema languages, namespace processing, XPath, multipart upload handling, decompression or an invoice storage service. It is a bounded example of translating two simple data syntaxes into one business object. A production integration must add service-level time/memory budgets and evaluate its actual parser features, document sizes and supported versions. Preserve the regression fixture whenever that vocabulary changes. ## Guided practice Review seven invoice candidates under the exact contract, repair the real parser, and explain which boundary rejects each case. No external documents or infrastructure are needed. ### Supplied material ```text J1 {"currency":"USD","items":[{"cents":150},{"cents":225}]} J2 {"currency":"USD","items":[{"cents":true}]} J3 {"currency":"EUR","currency":"USD","items":[{"cents":1}]} J4: 11 items, each cents=10,000,000; currency=USD. X1 X2 ]> X3 All candidates fit the byte cap. The caller’s permission is not supplied. ``` ### 1. Calculate the positive controls Determine normalized results for J1 and X1. Worked answer: J1 produces `USD`, `[150,225]`, total 375. X1 produces `USD`, `[1]`, total 1. The same business vocabulary can have two explicitly supported serializations. ### 2. Catch ambiguity and coercion Explain why J2 and J3 must fail before a downstream business action. Worked answer: J2 contains a Boolean, not integer cents; Python coercion would wrongly invent 1. J3 repeats `currency`; a normal dictionary might retain only `USD`. The pair-aware parser must reject the ambiguity before a single interpretation is assumed. ### 3. Apply the aggregate cap Compute J4 and identify why per-item validation does not suffice. Worked answer: 11 × 10,000,000 = 110,000,000, above the 100,000,000 document cap. Every individual amount is within range, so the aggregate check is independently necessary. ### 4. Separate capability from schema Give different rejection reasons for X2 and X3. Worked answer: X2 declares a DTD/entity, a parser capability excluded by the vocabulary even though this entity expands only to `USD`. X3 has an unknown `approved` attribute and fails the exact schema. Neither document is allowed to supply a server-side approval decision. ### 5. Repair and retain the evidence Run the downloadable project, then state what all 24 passing checks do and do not establish. Worked answer: They show this actual JSON/Expat implementation accepts the supplied positive controls and rejects the bounded negative cases. They do not establish caller authorization, safe database commit, general-purpose XML safety or resource behavior beyond the stated input limits. Keep those as separate controls. Deliverable: A seven-case decision table, a repaired parser with 24 passing tests, and an explanation of the authorization boundary that still follows parsing. Scope: Tiny synthetic documents, actual local parsers, no arbitrary object reconstruction or external entity retrieval. Limits and exact-schema rules are specific to the invoice vocabulary. ## Executable practice - [XXE lab: Parse an uploaded invoice without resolving external entities](https://softwaresecurity.com/labs/xxe-lxml/) - [Pickle deserialization lab: Load a session token without pickle](https://softwaresecurity.com/labs/pickle-deserialization/) - [Parse bounded JSON and XML invoices safely: project ZIP](https://softwaresecurity.com/downloads/projects/bounded-data-parsing-aa166a936b50b8b5.zip) — Python 3.13+ · Expat 2.6.0+ · standard library Articles and teaching material: CC BY 4.0. Original executable code: MIT. Attribution: Software Security. ## Sources - [Python JSON decoder and compliance behavior](https://docs.python.org/3/library/json.html). Python 3.15. - [Python XML security considerations](https://docs.python.org/3/library/xml.html). Python 3.15. - [Python Expat parsing callbacks and version behavior](https://docs.python.org/3/library/pyexpat.html). Python 3.15. - [Python pickle: trusted data requirement](https://docs.python.org/3/library/pickle.html). Python 3.15. - [libexpat changelog](https://github.com/libexpat/libexpat/blob/master/expat/Changes). Release 2.9.0, 5 October 2026. - [OWASP XML External Entity Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html) - [MITRE CWE-776: XML entity expansion](https://cwe.mitre.org/data/definitions/776.html). CWE 4.20.