Parse an uploaded invoice without resolving external entities

An invoice importer parses uploaded XML with a parser that resolves external entities, so a crafted DOCTYPE reads a local file into the response (XXE). Repair parse_invoice so the parser never resolves external entities, loads no DTD and makes no network request.

Requires Node.js 22 or newer, which runs the tests, and Python 3.12 or newer, which runs your code (the command python3, or the program named in the PYTHON environment variable). This lab also needs lxml==6.1.3, the version the browser runs. Install it into a virtual environment: python3 -m venv .venv, then .venv/bin/pip install lxml==6.1.3, and run the tests with PYTHON=.venv/bin/python node --test exercise.test.mjs (on Debian or Ubuntu, install the python3-venv package first). No server needed. To work in a container instead, open the folder with VS Code Dev Containers or GitHub Codespaces: .devcontainer/devcontainer.json uses the Dev Containers Node.js 24 image, pinned by digest.
1. Unzip all files into one folder.
2. In that folder, run: node --test exercise.test.mjs
3. Edit invoice.py, then run the same command again.

The starter deliberately fails some tests. A successful repair passes every case, including cases that catch common wrong fixes. The browser runs this same grader with your code in Pyodide (CPython compiled to WebAssembly); locally it runs your code with python3. If you completed the optional first stage in the browser, Download my work adds your input as stage1-input.json and the tests include it. Only run code you wrote or reviewed.

1. Reproduce the file read
Run the starter. The crafted invoice declares an entity pointing at a local file, and the starter returns that file’s contents as the supplier name.

2. Turn off external entity resolution
Build an XMLParser with resolve_entities=False. An undefined external entity is then left unexpanded instead of being read from disk, so the secret file no longer appears in the result.

3. Close the DTD and network paths too
Add no_network=True and load_dtd=False, so the parser neither loads an external DTD nor fetches a network entity. Checking only for "file://" in the text is not enough: an entity can use a bare path or a different scheme.

4. Keep legitimate invoices working
Confirm ordinary invoices still parse, that &amp; and &#233; expand, and that a harmless DOCTYPE is not rejected.

SUPPLIED CONTRACT
parse_invoice(xml) takes the XML text of a purchase order and returns {"supplier": …, "total": …} from the <supplier> and <total> elements. Build an lxml.etree.XMLParser with resolve_entities=False, no_network=True and load_dtd=False, then etree.fromstring(xml.encode("utf-8"), parser). The built-in entities (&amp;, &lt;) and numeric references (&#233;) still expand, so legitimate markup keeps working; external entities declared in a DOCTYPE must not be resolved. Read each element with root.findtext(name) or "".

REFERENCE
reference.py contains an explained solution: With resolve_entities=False, no_network=True and load_dtd=False, an ordinary invoice returns its supplier and total, and &amp; and the numeric &#233; still expand (lxml always resolves the predefined and character entities). A DOCTYPE entity pointing at file://…/secret.txt or at a bare absolute path is left unresolved, so the supplier comes back empty rather than carrying the file’s contents; a network entity such as the cloud-metadata URL is never fetched. A harmless DOCTYPE with no entity still parses. External general entities are only read when a parser opts in with resolve_entities=True (lxml’s default since 5.0 is resolve_entities='internal'), so the fix is to opt out explicitly and keep DTD loading and network access off.

SCOPE
This lab parses one document. libxml2 (which lxml wraps) already caps entity amplification, so a classic billion-laughs document is rejected by default; still bound input size and parse time for untrusted XML. XInclude, XSLT document() and schema/DTD processing have their own external-reference switches that must also stay off for untrusted input. The behaviour here was confirmed against lxml 6.1.3 running in the browser (Pyodide); a server’s libxml2 build may differ, so test yours.

NEXT TASK
Which parsers in your stack accept uploaded or third-party XML (SAML, SOAP, office documents, feeds), and is external entity resolution disabled on each, by configuration rather than by inspecting the bytes?

SOURCE
https://lxml.de/FAQ.html
