Treat model releases as executable supply-chain inputs

A model release can include weights, tokenizer files, configuration, adapters, custom code and dependency instructions. Several loading paths execute code: Python’s pickle format, used by many .bin, .pt and .pkl checkpoints, can run arbitrary functions during unpickling (classic insecure deserialization: /fix/deserialization/), and repositories can ship modeling code that a loader imports. A familiar model name or a popular repository does not establish that the downloaded revision is the intended artifact or safe for a privileged environment.

Inventory the complete release unit. Record exact revisions and digests for weights, tokenizer, configuration and adapters, plus the loader and its dependencies. A signature or digest can support integrity or publisher identity under a trust policy; it does not prove the model behaves acceptably or that its training data was authorized. The OWASP Top 10 for LLM Applications 2026 lists these risks as Supply Chain (LLM04:2026).

Release R7 contains weights W7, tokenizer T7, configuration C7, adapter A7 and loader environment L7. A checksum of W7 establishes only those weights’ bytes. In candidate R8 the weights remain identical while the tokenizer and custom loader change. A promotion gate comparing only W7 would accept a materially different executable system.

The code below checks a whole release folder against an approved manifest of SHA-256 digests: any changed file, missing file or unlisted file stops the load, so an added modeling_custom.py or a swapped tokenizer is caught even when the weights are unchanged. The manifest must come from an authenticated release process: an attacker-controlled file and an attacker-controlled digest prove only that they agree with each other. /domains/supply-chain/provenance/artifact-signature-policy/ covers deciding which signers to trust.

Only after verification does it call from_pretrained with local_files_only=True, use_safetensors=True and trust_remote_code=False. A data-only weights format narrows one loading risk but does not turn adjacent custom code into data, which is why remote code stays disabled and the manifest also covers code files. Treat a request to enable remote code as a change that needs review of the code at a pinned commit, not as a routine fix for a load error.

Finally measure behavior. An authentic, signed release can still contain a poisoned adapter, regressions or unacceptable privacy behavior. Integrity, publisher authorization, load isolation and functional and security evaluation answer different questions. Promotion requires all of them, and rollback must restore the whole known-good combination rather than W7 plus whatever tokenizer resolves today.

import hashlib
from pathlib import Path

from transformers import AutoModelForCausalLM


def sha256(path):
    digest = hashlib.sha256()
    with open(path, "rb") as f:
        for block in iter(lambda: f.read(1 << 20), b""):
            digest.update(block)
    return digest.hexdigest()


def verify_snapshot(model_dir, manifest):
    """Every approved file must match its digest, and no other file may be present."""
    root = Path(model_dir)
    present = {
        p.relative_to(root).as_posix()
        for p in root.rglob("*")
        if p.is_file()
        and ".cache" not in p.relative_to(root).parts  # download metadata
    }
    expected = manifest[
        "files"
    ]  # {"model.safetensors": "<sha256>", "config.json": ...}
    problems = [f"unlisted file: {name}" for name in sorted(present - expected.keys())]
    for name, want in sorted(expected.items()):
        if name not in present:
            problems.append(f"missing file: {name}")
        elif sha256(root / name) != want:
            problems.append(f"digest mismatch: {name}")
    return problems


def load_approved(model_dir, manifest):
    problems = verify_snapshot(model_dir, manifest)
    if problems:
        raise RuntimeError("; ".join(problems))
    return AutoModelForCausalLM.from_pretrained(
        model_dir,  # the verified local copy, never a Hub name resolved at load time
        local_files_only=True,
        use_safetensors=True,  # refuse pickle checkpoints
        trust_remote_code=False,  # never import Python shipped with the model
    )

Download through a controlled acquisition step that pins the revision to a full commit hash and filters files, as below. allow_patterns keeps pickle checkpoints and repository Python out of the staging folder; list the exact JSON, tokenizer and safetensors files your model needs. Review license and use terms through the appropriate process, and never let a production worker resolve model names or branches from user input.

If a model ships only a pickle checkpoint, convert it to safetensors once, in an isolated environment, and review the result like any new release. Signing the approved release, for example with the OpenSSF model-signing tool, which records signatures in the Sigstore transparency log, lets later stages verify publisher identity as well as integrity.

import re

from huggingface_hub import snapshot_download


def acquire(repo_id, commit, staging="staging"):
    """Download one immutable revision into an isolated staging folder for review."""
    if not re.fullmatch(r"[0-9a-f]{40}", commit):
        raise ValueError("pin a full commit hash, never a branch or tag such as main")
    return snapshot_download(
        repo_id=repo_id,
        revision=commit,
        allow_patterns=["*.safetensors", "*.json"],  # no .py, .bin, .pt or .pkl files
        local_dir=f"{staging}/{repo_id.replace('/', '__')}-{commit[:12]}",
    )

Run new artifacts first in an isolated environment without production credentials, sensitive datasets or broad outbound access. Limit memory, disk, CPU and accelerator resources and startup time. Capture attempted network access and unexpected process or file activity. Review any exception before promotion rather than giving the loader broad authority to make the model work.

If you must read a PyTorch checkpoint directly, keep torch.load(path, weights_only=True), the default since PyTorch 2.6, and allow additional classes with torch.serialization.add_safe_globals rather than disabling the restriction. Treat adapters and fine-tunes as separate changes that need compatibility and behavior checks against the pinned base model, and keep a reproducible manifest so rollback returns to a known combination.

Describe the release in the same inventory formats as the rest of your software (/domains/supply-chain/sbom/generate-release-inventory/). CycloneDX supports machine-learning bills of materials that describe models, datasets and their configuration, and SPDX 3 has AI and dataset profiles. Record the base model and commit, adapters, training and evaluation dataset versions, loader dependencies and the approved digests, so an incident responder can find every deployment of a bad component.

Two 2024–2026 government baselines describe the same evidence. NIST SP 800-218A, the SSDF profile for generative AI, adds tasks to protect training data (PS.1.2) and model weights and configuration (PS.1.3) from unauthorized access and modification, suggesting hashes, signatures and multi-party authorization. In May 2026 the G7 Cybersecurity Working Group published Software Bill of Materials for AI – Minimum Elements, a voluntary list of model, dataset and infrastructure details to add to an ordinary SBOM.

Alter one byte of a test artifact and confirm the promotion path rejects it. Change a configuration file or add a code file while keeping the weights constant and confirm the manifest detects the different release. Try to load a pickle checkpoint and confirm the loader refuses it. Then run representative functional and security evaluations: a matching digest, a safe file format or a clean malware scan cannot establish accuracy, privacy or resistance to malicious behavior.

A sandbox observation with no unexpected network requests covers only the exercised load and inference paths. It does not prove every dormant path is harmless. Combine isolation with code/dependency review and permission constraints that remain present in production.

A model hub repository may include several files with different purposes. Do not assume all files are required, or that a loader fetches only the file you selected. Capture resolved dependencies and network acquisition during a controlled build so the manifest represents what actually ships.

Each row is a candidate release or claim and the promotion decision that the manifest check and evaluation must produce.

Sources & further reading

The standards, specifications and documentation behind the guidance above. Each is the evidence for a claim on this page; consult it for detail beyond the defaults and versions listed here.

1. OWASP Secure AI Model Ops Cheat Sheet
2. OWASP Top 10 for LLM Applications 2026 (LLM04 Supply Chain). 3 August 2026.
3. Hugging Face Transformers: from_pretrained parameters
4. PyTorch serialization semantics: weights_only loading. PyTorch 2.14.
5. Hugging Face Safetensors documentation
6. OpenSSF model signing (sigstore/model-transparency)
7. NIST SP 800-218A: Secure software development practices for generative AI and dual-use foundation models. July 2024.
8. G7 Cybersecurity Working Group: Software Bill of Materials (SBOM) for Artificial Intelligence – Minimum Elements. 12 May 2026.

Reference information

Tags

Unsafe deserializationSupply chain attacksSigning and software updatesModels and training data

SDLC phase

DevelopmentRelease

Role

DevelopersAppSec engineersPlatform engineers and SREs

ASVS 5.0.0 requirements

v5.0.0-1.5.2 · v5.0.0-15.2.4 · v5.0.0-15.2.5

Download this guide as Markdown

Continue exploring