---
key: "model-artifact-security"
title: "Treat model releases as executable supply-chain inputs"
url: "https://softwaresecurity.com/domains/ai-emerging/ml-security/model-artifact-security/"
api: "https://softwaresecurity.com/api/v1/articles/model-artifact-security.json"
domain: "ai-emerging"
topic: "AI security"
type: "Guide"
identifiers: ["CWE-502","CWE-494","CWE-829","A03:2025","A08:2025","LLM04:2026","V1.5.2","V15.2.4","V15.2.5"]
tags: ["Unsafe deserialization","Supply chain attacks","Signing and software updates","Models and training data"]
stages: ["build","release"]
roles: ["developer","appsec-engineer","platform-sre"]
license: "CC-BY-4.0"
license_url: "https://creativecommons.org/licenses/by/4.0/"
code_license: "MIT"
---

# Treat model releases as executable supply-chain inputs

Verify provenance, constrain loading and preserve the relationship between model weights, configuration and runtime dependencies.

Canonical URL: https://softwaresecurity.com/domains/ai-emerging/ml-security/model-artifact-security/

## In short

Treat a downloaded model as executable input. Pin an immutable commit, download only the weight and configuration files you expect, and check each against a digest from an authenticated release record before loading. Load safetensors with `trust_remote_code=False`, and never unpickle untrusted checkpoints. Run new models first without credentials or network access, then evaluate behavior: a valid signature proves who published the bytes, not what the model does.

- Supply Chain is LLM04:2026. Weights, adapters, tokenizers, configuration, datasets and loader code all belong to the release.
- Pickle checkpoints (`.bin`, `.pt`, `.pkl`) can run code while loading; safetensors files hold tensors only.
- `trust_remote_code=True` imports Python from the model repository into your process with its permissions.
- `torch.load` defaults to `weights_only=True` since PyTorch 2.6; setting it to `False` on untrusted files reopens code execution.
- Pin a full commit hash, not a branch or tag, and verify every file against an approved digest, including files you did not expect.
- A signature, for example from OpenSSF model signing, establishes publisher identity and integrity, not acceptable behavior.

## Recommended defaults

| Setting | Value | Note | Source |
| --- | --- | --- | --- |
| `revision` | A full 40-character commit hash | Branches and tags can move; a commit cannot. | Transformers `from_pretrained` documentation |
| `use_safetensors` | `True` | Loading fails instead of falling back to a pickle checkpoint. | Transformers `from_pretrained` documentation |
| `trust_remote_code` | `False` (the default) | Enable only after reviewing the code at a pinned commit. | Transformers `AutoConfig` documentation |
| `torch.load` `weights_only` | `True` (the default since PyTorch 2.6) | Allow extra types with `torch.serialization.add_safe_globals`, never by turning it off. | PyTorch serialization notes |
| Download filter | `allow_patterns` limited to `*.safetensors` and the exact JSON or tokenizer files needed | — | — |
| First run | No credentials, no outbound network, bounded memory and accelerator time | — | — |

## Verify it

1. Change one byte of an approved weight file and confirm the promotion step rejects it.
2. Add an unlisted `.py` file to the model folder and confirm loading stops.
3. Point the loader at a pickle-only checkpoint and confirm it refuses to load.
4. Check deployment configuration pins a full commit hash and keeps `trust_remote_code` off.
5. Run the candidate with no credentials and no outbound network, and record any attempted access.

## What you will learn

- Define the release unit beyond the weights file
- Separate authentic artifact identity from safe loading and behavior
- Design a promotion decision that detects mixed component revisions

Audience: Application, platform and software security engineers

Before you start:

- Understand artifact formats, deserialization, source revisions and signed release identity.

Time: About 18 minutes: 6 to read and 12 for the exercise

## A model package contains more than weights

A model release can include weights, tokenizer files, configuration, adapters, custom code and dependency instructions. Several loading paths execute code: Python’s pickle format, used by many `.bin`, `.pt` and `.pkl` checkpoints, can run arbitrary functions during unpickling (classic insecure deserialization: /fix/deserialization/), and repositories can ship modeling code that a loader imports. A familiar model name or a popular repository does not establish that the downloaded revision is the intended artifact or safe for a privileged environment.

Inventory the complete release unit. Record exact revisions and digests for weights, tokenizer, configuration and adapters, plus the loader and its dependencies. A signature or digest can support integrity or publisher identity under a trust policy; it does not prove the model behaves acceptably or that its training data was authorized. The OWASP Top 10 for LLM Applications 2026 lists these risks as Supply Chain (LLM04:2026).

## Worked release: unchanged weights, changed executable loader

Release R7 contains weights W7, tokenizer T7, configuration C7, adapter A7 and loader environment L7. A checksum of W7 establishes only those weights’ bytes. In candidate R8 the weights remain identical while the tokenizer and custom loader change. A promotion gate comparing only W7 would accept a materially different executable system.

The code below checks a whole release folder against an approved manifest of SHA-256 digests: any changed file, missing file or unlisted file stops the load, so an added `modeling_custom.py` or a swapped tokenizer is caught even when the weights are unchanged. The manifest must come from an authenticated release process: an attacker-controlled file and an attacker-controlled digest prove only that they agree with each other. /domains/supply-chain/provenance/artifact-signature-policy/ covers deciding which signers to trust.

Only after verification does it call `from_pretrained` with `local_files_only=True`, `use_safetensors=True` and `trust_remote_code=False`. A data-only weights format narrows one loading risk but does not turn adjacent custom code into data, which is why remote code stays disabled and the manifest also covers code files. Treat a request to enable remote code as a change that needs review of the code at a pinned commit, not as a routine fix for a load error.

Finally measure behavior. An authentic, signed release can still contain a poisoned adapter, regressions or unacceptable privacy behavior. Integrity, publisher authorization, load isolation and functional and security evaluation answer different questions. Promotion requires all of them, and rollback must restore the whole known-good combination rather than W7 plus whatever tokenizer resolves today.

```python
import hashlib
from pathlib import Path

from transformers import AutoModelForCausalLM


def sha256(path):
    digest = hashlib.sha256()
    with open(path, "rb") as f:
        for block in iter(lambda: f.read(1 << 20), b""):
            digest.update(block)
    return digest.hexdigest()


def verify_snapshot(model_dir, manifest):
    """Every approved file must match its digest, and no other file may be present."""
    root = Path(model_dir)
    present = {
        p.relative_to(root).as_posix()
        for p in root.rglob("*")
        if p.is_file()
        and ".cache" not in p.relative_to(root).parts  # download metadata
    }
    expected = manifest[
        "files"
    ]  # {"model.safetensors": "<sha256>", "config.json": ...}
    problems = [f"unlisted file: {name}" for name in sorted(present - expected.keys())]
    for name, want in sorted(expected.items()):
        if name not in present:
            problems.append(f"missing file: {name}")
        elif sha256(root / name) != want:
            problems.append(f"digest mismatch: {name}")
    return problems


def load_approved(model_dir, manifest):
    problems = verify_snapshot(model_dir, manifest)
    if problems:
        raise RuntimeError("; ".join(problems))
    return AutoModelForCausalLM.from_pretrained(
        model_dir,  # the verified local copy, never a Hub name resolved at load time
        local_files_only=True,
        use_safetensors=True,  # refuse pickle checkpoints
        trust_remote_code=False,  # never import Python shipped with the model
    )
```

## Separate retrieval from execution

Download through a controlled acquisition step that pins the revision to a full commit hash and filters files, as below. `allow_patterns` keeps pickle checkpoints and repository Python out of the staging folder; list the exact JSON, tokenizer and safetensors files your model needs. Review license and use terms through the appropriate process, and never let a production worker resolve model names or branches from user input.

If a model ships only a pickle checkpoint, convert it to safetensors once, in an isolated environment, and review the result like any new release. Signing the approved release, for example with the OpenSSF `model-signing` tool, which records signatures in the Sigstore transparency log, lets later stages verify publisher identity as well as integrity.

```python
import re

from huggingface_hub import snapshot_download


def acquire(repo_id, commit, staging="staging"):
    """Download one immutable revision into an isolated staging folder for review."""
    if not re.fullmatch(r"[0-9a-f]{40}", commit):
        raise ValueError("pin a full commit hash, never a branch or tag such as main")
    return snapshot_download(
        repo_id=repo_id,
        revision=commit,
        allow_patterns=["*.safetensors", "*.json"],  # no .py, .bin, .pt or .pkl files
        local_dir=f"{staging}/{repo_id.replace('/', '__')}-{commit[:12]}",
    )
```

## Test loading in a constrained environment

Run new artifacts first in an isolated environment without production credentials, sensitive datasets or broad outbound access. Limit memory, disk, CPU and accelerator resources and startup time. Capture attempted network access and unexpected process or file activity. Review any exception before promotion rather than giving the loader broad authority to make the model work.

If you must read a PyTorch checkpoint directly, keep `torch.load(path, weights_only=True)`, the default since PyTorch 2.6, and allow additional classes with `torch.serialization.add_safe_globals` rather than disabling the restriction. Treat adapters and fine-tunes as separate changes that need compatibility and behavior checks against the pinned base model, and keep a reproducible manifest so rollback returns to a known combination.

## Record provenance in a machine-readable form

Describe the release in the same inventory formats as the rest of your software (/domains/supply-chain/sbom/generate-release-inventory/). CycloneDX supports machine-learning bills of materials that describe models, datasets and their configuration, and SPDX 3 has AI and dataset profiles. Record the base model and commit, adapters, training and evaluation dataset versions, loader dependencies and the approved digests, so an incident responder can find every deployment of a bad component.

Two 2024–2026 government baselines describe the same evidence. NIST SP 800-218A, the SSDF profile for generative AI, adds tasks to protect training data (PS.1.2) and model weights and configuration (PS.1.3) from unauthorized access and modification, suggesting hashes, signatures and multi-party authorization. In May 2026 the G7 Cybersecurity Working Group published Software Bill of Materials for AI – Minimum Elements, a voluntary list of model, dataset and infrastructure details to add to an ordinary SBOM.

## Test integrity and useful behavior independently

Alter one byte of a test artifact and confirm the promotion path rejects it. Change a configuration file or add a code file while keeping the weights constant and confirm the manifest detects the different release. Try to load a pickle checkpoint and confirm the loader refuses it. Then run representative functional and security evaluations: a matching digest, a safe file format or a clean malware scan cannot establish accuracy, privacy or resistance to malicious behavior.

## Why a clean sandbox run and one file hash miss the release

A sandbox observation with no unexpected network requests covers only the exercised load and inference paths. It does not prove every dormant path is harmless. Combine isolation with code/dependency review and permission constraints that remain present in production.

A model hub repository may include several files with different purposes. Do not assume all files are required, or that a loader fetches only the file you selected. Capture resolved dependencies and network acquisition during a controlled build so the manifest represents what actually ships.

## Check the result against concrete cases

Each row is a candidate release or claim and the promotion decision that the manifest check and evaluation must produce.

| Case | Expected result | What it establishes |
| --- | --- | --- |
| X exact manifest | Eligible for remaining promotion checks | Component identity matches |
| Y same weights, new tokenizer/loader | New release review required | Executable surroundings matter |
| Z changed adapter | New release review required | Behavior-changing adapter is versioned |
| Signature valid from unauthorized publisher | Reject trust policy | Valid signature is not approved identity |
| Exact manifest but failed task/privacy evaluation | Do not promote | Integrity is not acceptable behavior |

## Guided practice

Review X, Y and Z under the stated promotion contract and identify the missing evidence in the release ticket.

### Supplied material

```text
Approved R7 manifest (symbolic component IDs): weights=W7 tokenizer=T7 config=C7 adapter=A7 loader=L7.
Candidate X: W7 T7 C7 A7 L7.
Candidate Y: W7 T8 C7 A7 L8; L8 imports repository custom code.
Candidate Z: W7 T7 C7 A8 L7.
All candidates report the marketing name Model-Blue. Promotion policy: exact approved manifest OR a newly reviewed and evaluated release.
Ticket: “R8 safe: W7 hash matches, model loads, and a signature validates.”
Observed loader imports L8 custom code.
Signature identity=publisher-unknown; approved identity=publisher-blue.
Behavior check: 10/10 ordinary summaries pass; tenant-isolation and canary tests not run.
Rollback script pins W7 but resolves tokenizer from latest.
```

### 1. Compare release units

Decide whether X, Y and Z are the same approved release.

Worked answer: Only X matches the full R7 manifest. Y changes tokenizer and executable loader; Z changes the adapter. Their unchanged W7 and shared display name do not make them R7.

### 2. Evaluate the signature evidence

Explain why “signature validates” does not satisfy the stated publisher policy.

Worked answer: Cryptographic verification must be paired with an allowed signer identity and artifact binding. `publisher-unknown` is not `publisher-blue`, so the trust decision fails even if the signature mathematics are valid.

### 3. Separate evaluation claims

State what the ten summary results establish and what they leave open.

Worked answer: They establish those ordinary summaries passed under the recorded environment. They do not establish tenant isolation, absence of canary disclosure, acceptable custom-code behavior or broader privacy properties. Run the relevant bounded evaluations before claiming those results.

### 4. Repair rollback

Specify what the rollback script must restore.

Worked answer: Pin and verify the complete R7 combination W7/T7/C7/A7/L7 and its dependencies/configuration. Resolving the tokenizer from `latest` can create an untested mixed release, so the current script cannot promise a return to R7.

Deliverable: A promotion decision for X, Y and Z, the missing evidence in the R8 ticket, and a corrected rollback specification.

Scope: Component IDs are symbolic, not real digests. The exercise does not load a model; the code above shows how the same checks run on real files.

Articles and teaching material: CC BY 4.0. Original executable code: MIT. Attribution: Software Security.

## Sources

- [OWASP Secure AI Model Ops Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Secure_AI_Model_Ops_Cheat_Sheet.html)
- [OWASP Top 10 for LLM Applications 2026 (LLM04 Supply Chain)](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/). 3 August 2026.
- [Hugging Face Transformers: from_pretrained parameters](https://huggingface.co/docs/transformers/main_classes/model)
- [PyTorch serialization semantics: weights_only loading](https://docs.pytorch.org/docs/2.14/notes/serialization.html). PyTorch 2.14.
- [Hugging Face Safetensors documentation](https://huggingface.co/docs/safetensors/index)
- [OpenSSF model signing (sigstore/model-transparency)](https://github.com/sigstore/model-transparency)
- [NIST SP 800-218A: Secure software development practices for generative AI and dual-use foundation models](https://csrc.nist.gov/pubs/sp/800/218/a/final). July 2024.
- [G7 Cybersecurity Working Group: Software Bill of Materials (SBOM) for Artificial Intelligence – Minimum Elements](https://www.bsi.bund.de/SharedDocs/Downloads/EN/BSI/KI/SBOM-for-AI_minimum-elements.html). 12 May 2026.