Catalog My learning Blog
AI Security Course: OWASP LLM Top 10 2026 and Agentic AI · Phase 4 · Lesson 16 · 4 min read

AI Supply Chain Risks: Pickle Models and MCP Servers

On this page
  1. Loading a model can run code
  2. See it without getting hurt
  3. Use model formats that can't run code
  4. Securing the rest of the AI supply chain
  5. Check yourself

Loading a model can run code

Here's a fact that surprises many developers: the most common way to save PyTorch models, Python's pickle format, can run arbitrary code the moment you load it. A pickle file doesn't just hold data. It holds instructions for rebuilding objects, and those instructions can call any Python function, including os.system.

Attackers know this. In 2024 JFrog found around 100 malicious models on Hugging Face whose pickle files opened a reverse shell or ran other payloads when loaded. In February 2025 ReversingLabs described "nullifAI", models using deliberately broken pickle files that slipped past Hugging Face's pickle scanner but still executed their payload.

LLM04, Supply Chain, covers this and more: everything in your AI stack that you didn't build yourself.

ComponentWhat can go wrongControl
Model weightsPickle payloads run on loadsafetensors or GGUF, scan anything else
Fine-tuned adapters, LoRAsBackdoored behavior (Day 17)Trusted sources, evaluation before use
DatasetsPoisoned or mislabeled samplesProvenance, sampling and review
Python and npm packagesTyposquats, hijacked maintainersPin versions and hashes, lockfiles
MCP serversMalicious code, or tool descriptions that change laterPin versions, review, least privilege
Model APIs and gatewaysA changed model behind the same namePin model versions, rerun evals on change

See it without getting hurt

This builds a malicious pickle and then inspects it without loading it, using the standard library's pickletools to list the functions it would import:

Python
import pickle, pickletools, io, os

class Payload:
    def __reduce__(self):
        return (os.system, ("echo pwned > /tmp/pwned.txt",))

blob = pickle.dumps({"weights": [0.1, 0.2], "x": Payload()})   # never pickle.loads() this

def pickle_imports(data: bytes) -> set[str]:
    found, last_strings = set(), []
    for op, arg, _ in pickletools.genops(io.BytesIO(data)):
        if op.name in ("GLOBAL", "INST"):
            found.add(arg.replace(" ", "."))
        elif op.name in ("SHORT_BINUNICODE", "BINUNICODE", "UNICODE"):
            last_strings = (last_strings + [arg])[-2:]
        elif op.name == "STACK_GLOBAL" and len(last_strings) == 2:
            found.add(".".join(last_strings))
    return found

print(pickle_imports(blob))      # {'posix.system'} on Linux and macOS

A model file that wants to call posix.system is not a model file. Real scanners such as picklescan and ModelScan work on this idea, with allowlists of the classes legitimate PyTorch checkpoints use. Nullifai shows their limit: scanners parse, and a broken file can fool the parser while Python's loader still runs the payload. So scanning helps, but changing formats is better.

Use model formats that can't run code

safetensors stores tensors as raw bytes plus a JSON header. Loading it never executes code:

Code
import numpy as np
from safetensors.numpy import save_file, load_file

save_file({"w": np.array([0.1, 0.2], dtype=np.float32)}, "model.safetensors")
print(load_file("model.safetensors"))

If you must load PyTorch checkpoints, keep torch.load(..., weights_only=True), which has been the default since PyTorch 2.6. It refuses to rebuild arbitrary objects. Never turn it off for files you didn't produce yourself.

Securing the rest of the AI supply chain

Packages. Pin exact versions with hashes in a lockfile and review new dependencies. AI coding assistants add a new twist: they sometimes suggest packages that don't exist, and attackers register those names, called slopsquatting (the AI-assisted coding course covers it).

MCP servers. An MCP server is code running with whatever access you give it, and its tool descriptions go straight into your model's context. In September 2025, researchers at Koi Security found the npm package postmark-mcp quietly added a BCC to an attacker's address in a later version, copying every email it sent. Pin server versions, read the code or use servers from vendors you trust, run them with minimal credentials, and alert when tool descriptions change between versions.

Hosted models. A provider can update the model behind an alias. Pin dated model versions where offered, and rerun your evals and red-team suite before switching.

Check yourself

With weights_only=False, any code in the pickle runs on load, with your user's permissions. Ask for a safetensors version, or inspect the file in an isolated sandbox first, and never disable the safety for untrusted files.

Tool descriptions are injected into the model's context, so a changed description can carry instructions or shadow other tools. New versions can also add hidden behavior, as postmark-mcp did.

Scanners parse the file, and attackers craft files that confuse the parser but still execute when Python loads them, as nullifAI showed. Formats that can't execute code remove the problem instead of detecting it.

Safetensors stores only tensors and metadata, so loading it can't execute code, unlike pickle-based formats. It removes a whole class of supply chain risk.