AI Supply Chain Risks: Pickle Models and MCP Servers
On this page
Loading a model can run code
Here's a fact that surprises many developers: the most common way to save PyTorch models, Python's pickle format, can run arbitrary code the moment you load it. A pickle file doesn't just hold data. It holds instructions for rebuilding objects, and those instructions can call any Python function, including os.system.
Attackers know this. In 2024 JFrog found around 100 malicious models on Hugging Face whose pickle files opened a reverse shell or ran other payloads when loaded. In February 2025 ReversingLabs described "nullifAI", models using deliberately broken pickle files that slipped past Hugging Face's pickle scanner but still executed their payload.
LLM04, Supply Chain, covers this and more: everything in your AI stack that you didn't build yourself.
| Component | What can go wrong | Control |
|---|---|---|
| Model weights | Pickle payloads run on load | safetensors or GGUF, scan anything else |
| Fine-tuned adapters, LoRAs | Backdoored behavior (Day 17) | Trusted sources, evaluation before use |
| Datasets | Poisoned or mislabeled samples | Provenance, sampling and review |
| Python and npm packages | Typosquats, hijacked maintainers | Pin versions and hashes, lockfiles |
| MCP servers | Malicious code, or tool descriptions that change later | Pin versions, review, least privilege |
| Model APIs and gateways | A changed model behind the same name | Pin model versions, rerun evals on change |
See it without getting hurt
This builds a malicious pickle and then inspects it without loading it, using the standard library's pickletools to list the functions it would import:
import pickle, pickletools, io, os
class Payload:
def __reduce__(self):
return (os.system, ("echo pwned > /tmp/pwned.txt",))
blob = pickle.dumps({"weights": [0.1, 0.2], "x": Payload()}) # never pickle.loads() this
def pickle_imports(data: bytes) -> set[str]:
found, last_strings = set(), []
for op, arg, _ in pickletools.genops(io.BytesIO(data)):
if op.name in ("GLOBAL", "INST"):
found.add(arg.replace(" ", "."))
elif op.name in ("SHORT_BINUNICODE", "BINUNICODE", "UNICODE"):
last_strings = (last_strings + [arg])[-2:]
elif op.name == "STACK_GLOBAL" and len(last_strings) == 2:
found.add(".".join(last_strings))
return found
print(pickle_imports(blob)) # {'posix.system'} on Linux and macOSA model file that wants to call posix.system is not a model file. Real scanners such as picklescan and ModelScan work on this idea, with allowlists of the classes legitimate PyTorch checkpoints use. Nullifai shows their limit: scanners parse, and a broken file can fool the parser while Python's loader still runs the payload. So scanning helps, but changing formats is better.
Use model formats that can't run code
safetensors stores tensors as raw bytes plus a JSON header. Loading it never executes code:
import numpy as np
from safetensors.numpy import save_file, load_file
save_file({"w": np.array([0.1, 0.2], dtype=np.float32)}, "model.safetensors")
print(load_file("model.safetensors"))If you must load PyTorch checkpoints, keep torch.load(..., weights_only=True), which has been the default since PyTorch 2.6. It refuses to rebuild arbitrary objects. Never turn it off for files you didn't produce yourself.
Securing the rest of the AI supply chain
Packages. Pin exact versions with hashes in a lockfile and review new dependencies. AI coding assistants add a new twist: they sometimes suggest packages that don't exist, and attackers register those names, called slopsquatting (the AI-assisted coding course covers it).
MCP servers. An MCP server is code running with whatever access you give it, and its tool descriptions go straight into your model's context. In September 2025, researchers at Koi Security found the npm package postmark-mcp quietly added a BCC to an attacker's address in a later version, copying every email it sent. Pin server versions, read the code or use servers from vendors you trust, run them with minimal credentials, and alert when tool descriptions change between versions.
Hosted models. A provider can update the model behind an alias. Pin dated model versions where offered, and rerun your evals and red-team suite before switching.
Check yourself
With weights_only=False, any code in the pickle runs on load, with your user's permissions. Ask for a safetensors version, or inspect the file in an isolated sandbox first, and never disable the safety for untrusted files.
Tool descriptions are injected into the model's context, so a changed description can carry instructions or shadow other tools. New versions can also add hidden behavior, as postmark-mcp did.
Scanners parse the file, and attackers craft files that confuse the parser but still execute when Python loads them, as nullifAI showed. Formats that can't execute code remove the problem instead of detecting it.
Safetensors stores only tensors and metadata, so loading it can't execute code, unlike pickle-based formats. It removes a whole class of supply chain risk.