An AI model security checklist should begin before a model reaches a development environment, not after it has been connected to customer data or production infrastructure. A model file may look like a self-contained artifact, but its security depends on the repository, dependencies, loading process, runtime permissions, and surrounding software supply chain.
This comparison helps engineering, DevSecOps, procurement, and product teams decide whether an external model or related AI artifact is ready for further evaluation. This article compares verification controls for model intake and proposes an approval workflow; it is not a complete 0-to-100 implementation tutorial.
Map the AI model supply chain and its threat exposure
Teams often focus on model weights and overlook the components that make those weights usable. An AI supply chain includes the artifact and its data assets, such as checkpoints, conversion files, labels, annotations, and training or evaluation data. It also includes the software and delivery layer: processing scripts, frameworks, libraries, dependencies, containers, build images, deployment configuration, registries, repositories, and download clients. For systems with tool access, plugins, APIs, accelerators, and connected services belong in the review as well. A batch classifier may depend on a tokenizer, container image, and preprocessing package, while a tool-using agent may add plugins, APIs, and credentialed services. These related components can introduce risk even when the model file appears unchanged.
MITRE describes AI supply-chain compromise as a threat area that can affect models, data, software, hardware, containers, registries, and connected tools—not only the final artifact consumed by an application 5.
NIST’s AI-focused secure-development guidance separates responsibilities among model producers, AI-system producers, and AI-system acquirers 4. In procurement, that allocation means acquiring a model does not transfer all responsibility or risk to the supplier. The deploying organization still needs to understand how the artifact enters its own environment.
Compare integrity controls instead of treating them as interchangeable
Controls differ in the assurance they add, the limitations they leave, the operational effort they require, and the workloads they suit. Matching available evidence to the approval risk makes clear why these controls are complementary rather than interchangeable.For each control, the comparison below makes its advantage explicit, distinguishes it from related controls, identifies its limitations, and indicates the conditions that should guide selection.
| Control | Threat addressed | Assurance provided | Blind spots | Operational effort | Appropriate use case |
|---|---|---|---|---|---|
| Revision pinning | Drift between review and retrieval | Identifies the repository state requested | Content benignity remains unverified | Low | Every controlled intake |
| Digest verification | Substitution or transfer corruption | Confirms that retrieved bytes match the reviewed artifact | Author intent and runtime behavior remain unresolved | Low | Required when an artifact is approved |
| Signatures | Unverifiable artifact origin or tampering | Can link an integrity claim to an identified signer when the signing process is trusted | Leaves suitability, code behavior, and workload safety unresolved | Moderate | Artifacts with an established signing process |
| Transparency records | Unrecorded or incomplete signing history | Can provide evidence that signing activity was recorded | Still requires local review of the artifact and its use | Moderate | Higher-assurance procurement and release workflows |
| Serialization review | Code execution during deserialization | Identifies risky loading paths and executable formats | Does not assess all repository code or model behavior | Moderate | Any artifact with custom or executable loading paths |
| Code and dependency scanning | Malicious code and known package weaknesses | Surfaces detectable patterns, vulnerable packages, and unexpected components | Cannot fully interpret behavior or prove absence of threats | Moderate to high | Repositories, containers, and build pipelines |
| Sandboxing | Excessive runtime access or containment failure | Limits network, credential, filesystem, and host exposure during evaluation | Does not make the artifact trustworthy or eliminate escape risk | High | Untrusted components and sensitive environments |
| Behavioral evaluation | Unsafe outputs or interactions in the target workflow | Produces controlled observations about model behavior | Future behavior across all inputs and environments remains untested | High | Models connected to tools, data, or automated actions |
| Rollback evidence | Inability to restore a known-good deployment | Preserves the version, configuration, and decision needed for reversal | Does not prevent compromise or new vulnerabilities | Moderate | Any production-bound approval |
*Table note: Supporting sources map to the control categories as follows: revision pinning and digest verification [1][2][6]; signatures and transparency records [1][2]; serialization review and scanning [1][3][6]; sandboxing and behavioral evaluation [1][4][6]; rollback evidence [1][4].*
Model Transparency distinguishes digest recomputation, signer identity, signatures, and transparency-log evidence as supply-chain signals 2. Donusoft’s proposed workflow would assess those signals against the artifact and workload.
Platform-specific guidance from Hugging Face recommends practices such as pinning reviewed revisions and checking model files before use 6. That guidance should be understood in its platform context, not as a universal guarantee for every repository or deployment architecture.
Serialization security deserves a separate decision
A key part of the pre-deployment review is understanding how the artifact is loaded. File format is not merely a storage preference; some serialization paths can trigger executable behavior during deserialization.
Pickle-based files are a notable example. Hugging Face’s pickle-scanning documentation explains that loading an untrusted pickle file can enable arbitrary code execution and that static inspection can identify potentially dangerous instructions without executing the file 3.
Before accepting a model, reviewers should identify its serialization path, assess whether safer tensor storage is available, and inspect custom model or tokenizer code. They should also examine conversion scripts, installation hooks, shell commands, remote-code requirements, and loading-time code. These checks inform loading risk; separate evaluation is still needed for workflow behavior.
The safetensors format is intended for tensor storage rather than executable object loading, so it can reduce exposure to the specific loading-time deserialization risk described in [3]. Repository code, dependencies, permissions, and broader runtime behavior still require review.
Scanning is evidence, not an approval decision
Scanning should be read as surface-specific evidence: pickle inspection can flag suspicious instructions, malware tools can detect known patterns, and dependency or container scanners can surface vulnerable or unexpected packages. Repository review and behavioral evaluation cover questions those tools cannot resolve.
Scan findings are most useful alongside revision and digest records, code review, sandboxing, and behavioral evaluation when the workload warrants them. A clean scan reports only the patterns and package weaknesses within its scope; repository behavior, permissions, tool use, and workload-specific outputs require separate review. Source [3] describes pickle scanning as an inspection aid, not as proof that the broader implementation is safe.
Assess the workload after reviewing the artifact. Compare data sensitivity, runtime permissions, available actions, human approval points, and consequences of failure. A classifier processing public text in an isolated batch job therefore requires a different approval record from an agent that can modify tickets, send messages, query internal databases, or execute code.
MITRE’s AI supply-chain perspective broadens this analysis beyond model files [5]. Hugging Face security guidance also recommends reviewing remote code and isolating untrusted components [6]. For sandboxing, record the actual boundaries: network policy, credentials, host integration, filesystem permissions, and escape resistance.
Proposed engineering design: intake, approval, and evidence
Donusoft’s proposed workflow combines the documented controls into an internal approval record. It is a design input that organizations can adapt to their architecture, procurement process, risk tolerance, and regulatory obligations.
Register the artifact, supplier, use case, and source. Pin the reviewed revision and record the artifact digest. Compute a local digest and compare it with the reviewed record. Verify available signatures and examine signer identity. Inspect serialization, loading behavior, and repository code. Run applicable dependency, container, and artifact scans. Evaluate the model in an isolated environment with restricted access. Record findings and assign an internal risk tier. Approve, reject, or escalate the artifact for additional review. Retain the evidence and define a rollback reference.
An AI model security checklist for approval meetings
Use this AI model security checklist to organize approval meetings. Record the source, owner, intended use, immutable revision, digest, signatures, serialization format, loading behavior, repository code, remote-code requirements, scan scope and limitations, evaluation boundaries, approval threshold, reviewer ownership, and rollback reference.
- [ ] Artifact identity, review evidence, and ownership are recorded.
- [ ] Serialization, repository code, dependencies, and scanning limits are reviewed.
- [ ] Evaluation access matches the model’s intended workload.
- [ ] Approval thresholds, reviewer ownership, and exceptions are explicit.
- [ ] A rollback artifact and deployment configuration are available.
Consider the same model in two approval meetings. If it classifies public text in an isolated batch job, the team may approve it after confirming the revision, digest, serialization path, and dependency findings within the review scope. This is a proposed approval threshold for that bounded example, not a generally sufficient security standard. If the model is placed behind a tool-using agent that can query internal records or send messages, the approval should require stronger isolation, credential controls, behavioral evaluation, explicit human approval points, and rollback evidence. The artifact is unchanged; the required evidence changes because the workload and available actions change.
Why verification must include rollback planning
Approval is not the end of the lifecycle. Repositories can change, dependencies can become unavailable, new vulnerabilities can be disclosed, and operational behavior can differ from initial evaluation.
Rollback planning should identify the previously approved version, its recorded digest, and the deployment image and configuration used with it. The plan should also specify how the team can disable the model without removing unrelated services, who can authorize an emergency rollback, and which evidence must be preserved after an incident or rejection.
Government guidance on AI and machine-learning supply-chain risks highlights controls such as trusted sources, hashes, signatures, scanning, testing, reproducibility, registries, and least-privilege practices 1. Their value in an approval decision depends on linking the evidence to operational records and a reversible deployment process.
Donusoft’s potential role in the verification workflow
Donusoft may connect model registries and deployment systems, automate evidence collection, add monitoring, and design rollback-aware architecture. The proposed focus is maintainable workflows and reviewable evidence; it does not represent certification or a measured security outcome.
Conclusion: verification is a workflow, not a single tool
An AI model security checklist helps software teams compare controls against the exposure of a specific artifact and workload. A low-risk batch classifier processing public text may warrant identity checks, serialization review, and limited scanning, while a tool-using agent handling sensitive data may justify signatures, broader dependency review, sandboxing, behavioral evaluation, and stronger rollback evidence.
Software teams should be able to identify what they received, understand how it loads and behaves, limit its access, document the approval decision, and restore a known version if conditions change. The approval record should make those judgments visible to the next reviewer and operational owner.
Sources
- [1] Artificial intelligence and machine learning: Supply chain risks and mitigations
- [2] Model Transparency: Supply chain security for ML
- [3] Pickle Scanning
- [4] Secure Software Development Practices for Generative AI and Dual Use Foundation Models: An SSDF Community Profile
- [5] AI Supply Chain Compromise — MITRE D3FEND / ATLAS
- [6] Hugging Face Hub Security Policy
Frequently Asked Questions
What is an AI model supply-chain attack?
An AI model supply-chain attack is a compromise involving an AI artifact or one of the systems that produces, distributes, loads, or connects to it. The affected component may be model weights, training data, dependencies, containers, registries, hardware, or agent tools—not only the model file itself [5].
Is a signed AI model automatically safe?
No. A signature can provide evidence about integrity and signer identity for a defined artifact. It does not prove that the model is safe for a particular workload, that its code is benign, or that its outputs are appropriate.
Are safetensors models risk-free?
No. Safetensors can address specific risks associated with executable serialization paths, but teams must still review repositories, dependencies, custom code, permissions, and runtime behavior. Safer serialization is a useful control, not a complete security assessment [3].
Can malware scanning detect malicious model behavior?
Not necessarily. Scanning can identify some detectable threats, but it cannot replace code review, threat modeling, isolated evaluation, or permission controls. Retain scan findings alongside the other evidence in the approval record.
What should teams retain after approving a model?
Teams should preserve the reviewed revision, file digest, signature and verification results, scan findings, serialization and code-review notes, reviewer decision, deployment identifier, and rollback information. This record supports later investigation and controlled reversal.
;)
