Software Supply Chain Security Architecture in Depth

The chain, end to end

A control only makes sense once you can see where it sits. The chain runs: a developer’s commit, the dependency resolution that turns a manifest into actual bytes, the build that combines them, the metadata the build emits about itself, the registry that stores the result, and the admission decision that lets it run. Each hop is a place where what you intended and what you got can diverge.

Trace one release through it. A reviewed commit lands on a protected branch. CI resolves dependencies through a private proxy, checking each against the hash in the lockfile. The build runs in a fresh, network-isolated container over that verified input set, so the input list is complete by construction. The builder emits a signed provenance attestation naming the source, the commit, the builder identity and the resolved inputs, plus an SBOM generated from the finished artifact. The artifact is signed by digest and pushed. At deploy time an admission controller resolves the tag to a digest, checks the signature and the signer’s identity, evaluates the provenance against policy, and only then admits the workload. Nothing in that flow asks anyone to take a human’s word.

Four ways trust breaks

You do not need an exhaustive attack catalogue to design the defenses. Four failure shapes cover almost everything, and each maps to one control. (For enumerating a system’s own threats, see threat modeling; this section exists only to motivate what follows.)

Dependency compromise. A package you depend on — directly or four levels down the transitive graph — ships code nobody reviewed. The cause may be a maintainer account takeover, an abandoned package that changed hands, or a resolver that silently preferred a public name over your internal one. Control: pin by content hash, and control which registry you resolve from.

Build-system compromise. The source is clean but the environment is not: a reused runner carrying state from a previous job, a step that fetches an unpinned script at build time, a workflow file changed without review. Control: hermetic builds on ephemeral runners, plus provenance recording what built what.

Artifact tampering. The bytes change between build and deploy — in a registry, a mirror, a cache, or because a mutable tag was repointed. Control: sign by digest and verify at every consumption point.

Publishing-credential theft. Nothing in your build breaks; someone simply publishes under your identity with a stolen token. Control: have no long-lived publish credential to steal.

Pinning, lockfiles, and integrity hashes

A version constraint is not a pin. A range such as ^4.17.0 resolves to whatever is newest at install time, so two builds of the same commit can contain different code. Three levels exist and only the third is a real control. A range guarantees nothing. An exact version gives reproducible resolution, but a version number is a label, not an authenticator; it says nothing about the bytes behind it. Version plus integrity hash makes the resolver refuse anything whose content does not match, so republishing under an existing version fails loudly instead of silently. Every mature ecosystem has this: package-lock.json integrity fields, pip’s --require-hashes, go.sum backed by a checksum database, Cargo.lock, and container references by @sha256: digest rather than by tag.

# pip: with --require-hashes, an unhashed requirement is a hard error
requests==2.32.3 \
    --hash=sha256:63ac...   # wheel
    --hash=sha256:55365...  # sdist

# containers: pin the digest, because tags are mutable pointers
FROM python:3.12-slim@sha256:2a1f...

# CI steps are dependencies too - pin by commit SHA, not by tag
uses: actions/checkout@8f4b7f8...   # 40-hex commit, not @v4

Two things pinning does not do. It does not cover what the lockfile omits: install-time scripts, build plugins and CI actions routinely sit outside it, and an action pinned to a mutable tag is an unpinned dependency running with your job’s privileges. And pinning freezes your vulnerability exposure along with your builds. A pin without an update pipeline is how a service ends up years behind on a library with a known remote-code-execution advisory. Pair every pin with automated bump pull requests, batch low-risk ones weekly, and require human review for major versions and for any package whose maintainer set has changed.

Narrowing the intake: proxies, allowlists, vendoring

Pinning verifies what you fetched. Narrowing the intake reduces what you can fetch at all, and gives you one place to observe and enforce.

A pull-through proxy in front of every public index is the highest-leverage piece. It caches every artifact you have ever resolved, so an upstream deletion or republish cannot change your build; it produces one audit log of what entered the organisation; and it is the natural place to enforce an allowlist or a quarantine window on brand-new releases.

Scoped namespaces matter more than teams expect. Register internal package names or scopes in the private registry and configure resolvers never to fall back to a public index for them. A resolver that treats public and private sources as one flat namespace, preferring the higher version number, is the most common way an internal name gets satisfied by something you did not publish.

Vendoring — committing dependency source into your own repository — is the strongest intake control and the worst ergonomically: every upgrade becomes a reviewable diff in your own pull requests, which is exactly the point and exactly the cost. Reasonable for a small, slow-moving dependency set; unreasonable at twelve hundred transitive packages. Base images deserve the same thinking — mirror them, rebuild on a schedule, pin downstream to your mirror’s digests.

SBOM: inventory you can query, not a control

A software bill of materials is a machine-readable list of the components inside an artifact, in SPDX or CycloneDX. Be blunt about what it does: an SBOM prevents nothing. It is not a gate and not a signature. What it buys is answer latency at incident time. When an advisory lands on a widely used library at 2am, the question is which of your four hundred deployed artifacts contain it, at which versions, in which clusters — and an SBOM inventory answers in seconds instead of a day of grepping build logs.

Two details decide whether you get that payoff. Generate the SBOM from the built artifact, not the manifest: the manifest describes intent, the artifact contains reality, including base-image packages, statically linked libraries and vendored copies no manifest mentions. And key the inventory by artifact digest, joined to deployment data, so a query returns running workloads rather than file names. An SBOM attached as an attestation and mirrored into a queryable store is useful; one written to a build log is decoration.

Expect imperfection: detection quality varies sharply by ecosystem and version strings are often ambiguous for vendored code. Treat matches as a triage list, not a verdict.

Provenance attestations: in-toto and SLSA as a maturity model

Provenance answers a different question from the SBOM: not what is inside this artifact but what produced it. An in-toto attestation is a signed statement binding a subject (an artifact digest) to a typed predicate (the claims). SLSA provenance is one such predicate, carrying the source repository and commit, the builder identity, the build entry point, the parameters and the resolved inputs. The envelope is signed by the builder, not by a developer — the value is that the claim comes from the system that observed the build, not from someone asserting it afterwards.

Read SLSA’s build levels as a maturity ladder, not a certification. Level 1 means provenance exists and is available; alone it stops nothing, but it forces the build to be scripted and consistent. Level 2 adds signed provenance from a hosted build service, so it can no longer be fabricated on a laptop. Level 3 requires hardened, isolated infrastructure where the provenance signing key is unreachable from the build steps — the first level that meaningfully resists a compromised build step.

Sequence it instead of attempting a fleet-wide jump: Level 1 everywhere because it is nearly free, Level 2 on anything shipping outside the team, Level 3 on the few artifacts with the widest blast radius. The step teams skip is verification. Provenance nobody checks is a receipt in a drawer; policy must assert concrete fields — builder identity, expected source repository, protected ref.

Hermetic and reproducible builds

A hermetic build declares all its inputs and does not reach the network for anything undeclared. In practice: split the build into an explicit fetch phase that resolves and hash-verifies dependencies into a content-addressed cache, and a compile phase that runs with networking disabled against only that cache. Two properties follow — the build cannot silently acquire an input, and the input list in the provenance is complete rather than best-effort.

Reproducible goes further: the same declared inputs produce a bit-identical output on a different machine, at a different time, run by a different party. This is the strongest verification available anywhere in the stack, because it removes the builder from the trusted set. An independent rebuilder can rebuild your artifact and compare digests; if they match, no signature is needed to believe the artifact corresponds to the source, and if they diverge you know something between source and artifact is wrong without needing any insider access.

Getting there is real engineering. The recurring sources of nondeterminism, roughly in the order they bite: embedded build timestamps (fix with a set SOURCE_DATE_EPOCH); filesystem iteration order and metadata inside archives (sort entries, normalise uid, gid, mtime); absolute paths, hostnames and build-user names baked into debug information (build under a fixed root); locale- and timezone-sensitive formatting; parallel steps racing on a shared output; and randomised hash seeds in code generators. A useful intermediate goal is diff-reproducible: rebuild twice, diff, and drive the difference down to a short documented list of benign fields.

Signing at build, verifying at admission

Signing binds an identity to an artifact digest, and there are two operational models. Long-lived keys hand you a key-management problem: secure storage, rotation, trust-root distribution, and the uncomfortable fact that a leaked key stays silently valid until someone notices. Short-lived, identity-bound certificates — the Sigstore model — avoid that: the build presents an OIDC identity token, a certificate authority issues a certificate valid for minutes and bound to that workload identity, the signature and certificate are recorded in a public append-only transparency log, and the private key is discarded. No durable secret exists to steal, and the log makes issuance auditable after the fact.

The critical detail: verification must assert who signed, not merely that a signature exists. Anyone can sign anything, so a policy accepting any valid signature is close to no policy at all.

# sign at build - keyless, identity comes from the CI OIDC token
cosign sign --yes "$IMAGE@$DIGEST"

# verify - always pin the identity and the issuer
cosign verify \
  --certificate-identity-regexp \
    '^https://github.com/acme/[^/]+/.github/workflows/release.yml@refs/heads/main$' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  "$IMAGE@$DIGEST"

Enforcement belongs at admission, where the artifact is about to run. A Kubernetes admission controller such as Kyverno or Gatekeeper resolves the image reference to a digest, verifies the signature and identity claims, evaluates the provenance predicate against policy, and rejects the workload otherwise. Two operational notes. Resolve and pin the digest as part of admission, or a mutable tag can be repointed after the check — a time-of-check/time-of-use gap. And roll out in audit mode first, on one namespace, reading violations for a week: the first enforcement pass always surfaces a sidecar, an init container or a vendor image nobody knew was in the cluster.

Least-privilege CI: short-lived tokens, split release job

The highest-value change most teams can make is not another scanner. It is deleting the long-lived publish credentials sitting in CI secrets, because those turn a modest foothold into a release you did not make.

Federate instead of storing. Replace static registry and cloud tokens with workload identity federation: the job presents a signed OIDC token describing its repository, workflow and ref, and the registry or cloud IAM exchanges it for a credential that lives minutes and is scoped to one artifact path. The trust policy is where this succeeds or fails — pin the subject claim narrowly, because matching only the organisation, or wildcarding the ref, re-opens most of the hole you just closed.

Split build from release. Give the build job no publishing rights at all; it emits an artifact and a digest. A separate release job, gated by a protected environment and an approval, does the signing and publishing. A compromised test dependency then executes in a job that cannot publish anything.

Three rules that cost nothing: never expose secrets to workflows triggered by untrusted contributions, since fork pull requests should run in a no-secrets context; pin CI actions and build plugins by commit SHA, since they execute with your job’s privileges; and treat pipeline configuration as protected code, requiring review on workflow files and allowing release workflows only from protected refs. See privileged access management for the human-identity side of the same principle.

Detection, revocation, and rebuild-and-compare

Prevention is never complete, so design for the case where something gets through.

Monitor continuously, not at build time. A build that scanned clean today is a vulnerable artifact next Tuesday. Re-evaluate the SBOM inventory against advisory feeds on a schedule and alert on deployed digests, not repositories. Application-level testing is a separate layer with its own article, SAST and DAST; here the subject is code you did not write. Useful non-advisory signals arrive earlier: a package yanked or deprecated, a maintainer set that changed, a release published without a matching source tag, a version published from a different identity than previous ones.

Have a revocation path before you need one. Answer concretely: how do you mark a digest bad, and does that actually stop it deploying? A deny list keyed on digest, enforced by the same admission policy, is the mechanism — and it needs testing, because an untested revocation path fails on the day it matters. Rotation is the sibling question: OIDC-issued credentials rotate by construction, but anything still long-lived needs a rehearsed procedure coordinated with your secrets management tooling.

Rebuild and compare. On a suspected build-system compromise, rebuild from the inputs recorded in the provenance and compare digests. This is the only check that catches a compromise which never touched your source and never broke a signature — and it works only if you invested in reproducibility beforehand, which is the honest argument for doing that work early.