Container Supply Chain Security: Image Signing and Verification

1. Understand the Threat: Why Signing is Non-Negotiable

The container supply chain has several attack vectors:

• Tampered Base Images: An attacker could push a malicious version of a public base image (e.g., ubuntu:latest, node:16) to a public registry.
• Compromised Build Pipeline: If your CI/CD tool is breached, an attacker can inject code into your application and push a malicious image to your registry.
• Man-in-the-Middle Attacks: Images pulled over the internet could be intercepted and altered in transit.

Image signing and verification directly mitigate these risks by cryptographically proving that an image came from a trusted source (authenticity) and has not been altered since it was built (integrity).

2. The Mechanics: Signing with Cosign and Notary

The industry standard for container signing is Cosign (part of the Sigstore project), with Notary also being a common tool, especially in Docker ecosystems.

• How Cosign Works: Cosign uses public-key cryptography. Developers use a private key to generate a cryptographic signature for a container image. This signature is then pushed to the registry alongside the image. Anyone with the corresponding public key can verify the signature before running the container.

How to implement:

bash

# Generate a keypair (if not using keyless Sigstore)
cosign generate-key-pair

# Sign a container image after it's built
cosign sign --key cosign.key myregistry.com/myapp:v1.0.0

# Verify an image before deployment
cosign verify --key cosign.pub myregistry.com/myapp:v1.0.0

3. Integrate Signing into the CI/CD Pipeline (DevSecOps)

Signing shouldn’t be a manual step. It must be automated within your CI/CD workflow.

• Sign at the Build Stage: Immediately after the image is successfully built and scanned for vulnerabilities, the CI pipeline (e.g., GitHub Actions, GitLab CI, Jenkins) should call cosign sign. The private key must be stored securely as a secret in your CI system.
• Verify at the Deployment Stage: Before your deployment tool (e.g., ArgoCD, Flux, Jenkins) applies a Kubernetes manifest, it must verify the image signature using the public key. Deployment should fail if verification fails.

Why it matters: This creates a secure, automated gate that prevents unsigned or tampered images from ever reaching a runtime environment.

4. Enforce Policy with Admission Controllers (Kubernetes)

The most robust way to enforce verification is in the cluster itself using a Kubernetes Admission Controller.

• How it works: When a pod is deployed, an admission controller like Kyverno or OPA/Gatekeeper intercepts the request. It checks the image against a predefined policy (e.g., “all images must be signed by trusty-ci-key” or “must come from a trusted registry”). If the image fails the check, the pod is never created.

Example Kyverno ClusterPolicy:

yaml

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: check-image-signature
spec:
  validationFailureAction: Enforce
  rules:
  - name: verify-signature
    match:
      any:
      - resources:
          kinds:
          - Pod
    verifyImages:
    - image: "*myregistry.com/*"
      key: |-
        -----BEGIN PUBLIC KEY-----
        YOUR-PUBLIC-KEY-HERE
        -----END PUBLIC KEY-----

5. Beyond Signing: A Layered Security Approach

Signing is critical for integrity, but it’s one part of a holistic strategy:

• Image Scanning: Always scan images for vulnerabilities before signing them. Don’t sign images with critical CVEs.
• Software Bill of Materials (SBOM): Use Cosign to attach and verify an SBOM (e.g., using Syft). This provides deep transparency into image contents.
• Trusted Registries: Use private, secure registries and define policies that only allow images from these approved sources.

Conclusion: Build a Verifiable Supply Chain

Implementing container image signing and verification moves your security left, from a reactive to a proactive posture. By cryptographically attesting to the origin and integrity of every container you deploy, you build a verifiable and trusted software supply chain that can resist modern attacks, ensuring that what you built is exactly what is running in production.

---

FAQ: Container Supply Chain Security: Image Signing and Verification

Q1: Why can’t I just use image scanning? Why do I need signing too?
Image scanning and signing solve different problems. Scanning finds known vulnerabilities and misconfigurations inside the image. Signing verifies the origin and integrity of the image itself. Scanning tells you if an image is safe; signing tells you if it’s genuine and unaltered. You need both for a complete security posture.

Q2: What’s the difference between Cosign and Docker Notary?
Both tools provide signing capabilities, but Cosign (part of the Sigstore project) has become the modern de facto standard due to its simplicity, keyless option, and strong integration with the broader OSS ecosystem. Notary is still used, particularly in v1 of the Docker Content Trust system, but Cosign’s flexibility and developer-friendly approach have made it more popular for DevSecOps pipelines.

Q3: What is “keyless” signing with Sigstore?
Keyless signing is a revolutionary feature of Sigstore that removes the complexity of managing private keys. Instead of generating a keypair, your CI pipeline uses a temporary certificate issued by Sigstore’s certificate authority (Fulcio), which is tied to your identity (e.g., your GitHub identity in a GitHub Actions workflow). The signature is then stored in a transparency log (Rekor). This drastically simplifies getting started with signing.

Q4: How do I enforce signing in a Kubernetes cluster?
The only way to reliably enforce signing at the cluster level is by using a Kubernetes Admission Controller. Tools like Kyverno or OPA/Gatekeeper allow you to define policies that reject any pod attempting to run an image that does not have a valid signature from a trusted key. This acts as a final, mandatory gate for all deployments.

Q5: Where should the private signing key be stored?
The private key is a critical secret and must be stored in a secure secret manager integrated with your CI/CD system. Examples include:

• HashiCorp Vault
• AWS Secrets Manager / Azure Key Vault / GCP Secret Manager
• GitHub Secrets / GitLab CI Variables (protected)
  The CI pipeline should retrieve the key only at the moment of signing and never log or expose it.