CI/CD Secrets Management with HashiCorp Vault
This gets secrets out of CI/CD environment variables and .env files and into HashiCorp Vault. CI authenticates with GitHub's OIDC tokens (or AppRole where OIDC isn't available) and receives short-lived dynamic credentials in place of long-lived static secrets.
Enabling KV v2 and writing your first versioned secret
Vault's KV v2 secrets engine is the starting point: a versioned key-value store where overwriting a secret doesn't destroy history and you can roll back.
# Skip this line on a `vault server -dev` instance: dev mode already mounts
# secret/ as kv-v2, and enabling it again fails with "path is already in use".
vault secrets enable -path=secret kv-v2
vault kv put secret/ci/payments-api \
api_key="sk_live_..." \
webhook_secret="whsec_..."
vault kv get secret/ci/payments-api
vault kv get -version=1 secret/ci/payments-apiThat's already better than a CI env var. vault kv metadata get secret/ci/payments-api shows every version and when it was created, deleted or destroyed. It doesn't show who wrote it. For that you need Vault's audit log, which is off until you enable an audit device:
vault audit enable file file_path=/var/log/vault_audit.logEvery request and response is then logged as JSON with the calling identity (entity, auth method, policies) and the path. Secret values are HMAC-hashed rather than written in clear. Enable it before anything else touches Vault, because without an audit trail you can't reconstruct what happened after a leak. Make sure the Vault process can write to that path, and ship the file somewhere durable.
Versioning helps, but this is still a static secret sitting at a path. What matters more is controlling who can read it and how CI proves it's allowed to.
A policy that grants exactly what the pipeline needs
A human logging into Vault uses their own identity through userpass, OIDC or whatever your org runs. A CI pipeline has no identity of that kind, and giving it a personal token (or worse, a shared root token) only moves the credential into a CI secret field. Whatever auth method the pipeline uses, it ends up holding a token bound to a policy like this one:
# policy: ci-payments-read.hcl
path "secret/data/ci/payments-api" {
capabilities = ["read"]
}
path "secret/metadata/ci/payments-api" {
capabilities = ["read", "list"]
}
# dynamic database credentials, covered further down
path "database/creds/ci-readonly" {
capabilities = ["read"]
}vault policy write ci-payments-read ci-payments-read.hclGitHub Actions: OIDC instead of a stored Vault credential
GitHub Actions can mint a short-lived OIDC token (a signed JWT) for every job, with claims that say which repository, branch and workflow is running. Vault's JWT auth method verifies that token against GitHub's public keys and returns a Vault token. Nothing is stored in GitHub secrets, so there's no credential to leak or rotate.
vault auth enable jwt
vault write auth/jwt/config \
oidc_discovery_url="https://token.actions.githubusercontent.com" \
bound_issuer="https://token.actions.githubusercontent.com"
vault write auth/jwt/role/ci-payments-pipeline - <<'EOF'
{
"role_type": "jwt",
"bound_audiences": ["https://github.com/example-org"],
"bound_claims": {
"repository": "example-org/payments-api",
"ref": "refs/heads/main"
},
"user_claim": "repository",
"token_policies": ["ci-payments-read"],
"token_ttl": "15m",
"token_max_ttl": "30m"
}
EOFbound_claims does the access control here. Only jobs from example-org/payments-api running on main can log in with this role, and a fork, a feature branch or another repo in the org gets rejected. With token_ttl=15m, the resulting Vault token expires shortly after a normal job would finish. Vault needs outbound HTTPS to token.actions.githubusercontent.com to fetch GitHub's signing keys, and the runner needs to reach Vault. In practice that means a self-hosted runner or a Vault endpoint exposed to GitHub's runners.
# .github/workflows/deploy.yml
name: Deploy payments-api
on:
push:
branches: [main]
permissions:
id-token: write # lets the job request a GitHub OIDC token
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Fetch secret from Vault
uses: hashicorp/vault-action@v4
with:
url: https://vault.internal.example.com:8200
method: jwt
role: ci-payments-pipeline
jwtGithubAudience: https://github.com/example-org
secrets: |
secret/data/ci/payments-api api_key | PAYMENTS_API_KEY ;
secret/data/ci/payments-api webhook_secret | PAYMENTS_WEBHOOK_SECRET
- name: Deploy
run: ./scripts/deploy.sh
env:
PAYMENTS_API_KEY: ${{ env.PAYMENTS_API_KEY }}
PAYMENTS_WEBHOOK_SECRET: ${{ env.PAYMENTS_WEBHOOK_SECRET }}jwtGithubAudience must match bound_audiences on the Vault role, otherwise the login fails with an audience error. hashicorp/vault-action handles the login and masks the values in the Actions log. The secret arrives as a step output or env var and is never written to the runner's disk. If you call Vault's HTTP API directly, do the same and pipe the response into an environment variable or a build tool's secret input instead of a temp file you might forget to clean up.
Resist the urge to echo $PAYMENTS_API_KEY even temporarily while debugging a pipeline. Most CI systems only mask known secret values: one fetched fresh from Vault in that same job often isn't registered as a masked string yet, and a failed step's error output can dump environment context verbatim. The safest debug is confirming the variable is non-empty: test -n "$PAYMENTS_API_KEY" && echo "set".
AppRole: for runners that can't do OIDC
Some CI systems can't issue an OIDC token, an older Jenkins or a self-hosted runner fleet for example. For those, use AppRole, which splits authentication into two pieces. The role_id identifies the role, isn't secret, and is safe to put in a pipeline config. The secret_id is secret, short-lived and generated on demand.
vault auth enable approle
vault write auth/approle/role/ci-payments-pipeline \
token_policies="ci-payments-read" \
token_ttl=15m \
token_max_ttl=30m \
secret_id_ttl=10m \
secret_id_num_uses=1
vault read auth/approle/role/ci-payments-pipeline/role-idWith secret_id_num_uses=1 and secret_id_ttl=10m, a secret_id is single-use and expires within ten minutes. That means it can't be stored in a CI secret field, because the first run would consume it and the next one would fail. Instead, a trusted orchestrator (the Jenkins controller, a deploy service, anything that already has its own Vault identity) generates a fresh one per job and hands it over response-wrapped:
# On the orchestrator: its own token needs "update" on
# auth/approle/role/ci-payments-pipeline/secret-id
WRAP_TOKEN=$(vault write -wrap-ttl=60s -field=wrapping_token \
-f auth/approle/role/ci-payments-pipeline/secret-id)
# ...inject $WRAP_TOKEN into the job, never into its logs
# Inside the job:
SECRET_ID=$(vault unwrap -field=secret_id "$WRAP_TOKEN")
VAULT_TOKEN=$(vault write -field=token auth/approle/login \
role_id="$VAULT_ROLE_ID" secret_id="$SECRET_ID")
export VAULT_TOKENA wrapping token can only be unwrapped once. If the job's unwrap fails because the token was already used, assume the secret_id was intercepted in transit. Even then, the attacker gets read access to one Vault path for fifteen minutes, which is far less damage than root access to production.
Never use Vault's initial root token for day-to-day access, CI included. It bypasses every policy you write. Root tokens exist for bootstrapping and emergency break-glass: generated on demand, revoked immediately after, never stored, not even in Vault itself.
Dynamic secrets: stop handing out the password at all
OIDC and AppRole handle authenticating to Vault. Dynamic secrets go further, because the database password itself shouldn't be static either. With the database secrets engine, Vault creates a short-lived database user for each CI run, backed by a real CREATE ROLE on your Postgres instance.
vault secrets enable database
vault write database/config/payments-db \
plugin_name=postgresql-database-plugin \
connection_url="postgresql://{{username}}:{{password}}@db.internal:5432/payments" \
allowed_roles="ci-readonly" \
username="vault-admin" \
password="..."
vault write database/roles/ci-readonly \
db_name=payments-db \
creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
default_ttl="15m" \
max_ttl="30m"
vault read database/creds/ci-readonlyThat last command is what CI runs instead of reading a stored password; it gets back a username and password that exist only for this lease, scoped to exactly the grants in creation_statements.
These TTLs match the CI token's on purpose. A lease is a child of the token that requested it. When that Vault token expires (15 minutes, 30 at most) or is revoked, Vault revokes every lease it created, whatever the lease's own TTL says. Setting default_ttl="1h" here would promise a duration Vault can't give a CI caller, and it would mislead whoever reads the role later. Size the database role's TTLs to fit within the token that will request them. If a job needs longer, raise token_ttl on the auth role as well as the database TTL.
Leases and revocation: making "expired" actually mean something
Every dynamic secret Vault issues comes with a lease, a tracked object with a TTL that Vault itself manages, so you can see and control it directly, rather than trusting the pipeline to let go of it.
vault list sys/leases/lookup/database/creds/ci-readonly
vault lease revoke -prefix database/creds/ci-readonly
vault lease renew database/creds/ci-readonly/abcd1234Renewing a lease extends it up to the role's max_ttl, but never past the life of the token that created it. When a lease's TTL runs out, or its parent token expires, Vault (with the database plugin) drops the corresponding Postgres role automatically, so the credential is deleted outright. If you suspect a leak, vault lease revoke -prefix revokes every credential issued under that path immediately.
Where this leaves you
A single dev-mode Vault instance is enough to try this before rolling it out further (skip the secrets enable -path=secret step there, since it's already mounted). Once it's in place with an audit device enabled, each credential issuance is logged and any lease can be revoked on demand. The credentials also expire before they have time to go stale.
Want to actually run this in production?
This tutorial covers the concepts and architecture. If you want to implement it in your own infrastructure, or get good enough to own this problem long-term, I offer 1:1 mentoring built around your real environment, not a generic course.
This tutorial
- Core architecture & key concepts
- Illustrative code snippets
- The reasoning behind each decision
1:1 mentoring
- Working sessions on your own environment
- Direct answers to the edge cases you're hitting
- Feedback on your actual implementation
- Ongoing support as you build it out