An API deployment pipeline handles a great many secrets: gateway administration credentials, backend passwords, certificates, OAuth client secrets, tokens for access to repositories. In the chains we audit, whatever the gateway, they are often found in places that were not designed for them: CI variables, versioned configuration files, job parameters, even execution logs.
This article describes the path we follow to get these secrets out of pipelines. It relies on a secrets vault (HashiCorp Vault, or the equivalent offered by your platform) and on practices independent of the tools.
Why CI variables are not enough
CI tools all offer a mechanism for protected or masked variables. That is better than nothing, but it leaves several problems intact:
- masking is not protection: it avoids the accidental display of a value in the logs, but a script can transform it, write it to a file or send it elsewhere;
- secrets are static: they live for months, sometimes years, and their rotation is manual;
- access is broad: anyone able to change the pipeline can generally read the variables it uses;
- traceability is weak: one knows that a job used a secret, rarely who triggered it and why.
The principle: the pipeline owns no secret
The aim is that the pipeline stores no secret permanently. It obtains them when it needs them, from a vault, for a limited time, and only those it needs.
1. The repository contains only references
In the API repository, one never finds a value, only a reference: “the password of the billing backend in staging is at path X in the vault”. Reviewing a merge request shows which secret is used, without exposing it.
2. The CI authenticates with its identity, not with a secret
The classic paradox: to read secrets in the vault, the pipeline needs… a secret. It is resolved through identity federation: many CI tools can issue a signed token that describes the job (project, branch, environment), which the vault verifies before granting access. No master secret stored in the CI. Exact capabilities depend on your tool and its version: this should be checked.
3. Least privilege, per environment
The job that deploys to staging must not be able to read production secrets. Vault policies are written per project and per environment, and access to production secrets is reserved for jobs launched from the protected branch, after the planned approvals.
4. Short-lived secrets wherever possible
Some vaults can generate dynamic credentials: a database account or a token created for the job and revoked afterwards. Not every gateway or backend lends itself to this. When it is not possible, the aim is at least regular, automated rotation of static secrets, applying the overlap principle to avoid downtime.
Traceability: who did what, really
In many chains, every action on the gateway is performed by a single technical account. In an incident, the gateway log says “CI account”, which helps no one. We aim for end-to-end named traceability:
- every pipeline run is tied to the person who triggered or approved it;
- the vault logs every secret read, with the job’s identity;
- when the gateway allows it, the action is performed with an identity that reflects the user, or at least correlated with them.
The aim is not to monitor people, but to be able to reconstruct a sequence of events without conjecture.
Not letting secrets leak along the way
Getting secrets out of the repository is pointless if they reappear elsewhere. The points of attention we check:
- execution logs: no display of commands containing secrets, no verbose mode enabled in production;
- temporary files: a secret written to the disk of a build agent must be deleted, and shared agents should be avoided for sensitive environments;
- artefacts: a configuration archive must not embed sensitive values;
- Git history: a secret already committed must be treated as exposed and rotated, even after deletion.
A secret-detection tool run on every merge request usefully completes the setup.
The path, step by step
Doing everything at once is rarely realistic. The order we recommend:
- inventory the secrets used by the pipelines and their current locations;
- rotate those that have been exposed (Git history, logs);
- move the secrets into the vault, leaving references in the repository;
- federate the CI identity to remove the master secret;
- restrict policies per project and per environment;
- automate rotation, then introduce dynamic secrets where possible.
The duration of this path depends on the size of the estate and is defined with the teams.
What to take away
An API deployment pipeline should own no secret permanently: it obtains them at runtime, with its identity, for a minimal scope, and every access is traced. These principles depend neither on the gateway nor on the CI tool. They reduce the exposure surface and make rotations, and audits, far less painful.
Alongside its consulting work, CAB Ingénierie develops STOA, a sovereign iPaaS and API integration platform.
Let's talk about your project
Do your API pipelines still handle static secrets, or are you preparing an audit? Tell us about your context in a few lines: cab-ingenierie.com · contact@cab-ingenierie.com