Secrets Management

Secrets management covers secure issuance, storage, delivery, rotation, revocation, and expiration of credentials. OWASP’s guidance emphasizes controlled access and lifecycle handling; reducing plaintext exposure requires checking every component that receives a value.

The rule the whole practice reduces to: configuration that is versioned, shared, or otherwise stored persistently references a secret; it does not contain the plaintext. A value that a running process loads from a secret store is a different case. It exists in plaintext at runtime, so it has to travel through controlled delivery and exposure paths. Everything else is about making that reference convenient enough that nobody is tempted to shortcut it.

With one qualification that is easy to miss and expensive to miss: referencing a secret is not the same as never handling it. If a tool reads the value during a run and passes it on, that tool has the value and may persist it — so the question to ask of any “we reference it now” claim is which component ends up holding the plaintext. The section on state files below is the concrete case.

Where secrets actually leak

The leaks that matter are rarely dramatic. They are places a secret ends up because someone did the obvious thing.

• Infrastructure state and plan files. Put a value in a definition and the tool records it — an infrastructure state file commonly contains initial database passwords, in plain text when stored locally. Reading the value from a secret store during the run does not by itself change this: if the tool receives plaintext and passes it to a persistent resource argument, it can be stored; supported omission mechanisms are exceptions. Terraform is explicit that values marked sensitive are still kept — Terraform “stores values with the sensitive argument in both state and plan files” and the marking only redacts output — and that a saved plan holds sensitive data “in cleartext.” Omitting values needs a mechanism built for it, such as Terraform’s ephemeral values, which are “available at runtime” but omitted “from state and plan files entirely,” subject to version and provider support.
• CI logs and build output. Anything printed for debugging, and anything a tool echoes back, persists wherever logs are kept and for as long as they are kept.
• Image layers. A file added and later deleted in a subsequent layer is still in the image.
• Error reports and diagnostics. A crash dump or an environment listing includes the process environment, which includes whatever was injected into it.
• Repository history. Removing a committed secret from the current files does not remove it from history, so the remediation includes revoking and rotating the secret, not deletion alone.

That last point generalizes usefully: once a secret has been somewhere it should not be, the response centers on revoking the old credential, issuing a new one, and confirming that the old value no longer works at the protected service. Removing the exposed copies from code, logs, images, and reports is part of the same response, but deleting them cannot recall a value that was already copied.

The lifecycle

A secret’s lifecycle includes creation, rotation, revocation, and expiration. Implement and test the later stages as well as issuance, because they govern how access ends.

• Creation. Generate credentials securely, assign only the required privileges, and deliver them through protected channels. This applies least privilege at issue time.
• Rotation. Automate replacement and consumer updates where possible. Replacing an encryption key may also require re-encrypting data, so plan that migration separately.
• Revocation. The ability to make a credential stop working now, without waiting for expiry — which is the same capability as access revocation and should be tested rather than assumed.
• Expiration. A secret that ends on its own is worth more than a rotation policy nobody runs.

Short-lived credentials can reduce the opportunity for reuse when their lifetime and enforcement are controlled. They do not bound damage already caused by a stolen credential.

Short-lived credentials can limit reuse, but the current TTL is not an unconditional exposure bound. Check renewal permissions and maximum lifetime, actual expiry enforcement, and whether target-side revocation succeeds. Restarting an application does not itself revoke its previous credential. Vault can fail to revoke a database credential when it cannot reach the database. Monitor failed revocations and verify that the target rejects new authentication; test existing-session behavior separately. Expiry cannot undo data already accessed.

Treat compromise response as an end-to-end capability: monitor revocation failures, confirm denial at the protected service, and determine whether active sessions also need termination.

Five principles worth adopting

• Standardize interactions. Multiple secret services can be appropriate. Maintain consistent access, delivery, and incident procedures across them without requiring one product or interface.
• Limit access. Use fine-grained permissions for people and workloads; reading a secret creates an additional exposure path.
• Automate handling. Reduce manual access to plaintext through controlled pipelines and runtime credential issuance.
• Audit the store and the protected service. Protect audit records against tampering and correlate events without logging secret values. Vault records requests and responses to its own API, including issuance, reads, renewals, and revocations. Database or API credential use, authentication failures, and attempts to reuse expired credentials require downstream service logs as well. Correlate credential or lease identifiers and timestamps, document collection gaps, and define behavior when logging fails. An audit log at the secret store alone cannot prove how an external service was used.
• Design for availability. Secret delivery affects both application startup and incident recovery. Set service objectives and recovery procedures for this dependency.

That last point deserves a decision rather than a hope: a secrets service on the recovery path needs its own availability answer and a documented break-glass route, because “we could not get the credentials” is an unacceptable sentence in an incident review.

Configuration is not a secret, and the line matters

An environment variable or mounted file can deliver either ordinary configuration or a secret. Classify the value, not the mechanism: even a hostname can be sensitive operational information. Version non-sensitive settings and restrict sensitive values according to their classification.

Ask what disclosure would enable or reveal: authentication, decryption, personal information, or sensitive system details. Needing to change a published value is a useful warning sign, not a complete definition of a secret.

Where a pipeline is involved, generation can be moved there — “you can use pipeline tooling to generate secrets and either offer them directly to the service deployed by the tooling or provide the secret to a secrets management solution.” A variant keeps secrets encrypted in git, which is workable under two conditions worth stating because they are commonly violated: “developers cannot decrypt the secrets themselves” and “every consumer of a secret has its encrypted variant,” with a different secret per environment encrypted under a different key so that “a secret does not leak cross-environment.”

Finally, automate detection rather than relying on review, since “shift-left and DevSecOps principles apply to secrets detection as well.” Two practical notes from the same guidance: enable detection at the developer’s machine to avoid the commit in the first place, and “create standard test secrets and use them universally across the organization,” which keeps false positives manageable by giving scanners one known value per secret type to ignore.

How this applies to infrastructure definitions specifically — where the state and plan files are the leak nobody expects — is worked through in The File That Says What Exists, alongside infrastructure as code more generally.

References

• Terraform documentation, Manage sensitive data in your configuration
• Vault documentation, Lease, renew, and revoke
• OWASP, Secrets Management Cheat Sheet
• Vault, Troubleshoot irrevocable leases
• Vault, Audit devices