Platform Engineering

Environment Variables Are the Wrong Place for Secrets

Key takeaway: Environment variables are readable by any process in the container, inherited by children, and dumped by diagnostic tooling. They are convenient configuration and poor secret storage.

Where Environment Variables Leak

The exposure is broader than most teams expect.

Any process running in the container can read the full environment, including a subprocess spawned for an unrelated purpose. Container inspection commands print environment variables in plain text to anyone with sufficient platform access. Crash handlers and error reporting libraries frequently capture the environment as diagnostic context and transmit it to a third-party service. Orchestrator specifications store the values, so anyone who can read a pod definition can read the secret.

None of these require a vulnerability. They are all normal behaviour of tools operating as designed.

The Rotation Problem

Environment variables are set at process start and cannot change afterwards. Rotating a secret therefore requires a restart of every consumer.

That coupling has consequences. Rotation becomes a deployment event requiring coordination, which means it happens rarely, which means secrets stay valid for long periods. The operational cost of rotating is what determines how often it happens, and environment variables make that cost high.

Delivery method Rotation without restart Visible in inspection Readable by child processes
Environment variable No Yes Yes
Mounted file Yes No Only with file access
Fetched at runtime from a vault Yes No No
Sidecar-injected with refresh Yes No Limited

Better Delivery Mechanisms

Mounted files are the simplest improvement. A secret projected as a file is not in the environment, does not appear in inspection output, and can be updated in place — the orchestrator refreshes the file and the application re-reads it. File permissions provide an additional boundary.

Runtime retrieval from a secret manager is stronger. The application authenticates using its workload identity and fetches what it needs, so the secret exists only in memory and never in any specification or image. Short-lived credentials become practical because the application can refresh them itself.

Dynamic credentials are the strongest arrangement. Rather than storing a database password, the application requests one that is generated on demand and valid for an hour. There is no long-lived secret to leak, and revocation is automatic through expiry.

The Practical Path

Most organisations cannot move everything at once, and the sequencing matters.

Start by moving secrets out of environment variables into mounted files, which is usually a small change and removes the largest exposure paths. Then introduce a secret manager with workload identity authentication so nothing is stored in manifests. Then adopt dynamic credentials for the data stores that support them.

Throughout, ensure the application re-reads rather than caching at startup, because otherwise you have changed the delivery mechanism without gaining the ability to rotate.

Configure error reporting to exclude the environment explicitly, since that path leaks secrets to an external party and is easy to overlook.

The Bottom Line

Move secrets from environment variables to mounted files as a first step, then to runtime retrieval with workload identity, and prefer dynamic short-lived credentials where the backing service supports them. Make the application re-read rather than cache, so rotation stops requiring a deployment.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button