Secrets in Docker: How to Manage Keys

\n To get an API key into a container, you can put it in a .env , pass it as a build argument, or copy your project folder in, and the application can read it. But nobody checks where else the key ended up on the way in. \ A key the container reads at runtime is fine, but a key written into an image layer, printed by docker history , or sitting in a .env copied in is a key you have published, which is a serious concern. The image is just a file and can be passed around, pushed to a registry, and handed to a teammate, and the secret travels with it. Even the Dockerfile pulled from a registry has history associated with it. \n The quickest way to see this is docker history --no-trunc on almost any image that was built in a hurry. Build arguments and ENV lines can be easily read. If a key was in the build, it is in the history, and the history ships with the image. \ This article walks through the places a secret leaks in a normal Docker workflow, how to check for each one, and the two patterns that actually keep keys out: BuildKit secret mounts for build time, and runtime injection for everything else. The examples use the local agent stack from my earlier Compose article , because it is a realistic case. Several services, a couple of real API keys, and a .env sitting next to the Dockerfile. 1. The places where a secret leaks. A project’s secret can leak in many different places, which you might not expect. \ These are five different cases An image layer, when the key is passed as ARG or set with ENV . Layers are immutable and readable. The image config, which docker history and docker inspect print without running anything. A file copied into the image, usually a .env that came along with COPY . . . Version control, when that same .env gets committed. Logs, when the application or the proxy prints the key on startup. \n Most teams have a .gitignore with .env in it and stop there. That covers one of the five cases mentioned above. The image-layer leaks are the ones people miss, because the key never looks like it was written down anywhere. It just swooped in with the build. 2. Build time and run time are different problems. Secrets are usually needed at one of two stages, and each stage should be handled differently. Most leaks happen when people treat them the same way. \ The most common case is runtime secrets. For example, your app may need ANTHROPIC_API_KEY while it is running so it can call a model API. That secret should not be inside the image. Instead, pass it when the container starts, and keep it only for as long as the container is running. \ Build-time secrets are less common and easier to misuse. Sometimes, the build process itself needs access to something private, like installing a private Python package, downloading protected model weights, or cloning a private repository. In these cases, the secret is only needed for a single build step. \ A common mistake is using ARG for secrets. It feels correct because ARG is meant for builds and disappears after the build finishes. But the value can still remain in the image history. ARG should not be treated as a secure way to handle secrets. 3. The key baked into a layer. Here is the version that feels right and is not: \n #dockerfile FROM python:3.12-slim WORKDIR /app # Looks build-time-only. It is not. ARG ANTHROPIC_API_KEY ENV ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY src ./src \ You build it by passing the key in: docker build --build-arg ANTHROPIC_API_KEY=sk-ant-api03-xxxx -t agent . \n The application can now read the key, and everything works. Then you look at what the image is carrying: docker history --no-trunc agent | grep -i anthropic # ... ENV ANTHROPIC_API_KEY=sk-ant-api03-xxxx \ docker inspect agent --format '{{json .Config.Env}}' # ["PATH=...","ANTHROPIC_API_KEY=sk-ant-api03-xxxx", ...] \n Neither command runs the container. Anyone holding the image file can read the key without starting anything. Push this to a registry, even a private one, and the key is sitting in your registry too. \ Removing the ENV line in a later layer does not help. Layers stack, and the earlier layer that wrote the key is still in the image. Docker's own documentation warns about the ARG : build-time variable values are visible to anyone with the image through docker history . The ENV line just makes it worse by also putting the key into the environment of every container you start from the image 4. The key copied by accident. The other common leak does not require any special mistake at all. A simple COPY . . is enough. Earlier, we talked about using a .dockerignore file to improve build speed. Security is the other reason it matters. If you copy the entire project directory into the image, files like .env often get copied too. \ #dockerfile COPY . . \ docker run --rm agent cat .env # ANTHROPIC_API_KEY=sk-ant-api03-xxxx# LANGFUSE_SECRET_KEY=sk-lf-xxxx \n The fix is a .dockerignore that keeps the .env (and the .git directory, which can carry committed secrets of its own) out of the build context: .env .env.* .git __pycache__ \n The .gitignore handles the version-control path. These two files overlap less than people assume. .dockerignore controls what gets added to the image, .gitignore controls what gets added to the repo, and a .env needs to be in both. \ My own stack copies requirements.txt and src explicitly rather than running COPY . . , which sidesteps this more by accident than by design. The .dockerignore is the version that holds up when someone later swaps in a COPY . . for convenience, and doesn't think about what else is in the folder. 5. Secrets you actually need at build time. Sometimes a build genuinely does need access to a credential, and ARG is still the wrong tool for it. \ Modern Docker builds already have a safer option: BuildKit secrets. With RUN --mount=type=secret , the secret is exposed only for a single command and is never written into an image layer. \ For example, imagine a build step that needs to download model weights using a Hugging Face token: #dockerfile # syntax=docker/dockerfile:1 FROM python:3.12-slim WORKDIR /app COPY requirements.txt ./ RUN --mount=type=secret,id=hf_token \ HF_TOKEN=$(cat /run/secrets/hf_token) \ pip install --no-cache-dir -r requirements.txt \ You pass the secret at build time, from an environment variable or a file: # from an environment variable already in your shell docker build --secret id=hf_token,env=HF_TOKEN -t agent . # or from a file docker build --secret id=hf_token,src=./hf_token.txt -t agent . \n The token is mounted at /run/secrets/hf_token only while that RUN step executes. It is not copied into the layer, not in the final image, and not in the history: docker history --no-trunc agent | grep -i hf_token # (nothing shows up) \n The # syntax=docker/dockerfile:1 line at the top is doing real work here. At first glance, it looks like a comment, but Docker actually treats it as a special instruction. Docker has different builders and Dockerfile parsers. \ Features like: RUN --mount=type=secret \ Or cache mounts are BuildKit features, not part of the older Docker builder. \ It tells Docker to use the BuildKit frontend, which is the thing that understands --mount . 6. Secrets at run time. For the common case, the key should never touch the image. It should arrive at startup and live only in the container's memory and environment. \n Compose does this through environment injection, which is what the agent stack already uses: services: agent: build: . environment: ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY} \n At docker compose up , Compose reads ANTHROPIC_API_KEY from your shell or from the .env next to compose.yaml and passes it into the container. The key is never baked. Rebuilds do not capture it. The LiteLLM proxy in the same stack reads its key the same way, through os.environ/ANTHROPIC_API_KEY in the config rather than a literal value written into the YAML. For local development, that is enough, and it is the right default. \ One important limitation is that environment variables are not hidden. Anyone with access to the container can see them through docker inspect , and processes with sufficient access on the host can also read them from /proc/<pid>/environ . \ On a personal development machine, that is usually acceptable. On a shared host or multi-tenant environment, it becomes a real security consideration. \n When it does matter, Compose secrets put the value in a file instead of the environment: services: agent: build: . secrets: - anthropic_key secrets: anthropic_key: file: ./secrets/anthropic_key.txt \n The secret shows up as a file at /run/secrets/anthropic_key inside the container, out of the environment, and out of the image. The application reads it from the file. That keeps the key off the docker inspect env dump, at the cost of a small change in the app, which now opens a file instead of reading a variable. Whether that trade is worth it depends entirely on who else can reach the host. 7. Once a key has leaked. Suppose one slipped through. It made it into a layer, a pushed image, or a commit. \ Treat it as compromised and rotate it. Editing the Dockerfile, squashing the layers, or force-pushing over the commit does not undo exposure. The image may already have been pulled. The registry may keep old layers around. The git object may be cached on a mirror. The only reliable fix is to issue a new key and revoke the old one. Scrubbing history afterward is cleanup, not remediation. \n This is the reason the build-time and runtime patterns above are worth the small upfront effort. Rotation is annoying on a good day, and on a key tied to billing, it can be much worse. Putting it together. - Keep secrets out of ARG and ENV . Both end up in docker history - Put .env in both .dockerignore and .gitignore . They guard different paths. - Use RUN --mount=type=secret when the build itself needs a credential. - Inject runtime keys at startup through environment variables, or Compose secrets when env visibility is a concern. - Audit with docker history --no-trunc and docker inspect . If the key shows up, it has leaked. - If a key ever leaked, rotate it. Cleanup is not remediation. \