Welcome to from-docker-to-kubernetes

Docker Secrets Management

Comprehensive guide to securing sensitive data in Docker environments using secrets management best practices and tools

H A R S H H A A
@NotHarshhaa

Introduction to Docker Secrets Management

Secrets management addresses one of the most critical security challenges in containerized environments: how to securely store, distribute, and access sensitive data such as API keys, passwords, certificates, and encryption keys. Docker containers require special considerations for secrets handling:

  • Ephemeral nature: Containers are regularly destroyed and recreated
  • Immutable artifacts: Container images should not contain embedded secrets
  • Deployment portability: Secrets must work across different environments
  • Access control: Limit secret visibility to only the containers that need them
  • Audit capabilities: Track how and when secrets are accessed

This guide explores the approaches, tools, and best practices for implementing robust secrets management in Docker environments, helping you protect sensitive information throughout the container lifecycle.

Docker Native Secrets

Docker Swarm Secrets

Docker Swarm provides a built-in secrets management feature:

# Create a secret from a file
echo "myStrongPassword123!" > password.txt
docker secret create db_password password.txt
rm password.txt  # Remove the file immediately

# Create a secret directly from command line input
echo "myStrongPassword123!" | docker secret create db_password_cli -

# List available secrets
docker secret ls

Secrets are then made available to services:

# Create a service with access to the secret
docker service create \
  --name db \
  --secret db_password \
  --secret source=db_password_cli,target=admin_password \
  postgres:14

Inside the container, secrets appear as files in the /run/secrets directory:

# Read the secret inside the container
docker exec -it $(docker ps -q -f name=db) cat /run/secrets/db_password

Docker Swarm secrets provide:

  1. Encrypted storage: Secrets are encrypted at rest in the Swarm manager's Raft log
  2. Secure distribution: Secrets are securely transmitted to containers over TLS
  3. Memory-only access: Secrets are stored in an in-memory filesystem inside containers
  4. Fine-grained control: Specify which containers get access to which secrets
  5. Secret rotation: Update secrets by creating new versions

Docker Compose Integration

Docker Compose provides secrets integration for development environments:

Development Mode

In development mode, secrets are mounted from local files. In production mode with Swarm, the secrets must be created separately and referenced as external.

External Secrets Management Tools

HashiCorp Vault Integration

HashiCorp Vault provides comprehensive secrets management beyond Docker's native capabilities:

# docker-compose.yml for Vault server
version: '3.8'
services:
  vault:
    image: vault:latest
    ports:
      - "8200:8200"
    environment:
      - VAULT_DEV_ROOT_TOKEN_ID=myroot
      - VAULT_DEV_LISTEN_ADDRESS=0.0.0.0:8200
    cap_add:
      - IPC_LOCK
    volumes:
      - vault-data:/vault/data

volumes:
  vault-data:

Using Vault with Docker containers:

# Store a secret in Vault
docker exec -it vault vault kv put secret/myapp/db password="securepassword123"

# Retrieve a secret from Vault
docker exec -it vault vault kv get secret/myapp/db

For container integration, several patterns exist:

  1. Envconsul: Populates environment variables from Vault
    docker run --rm \
  --name envconsul \
  hashicorp/envconsul:latest \
  -vault-addr=http://vault:8200 \
  -secret=secret/myapp/db \
  -upcase \
  my-application
  2. Consul Template: Renders configuration files from templates with Vault secrets
    docker run --rm \
  --name consul-template \
  hashicorp/consul-template:latest \
  -vault-addr=http://vault:8200 \
  -template="/templates/config.json.tpl:/app/config.json" \
  my-application
  3. Init Containers: Fetch secrets during container initialization
    FROM vault:latest AS vault-client

FROM alpine:latest
COPY --from=vault-client /bin/vault /bin/vault
COPY fetch-secrets.sh /
RUN chmod +x /fetch-secrets.sh

ENTRYPOINT ["/fetch-secrets.sh"]
CMD ["my-application"]

AWS Secrets Manager

For Docker deployments on AWS, integration with AWS Secrets Manager provides cloud-native secrets handling:

# docker-compose.yml with AWS credentials
version: '3.8'
services:
  webapp:
    image: my-webapp:latest
    environment:
      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
      - AWS_REGION=us-west-2

Application code to retrieve secrets:

# Python example for retrieving AWS secrets
import boto3
import json

def get_secret(secret_name):
    client = boto3.client('secretsmanager')
    response = client.get_secret_value(SecretId=secret_name)
    return json.loads(response['SecretString'])

# Get database credentials
db_secret = get_secret('myapp/database')
username = db_secret['username']
password = db_secret['password']

Azure Key Vault

For Docker deployments on Azure, Key Vault provides secure secrets storage:

# docker-compose.yml for Azure Key Vault integration
version: '3.8'
services:
  webapp:
    image: my-webapp:latest
    environment:
      - AZURE_TENANT_ID=${AZURE_TENANT_ID}
      - AZURE_CLIENT_ID=${AZURE_CLIENT_ID}
      - AZURE_CLIENT_SECRET=${AZURE_CLIENT_SECRET}

Application code to retrieve secrets:

# Python example for retrieving Azure Key Vault secrets
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient

def get_secret(secret_name):
    credential = DefaultAzureCredential()
    vault_url = "https://myvault.vault.azure.net/"
    client = SecretClient(vault_url=vault_url, credential=credential)
    return client.get_secret(secret_name).value

# Get database credentials
password = get_secret("db-password")

Docker Build-Time Secrets

BuildKit Secret Mounting

Docker BuildKit enables mounting secrets during image building without embedding them in layers:

# Dockerfile with BuildKit secrets
FROM python:3.9-slim

WORKDIR /app
COPY requirements.txt .

# Mount npm token during build
RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) npm install

# Mount SSH key for private repository access
RUN --mount=type=ssh,id=github \
    git clone git@github.com:private/repo.git

COPY . .
CMD ["python", "app.py"]

Building with secrets:

# Build with secret from file
DOCKER_BUILDKIT=1 docker build \
  --secret id=npm_token,src=./npm_token.txt \
  -t my-app:latest .

# Build with SSH key
DOCKER_BUILDKIT=1 docker build \
  --ssh github=~/.ssh/id_rsa \
  -t my-app:latest .

This approach ensures:

  1. No secret leakage: Secrets aren't stored in image layers
  2. Build-time only: Secrets are only available during specific build steps
  3. Improved security: No need for insecure workarounds like ARG variables

Environment Variable Management

Environment File Approaches

While not ideal for production, environment files provide a simple approach for development:

# docker-compose.yml with environment file
version: '3.8'
services:
  webapp:
    image: my-webapp:latest
    env_file:
      - ./config/app.env

With an environment file like:

# app.env
DB_USER=admin
DB_PASSWORD=myStrongPassword123!
API_KEY=ab12cd34ef56gh78

To improve security:

  1. Gitignore env files: Prevent committing secrets to source control
    # .gitignore
*.env
secrets/
  2. Use env file templates: Commit templates without actual secrets
    # app.env.template
DB_USER=
DB_PASSWORD=
API_KEY=
  3. Implement validation: Ensure all required variables are set
    # check-env.sh
if [ -z "$DB_PASSWORD" ]; then
  echo "Error: DB_PASSWORD is not set"
  exit 1
fi

Multi-Environment Configuration

For multiple environments, implement structured configurations:

project/
├── docker-compose.yml
├── docker-compose.override.yml
├── docker-compose.prod.yml
└── environments/
    ├── development/
    │   └── .env
    └── production/
        └── .env

With environment-specific compose files:

# docker-compose.prod.yml
version: '3.8'
services:
  webapp:
    env_file:
      - ./environments/production/.env
    environment:
      - NODE_ENV=production
    deploy:
      secrets:
        - db_password
        - api_key

secrets:
  db_password:
    external: true
  api_key:
    external: true

Runtime Secrets Injection

Init Container Pattern

Use an initialization container to fetch and prepare secrets:

# docker-compose.yml with init container pattern
version: '3.8'
services:
  secrets-init:
    image: secrets-init:latest
    volumes:
      - secrets-volume:/secrets
    environment:
      - VAULT_ADDR=http://vault:8200
      - VAULT_TOKEN=${VAULT_TOKEN}
  
  webapp:
    image: my-webapp:latest
    depends_on:
      - secrets-init
    volumes:
      - secrets-volume:/run/secrets
    command: ["sh", "-c", "cat /run/secrets/config.json && exec node app.js"]

volumes:
  secrets-volume:
    driver: local
    driver_opts:
      type: tmpfs
      device: tmpfs

The initialization container fetches secrets and writes them to a shared in-memory volume.

Sidecar Pattern

The sidecar pattern uses a companion container to manage secrets:

# docker-compose.yml with sidecar pattern
version: '3.8'
services:
  webapp:
    image: my-webapp:latest
    depends_on:
      - secrets-sidecar
  
  secrets-sidecar:
    image: secrets-sidecar:latest
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - VAULT_ADDR=http://vault:8200
      - VAULT_TOKEN=${VAULT_TOKEN}
      - TARGET_CONTAINER=webapp

The sidecar continuously monitors and updates secrets as needed.

Environment Controllers

Environment controllers inject secrets directly into running containers:

# Using chamber to run a process with secrets
docker run -it --rm \
  -e AWS_REGION=us-west-2 \
  -e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
  -e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
  segmentio/chamber exec myapp-production -- my-application

Chamber retrieves secrets from AWS Parameter Store and injects them as environment variables.

Secret Rotation and Management

Automated Secret Rotation

Implement automated secret rotation to limit the impact of potential secret exposure:

# docker-compose.yml with secret rotation service
version: '3.8'
services:
  rotation-service:
    image: secret-rotator:latest
    environment:
      - ROTATION_INTERVAL=24h
      - VAULT_ADDR=http://vault:8200
      - VAULT_TOKEN=${VAULT_TOKEN}
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

Rotation mechanisms vary by secret type:

  1. Database credentials: Use dynamic credentials with limited lifetimes
  2. API keys: Create new keys before revoking old ones
  3. TLS certificates: Implement automated renewal with services like cert-manager

Secret Versioning

Implement versioning to manage secret transitions:

# Create a new version of a secret in Docker Swarm
echo "newStrongPassword456!" | docker secret create db_password_v2 -

# Update service to use new secret
docker service update \
  --secret-rm db_password \
  --secret-add source=db_password_v2,target=db_password \
  db

For external secret systems:

# Vault example of versioned secrets
docker exec -it vault vault kv put secret/myapp/db/v2 password="newStrongPassword456!"

# Transition application to new version
docker service update \
  --env-add SECRET_VERSION=v2 \
  webapp

Secret Auditing and Monitoring

Access Logging

Implement comprehensive logging for secret access:

# Vault audit configuration
docker exec -it vault vault audit enable file file_path=/vault/logs/audit.log

# View secret access logs
docker exec -it vault cat /vault/logs/audit.log | jq '.request.path'

Key aspects to monitor:

  1. Access patterns: Track which services access which secrets
  2. Failed attempts: Monitor unsuccessful access attempts
  3. Unusual activity: Alert on access from unexpected locations or times
  4. Secret creation/deletion: Track lifecycle changes to secrets

Security Scanning

Implement scanning to detect secrets accidentally committed to code or embedded in images:

# Scan Dockerfile and source code for secrets
docker run --rm -v $(pwd):/src zricethezav/gitleaks:latest detect --source="/src" --verbose

# Scan built images for secrets
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock goodwithtech/dockle:latest myapp:latest

Integration with CI/CD Pipelines

Secure CI/CD Integration

Integrate secrets management into CI/CD pipelines:

# GitHub Actions workflow with secrets
name: Build and Deploy

on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-west-2
      
      - name: Build with BuildKit
        uses: docker/build-push-action@v2
        with:
          context: .
          push: true
          tags: myregistry/myapp:latest
          secrets: |
            "npm_token=${{ secrets.NPM_TOKEN }}"

Secure Deployment Strategies

Implement secure deployment patterns:

  1. Just-in-time secrets: Provision secrets only when needed during deployment
  2. Credential bootstrapping: Use limited-scope credentials to access full credentials
  3. Immutable deployments: Never update secrets in running containers; deploy new ones instead
  4. Secret scoping: Limit secrets to specific deployment environments
# docker-compose.yml with environment-specific secrets
version: '3.8'
services:
  webapp:
    image: my-webapp:latest
    secrets:
      - source: db_password_${ENVIRONMENT}
        target: db_password

secrets:
  db_password_development:
    external: true
  db_password_staging:
    external: true
  db_password_production:
    external: true

Best Practices Summary

Security Principles

Follow these core principles for Docker secrets management:

  1. Least privilege access: Containers should only access the secrets they need
  2. Separation of concerns: Separate application code from secrets management
  3. Defense in depth: Implement multiple layers of protection for sensitive data
  4. No secrets in images: Never bake secrets into Docker images
  5. Immutable infrastructure: Rotate secrets by deploying new containers, not updating existing ones

Implementation Checklist

A comprehensive secrets management strategy should include:

  • Defined process for secret creation, distribution, and rotation
  • Centralized secrets storage with encryption at rest
  • Secure transport of secrets to containers
  • Audit logging of all secret access
  • Automated secret rotation mechanisms
  • Integration with CI/CD pipelines
  • Regular scanning for leaked secrets
  • Emergency revocation procedures

Conclusion

Effective secrets management is essential for securing Docker environments and preventing sensitive data exposure. By implementing the approaches outlined in this guide—whether using Docker's native secrets functionality, external secrets management tools, or cloud provider solutions—you can ensure that your containerized applications handle sensitive information securely throughout their lifecycle.

Remember that secrets management is not a one-time implementation but an ongoing process that requires regular review and updates to align with evolving security best practices and organizational requirements. The most effective strategies combine technical tools with well-defined processes and security-aware development practices.

Edit this page
Docker Service Discovery & DNS
Comprehensive guide to implementing service discovery and DNS solutions in Docker environments for efficient container networking
Kubernetes Architecture
Understanding Kubernetes architecture and core components