Registry & Distribution

Docker Hub

• Public registry service operated by Docker, Inc.
• Free tier for public repositories with rate limits (100/200 pulls per 6 hours for anonymous/authenticated users)
• Subscription plans for private repositories and higher rate limits
• Official images maintained by Docker and verified publishers for trusted content
• Automated builds triggered by source code changes in connected repositories
• Webhooks for integration with CI/CD pipelines and other systems
• Vulnerability scanning for detecting security issues in images
• Team management for collaborative development

Private Registry

• Self-hosted solution within your own infrastructure
• Complete control over data location, retention, and security policies
• Network isolation capabilities for air-gapped or high-security environments
• Integration with internal systems like LDAP/Active Directory
• Customizable storage backends (filesystem, S3, Azure Blob, etc.)
• Fine-grained access control policies
• No external rate limits or subscription fees
• Audit logging for compliance and security tracking
• Ability to implement custom validation and security policies

Cloud Provider Registries

• AWS Elastic Container Registry (ECR)
  • Native integration with AWS IAM for access control
  • Lifecycle policies for automatic cleanup
  • Cross-region replication for improved availability
  • Image scanning with Amazon ECR and integration with Amazon Inspector
• Google Container Registry (GCR) / Artifact Registry
  • Integration with Google Cloud IAM
  • Built-in vulnerability scanning
  • Global availability with regional storage
  • Integration with Google Cloud Build
• Azure Container Registry (ACR)
  • Geo-replication across Azure regions
  • Integration with Azure Active Directory
  • WebHooks for build and deployment automation
  • Premium tier offers enhanced throughput and scalability
• Digital Ocean Container Registry
  • Integration with Digital Ocean Kubernetes
  • Regional storage options
  • Simple pricing model
• GitHub Container Registry (GHCR)
  • Tight integration with GitHub Actions
  • Fine-grained permissions aligned with GitHub's model
  • Anonymous access for public images
  • Support for OCI artifacts beyond container images

Basic Authentication

Basic authentication provides simple username/password protection for your registry:

# Create auth directory
mkdir -p auth

# Create user with BCrypt hashing (more secure than the default)
docker run --rm --entrypoint htpasswd \
  httpd:2 -Bbn username password > auth/htpasswd

# Add additional users
docker run --rm --entrypoint htpasswd \
  httpd:2 -Bbn another-user another-password >> auth/htpasswd

# Run registry with authentication
docker run -d \
  -p 5000:5000 \
  --name registry \
  -v "$(pwd)"/auth:/auth \
  -v "$(pwd)"/registry-data:/var/lib/registry \
  -e "REGISTRY_AUTH=htpasswd" \
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
  -e "REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd" \
  registry:2

# Client authentication when pushing/pulling
docker login localhost:5000

Authentication can also be integrated with LDAP, OAuth, or other external providers for enterprise environments.

TLS Configuration

TLS encryption is critical for securing registry communications:

# Generate self-signed certificates (for testing only)
mkdir -p certs
openssl req -newkey rsa:4096 -nodes -sha256 \
  -keyout certs/domain.key -x509 -days 365 \
  -out certs/domain.crt \
  -subj "/CN=localhost"

# For production, use properly signed certificates from a trusted CA
# If using Let's Encrypt:
# certbot certonly --standalone -d registry.example.com

# Run registry with TLS
docker run -d \
  -p 443:5000 \
  --name registry \
  -v "$(pwd)"/certs:/certs \
  -v "$(pwd)"/registry-data:/var/lib/registry \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
  registry:2

# Copy cert to Docker trust store (for self-signed certs)
mkdir -p /etc/docker/certs.d/localhost:443
cp certs/domain.crt /etc/docker/certs.d/localhost:443/ca.crt

Storage Configuration

Configure backend storage options for production-grade registry deployment:

# Create a configuration file
cat > config.yml << EOF
version: 0.1
log:
  level: info
storage:
  filesystem:
    rootdirectory: /var/lib/registry
  delete:
    enabled: true
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
auth:
  htpasswd:
    realm: Registry Realm
    path: /auth/htpasswd
EOF

# For S3 storage backend
cat > config-s3.yml << EOF
version: 0.1
storage:
  s3:
    accesskey: AKIAIOSFODNN7EXAMPLE
    secretkey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    region: us-west-1
    bucket: docker-registry
    encrypt: true
    secure: true
EOF

# Configure registry with storage backend
docker run -d \
  -p 5000:5000 \
  --name registry \
  -v "$(pwd)"/config.yml:/etc/docker/registry/config.yml \
  -v "$(pwd)"/auth:/auth \
  registry:2

Access Control

For more advanced access control:

# Example of role-based access control configuration
cat > config-rbac.yml << EOF
version: 0.1
auth:
  token:
    realm: https://auth.example.com/token
    service: registry.example.com
    issuer: auth-service
    rootcertbundle: /etc/registry/auth-service.crt
EOF

# Run with RBAC configuration
docker run -d \
  -p 5000:5000 \
  --name registry \
  -v "$(pwd)"/config-rbac.yml:/etc/docker/registry/config.yml \
  -v "$(pwd)"/auth-service.crt:/etc/registry/auth-service.crt \
  registry:2

Cloud Provider Registries

• Amazon Elastic Container Registry (ECR)
  • Deeply integrated with AWS services (ECS, EKS, Lambda)
  • Private repositories with IAM authentication
  • Lifecycle policies for automated image cleanup
  • Vulnerability scanning with Amazon Inspector
  • Cross-region and cross-account replication
  • Pay-as-you-go pricing model
• Google Container Registry (GCR) / Artifact Registry
  • Native integration with Google Cloud Build and GKE
  • Automatic vulnerability scanning
  • IAM access controls and audit logging
  • Regional storage with global access
  • Support for Docker, OCI, and language-specific packages
  • Storage pricing + data transfer costs
• Azure Container Registry (ACR)
  • Integrated with Azure DevOps and AKS
  • Premium tier with geo-replication
  • Automated builds with Tasks
  • Content trust for image signing
  • Tiered pricing (Basic, Standard, Premium)
  • Webhook support for build and deployment automation
• DigitalOcean Container Registry
  • Simple integration with DO Kubernetes
  • Straightforward pricing by storage tiers
  • Global availability
  • Integrated vulnerability scanning

Self-Hosted Solutions

• Harbor
  • Open source, enterprise-focused registry
  • Role-based access control
  • Policy-based image replication
  • Vulnerability scanning integration
  • Image signing and verification
  • Support for multiple registries and Helm charts
  • WebHooks for event notification
  • Audit logging for compliance
• Nexus Repository
  • Multi-format artifact management (not just Docker)
  • Support for npm, Maven, NuGet, PyPI, etc.
  • Role-based access control
  • Component lifecycle management
  • Repository health check
  • Available in free OSS and commercial editions
  • Proxy and cache remote repositories
• JFrog Artifactory
  • Universal artifact management platform
  • High availability configuration
  • Advanced security features
  • Metadata-based search
  • Build integration
  • Extensive REST API
  • Commercial offering with enterprise support
  • Replication and federation capabilities
• GitLab Container Registry
  • Integrated with GitLab CI/CD
  • Built into GitLab installations
  • Project-based permissions
  • Vulnerability scanning
  • Image clean-up policies
  • No additional configuration needed with GitLab

Tagging Strategy

• Use semantic versioning (SemVer)
  • Format: MAJOR.MINOR.PATCH (e.g., 1.2.3)
  • MAJOR: Breaking changes
  • MINOR: New features, backward compatible
  • PATCH: Bug fixes, backward compatible
• Include build information
  • Add build numbers or timestamps
  • Format examples:
    • 1.2.3-build.456
    • 1.2.3-20230415.1
    • 1.2.3-alpha.1, 1.2.3-beta.2, 1.2.3-rc.1
• Never use "latest" in production
  • "latest" is mutable and unpredictable
  • Makes rollbacks difficult or impossible
  • Complicates auditing and versioning
  • Obscures which version is actually running
• Tag with git commit hashes
  • Provides direct traceability to source code
  • Format example: 1.2.3-a7ff23e
  • Helpful for debugging specific versions
  • Can automate in CI/CD pipelines
• Consider using digest references
  • Immutable and tamper-evident
  • Format: image@sha256:digest
  • Guarantees exact image content
  • Can be used with vulnerability scanners
  • Best for security-critical deployments
• Implement tag lifecycle policies
  • Automate cleanup of old tags
  • Preserve important historical versions
  • Document retention policies

Security

• Regular vulnerability scanning
  • Integrate scanners like Trivy, Clair, or Anchore
  • Block deployment of vulnerable images
  • Configure scheduled rescans of existing images
  • Track CVEs and patch affected images
• Image signing
  • Implement Docker Content Trust
  • Use Notary for advanced signing workflows
  • Enforce signature verification on pull
  • Store root keys securely offline
  • Rotate signing keys periodically
• Access control
  • Implement principle of least privilege
  • Use namespaces for project isolation
  • Configure role-based access control
  • Audit user access regularly
  • Implement approval workflows for sensitive repositories
• Audit logging
  • Log all registry operations
  • Capture who, what, when information
  • Store logs securely and immutably
  • Set up alerts for suspicious activities
  • Retain logs for compliance requirements
  • Forward logs to SIEM systems
• Regular garbage collection
  • Schedule routine garbage collection
  • Monitor storage utilization
  • Configure retention policies
  • Implement image lifecycle management
  • Remove untagged and outdated images
• Network security
  • Use TLS 1.2+ for all registry traffic
  • Implement proper certificate management
  • Consider network segmentation
  • Use VPNs or private endpoints for access

Efficiency

• Layer caching
  • Optimize Dockerfiles for cache utilization
  • Use BuildKit's improved caching
  • Implement shared build caches in CI/CD
  • Configure appropriate cache lifetimes
  • Consider remote caching for distributed teams
• Optimized storage backends
  • Choose appropriate backend for scale (S3, Azure Blob, etc.)
  • Configure compression settings
  • Implement data deduplication where available
  • Monitor storage performance metrics
  • Implement lifecycle policies for automated management
• Pull-through caching
  • Set up proxy registries for frequently used images
  • Cache external images locally to reduce bandwidth
  • Configure appropriate cache TTL values
  • Schedule cache warming for critical images
  • Implement health checks for upstream registries
• Load balancing
  • Distribute registry load across multiple instances
  • Implement round-robin or other load balancing strategies
  • Configure connection draining for maintenance
  • Monitor per-instance performance metrics
  • Set up high availability clusters
• Geographic distribution
  • Deploy registries close to users/clusters
  • Implement registry replication across regions
  • Use content delivery networks where appropriate
  • Configure smart routing based on client location
  • Synchronize metadata between distributed instances

Understanding Layers

• Each instruction in Dockerfile creates a layer
  • Every RUN, COPY, ADD command creates a new layer
  • Other instructions create metadata-only changes
  • Layers represent filesystem differences
  • Maximum of 127 layers per image (practical limit)
  • Older images may use different layering technology
• Layers are cached and reused
  • Unchanged layers use cache during builds
  • Cache invalidation occurs at first change
  • All subsequent layers must be rebuilt
  • Order instructions from least to most frequently changed
  • Use .dockerignore to prevent cache invalidation
• Efficient distribution relies on layer sharing
  • Common layers are stored once on disk
  • Images with same base share foundation layers
  • Registry transfers only missing layers
  • Improves pull performance and reduces storage
  • Enables efficient large-scale deployments
• Base images form common foundation layers
  • Choose appropriate base images for sharing
  • Consider using organization-specific base images
  • Standardizing on base images increases sharing
  • Update base images regularly for security patches
  • Track base image usage across organization
• Only changed layers are transferred during pulls
  • Docker client checks which layers it already has
  • Registry serves only missing layers
  • Layers are verified by content-addressable hashes
  • Network transfer is minimized
  • Consider squashing for production if sharing isn't beneficial

Layer Management

# View image layers and sizes
docker history my-image:latest

# View detailed layer information including SHA256 digests
docker history --no-trunc my-image:latest

# See layer details including mount points and driver data
docker inspect my-image:latest

# Extract a specific layer (advanced)
LAYER_ID=$(docker inspect my-image:latest | jq -r '.[0].RootFS.Layers[2]')
mkdir -p /tmp/extracted-layer
docker save my-image:latest | tar -xf - -C /tmp
tar -xf /tmp/$LAYER_ID/layer.tar -C /tmp/extracted-layer

# Analyze image layers with dive tool
docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock wagoodman/dive:latest my-image:latest

Layer Optimization

• Group related commands to reduce layer count
• Use multi-stage builds to eliminate build-only layers
• Remove temporary files in the same layer they're created
• Consider squashing layers for production images
• Use appropriate base images to maximize layer sharing

Content Trust

• Sign and verify images
  • Implement Notary for signature management
  • Configure separate keys for different environments
  • Document key management procedures
  • Implement hardware security modules for key storage
  • Set up automated signing in CI/CD pipelines
• Ensure image authenticity
  • Verify publisher identity
  • Confirm content hasn't been modified
  • Check signature freshness and expiry
  • Implement chain of trust validation
  • Integrate with secure software supply chain
• Prevent tampering
  • Store signatures separately from images
  • Use threshold signing for critical images
  • Implement key rotation procedures
  • Monitor for unauthorized signature attempts
  • Log all verification activities
• Enable in daemon.json
  {
    "content-trust": {
      "trust-pinning": {
        "official-library-images": true,
        "root-keys": {
          "docker.io/myorg/*": ["key1", "key2"]
        }
      }
    }
  }
  

• Verify before deployment
  • Add verification steps in deployment pipelines
  • Block deployment of unsigned images
  • Implement policy controllers in Kubernetes
  • Document verification requirements
  • Add signature metadata to deployment artifacts

Registry API

• RESTful API
  • Well-documented endpoints
  • Standard HTTP status codes
  • JSON response format
  • Authentication with Bearer tokens
  • Rate limiting and throttling controls
• Image manipulation
  • Push and pull operations
  • Layer uploads and downloads
  • Cross-repository blob mounting
  • Image manifest management
  • Content addressable blob storage
• Repository management
  • List and search repositories
  • Tag management operations
  • Access control configuration
  • Repository metadata handling
  • Namespace organization
• Catalog operations
  • Paginated repository listing
  • Search and filtering capabilities
  • Metadata aggregation
  • Tag enumeration
  • Cross-repository operations
• Webhook integration
  • Event-based notifications
  • Customizable event filtering
  • Delivery retry mechanisms
  • Authentication for webhook endpoints
  • Webhook delivery logging and auditing

Garbage Collection

# Run basic garbage collection
docker exec -it registry \
  registry garbage-collect /etc/docker/registry/config.yml

# Detailed garbage collection with verbose output
docker exec -it registry \
  registry garbage-collect -v /etc/docker/registry/config.yml

# Run garbage collection with dry run to preview deletions
docker exec -it registry \
  registry garbage-collect --dry-run /etc/docker/registry/config.yml 2>&1 | tee gc-preview.log

# Analyze how much space would be freed
docker exec -it registry \
  registry garbage-collect --dry-run /etc/docker/registry/config.yml 2>&1 | \
  grep "blob eligible for deletion" | \
  awk '{sum+=$7} END {print "Total bytes to be freed: " sum}'

# Run garbage collection and restart registry to apply changes
docker exec -it registry \
  registry garbage-collect /etc/docker/registry/config.yml
docker restart registry

Advanced Storage Management

• Implement storage quotas per repository/namespace
• Configure automated pruning policies
• Set up storage analytics and monitoring
• Implement data compression for storage efficiency
• Configure cross-region replication for disaster recovery