Autoscaling & Resource Management

Autoscaling in Kubernetes

Kubernetes provides several autoscaling mechanisms to adjust resources based on workload demands, ensuring optimal performance and cost efficiency. Autoscaling is essential for applications with variable workloads, as it helps maintain performance during traffic spikes while reducing costs during periods of low activity.

Kubernetes offers three complementary autoscaling mechanisms that work at different levels:

• Horizontal Pod Autoscaler (HPA): Scales the number of pod replicas
• Vertical Pod Autoscaler (VPA): Adjusts CPU and memory resources for containers
• Cluster Autoscaler: Changes the number of nodes in the cluster

Each autoscaler operates independently but can be used together to create a comprehensive scaling strategy.

Horizontal Pod Autoscaler (HPA)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2        # Minimum number of pods to maintain
  maxReplicas: 10       # Maximum number of pods to scale to
  metrics:
  - type: Resource      # Using built-in resource metrics
    resource:
      name: cpu         # CPU utilization as the scaling metric
      target:
        type: Utilization      # Based on percentage utilization
        averageUtilization: 80 # Target 80% CPU utilization across pods

metrics:
- type: Resource
  resource:
    name: cpu
    target:
      type: Utilization
      averageUtilization: 80
- type: Resource
  resource:
    name: memory
    target:
      type: Utilization
      averageUtilization: 80

Vertical Pod Autoscaler (VPA)

The Vertical Pod Autoscaler adjusts the CPU and memory resources allocated to containers within pods, rather than changing the number of pods. This is ideal for applications that cannot be horizontally scaled or that benefit from having properly sized resources.

VPA works by:

1. Analyzing historical resource usage of containers
2. Recommending optimal CPU and memory settings
3. Applying these settings automatically (in Auto mode) or providing them for manual implementation

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-app-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  updatePolicy:
    updateMode: Auto  # Options: Auto, Recreate, Off, Initial
  resourcePolicy:
    containerPolicies:
    - containerName: '*'  # Apply to all containers in the pod
      minAllowed:         # Minimum resources to allocate
        cpu: 100m
        memory: 50Mi
      maxAllowed:         # Maximum resources to allocate
        cpu: 1
        memory: 500Mi
      controlledResources: ["cpu", "memory"]  # Resources to manage

VPA update modes explained:

• Auto: Automatically updates pod resources during its lifetime (requires pod restart)
• Recreate: Similar to Auto, but all pods are restarted when recommendations change
• Off: Only provides recommendations without applying them (visible in VPA status)
• Initial: Only applies recommendations at pod creation time

Cluster Autoscaler

Cluster Autoscaler intelligently manages node scaling by:

• Regularly checking for pods that can't be scheduled
• Simulating scheduling to determine if adding a node would help
• Respecting pod disruption budgets when removing nodes
• Considering node groups and their properties
• Safely draining nodes before termination
• Respecting scale-down delays to prevent thrashing

# Example for GKE
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
      annotations:
        # Required to get proper metrics and avoid self-eviction
        cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
    spec:
      serviceAccountName: cluster-autoscaler  # Account with node management permissions
      containers:
      - name: cluster-autoscaler
        image: k8s.gcr.io/cluster-autoscaler:v1.18.1
        command:
        - ./cluster-autoscaler
        - --cloud-provider=gce            # Cloud provider-specific
        - --nodes=2:10:default-pool       # Min:Max:NodePoolName
        - --scale-down-delay-after-add=10m  # Wait time after scaling up
        - --scale-down-unneeded-time=10m    # Time before considering a node for removal
        - --skip-nodes-with-local-storage=false  # Consider nodes with local storage
        - --balance-similar-node-groups    # Balance node groups with similar instances
        resources:
          requests:
            cpu: 100m
            memory: 300Mi

For different cloud providers, the configuration varies:

AWS:

--node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/<cluster-name>

Azure:

--node-resource-group=<resource-group>
--azure-vm-type=AKS

Resource Management

Resource management is a critical aspect of Kubernetes that affects scheduling, performance, and reliability. Properly configuring resources ensures that applications get what they need while maintaining cluster efficiency.

Understanding the difference between requests and limits is crucial:

• Requests: What the container is guaranteed to get
• Limits: The maximum it can use before throttling/termination

When configuring resources, consider these principles:

1. Set requests based on actual application needs
2. Configure limits to prevent runaway containers
3. Monitor actual usage to refine values over time
4. Consider the application's behavior under resource constraints

QoS Classes

QoS classes affect several aspects of container runtime:

Pod Eviction Order:

• Under resource pressure, BestEffort pods are evicted first
• Then Burstable pods with the highest usage above requests
• Guaranteed pods are evicted only as a last resort

Resource Reclamation:

• Guaranteed pods have the highest priority for resource retention
• Burstable pods may face CPU throttling when the node is under pressure
• BestEffort pods receive resources only when available

Scheduling Priority:

• While not directly affecting scheduling order, QoS impacts pod placement
• Guaranteed pods have more predictable performance characteristics

# Guaranteed QoS Example
apiVersion: v1
kind: Pod
metadata:
  name: guaranteed-pod
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        memory: "200Mi"  # 200 mebibytes of memory
        cpu: "500m"      # 500 millicores = 0.5 CPU cores
      limits:
        memory: "200Mi"  # Must match request for Guaranteed QoS
        cpu: "500m"      # Must match request for Guaranteed QoS

Burstable QoS Example:

apiVersion: v1
kind: Pod
metadata:
  name: burstable-pod
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        memory: "100Mi"
        cpu: "200m"
      limits:
        memory: "200Mi"  # Higher than request
        cpu: "800m"      # Higher than request

BestEffort QoS Example:

apiVersion: v1
kind: Pod
metadata:
  name: besteffort-pod
spec:
  containers:
  - name: app
    image: nginx
    # No resources specified

Resource Quotas

Resource Quotas allow cluster administrators to restrict resource consumption per namespace. This is essential for multi-tenant clusters where fair resource distribution is important.

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: team-a
spec:
  hard:
    # Compute resource constraints
    requests.cpu: "10"      # Total CPU requests across all pods
    requests.memory: 20Gi   # Total memory requests across all pods
    limits.cpu: "20"        # Total CPU limits across all pods
    limits.memory: 40Gi     # Total memory limits across all pods
    
    # Object count constraints
    pods: "20"              # Maximum number of pods
    services: "10"          # Maximum number of services
    configmaps: "20"        # Maximum number of ConfigMaps
    persistentvolumeclaims: "5"  # Maximum number of PVCs
    
    # Storage constraints
    requests.storage: "500Gi"    # Total storage requests
    gold.storageclass.storage.k8s.io/requests.storage: "300Gi"  # Storage class specific

When a ResourceQuota is active in a namespace:

1. Users must specify resource requests and limits for all containers
2. The quota system tracks usage and prevents creation of resources that exceed quota
3. Requests for resources that would exceed quota are rejected with a 403 FORBIDDEN error
4. Quotas can be adjusted dynamically as business needs change

Limit Ranges

LimitRanges provide constraints on resource allocations per object in a namespace, allowing administrators to enforce minimum and maximum resource usage per pod or container. They also enable setting default values when not explicitly specified by developers.

apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: team-a
spec:
  limits:
  - default:          # Default limits if not specified
      memory: 512Mi
      cpu: 500m
    defaultRequest:   # Default requests if not specified
      memory: 256Mi
      cpu: 250m
    type: Container   # These apply to containers

    # Optional fields for comprehensive control:
    # max:            # Maximum limits allowed
    #   memory: 2Gi
    #   cpu: 2
    # min:            # Minimum requests required
    #   memory: 50Mi
    #   cpu: 50m
    # maxLimitRequestRatio:  # Maximum limit to request ratio
    #   memory: 4
    #   cpu: 4

apiVersion: v1
kind: LimitRange
metadata:
  name: constraint-limits
spec:
  limits:
  - min:
      cpu: 100m
      memory: 50Mi
    max:
      cpu: 2
      memory: 4Gi
    type: Container

Resource Metrics

Monitoring resource usage is essential for effective resource management and troubleshooting. Kubernetes provides built-in commands to view current resource consumption.

# View resource usage for pods
kubectl top pod -n default
# Sample output:
# NAME                       CPU(cores)   MEMORY(bytes)
# nginx-6799fc88d8-dnz7t     1m           9Mi
# redis-79486b66b7-lrm48     3m           11Mi

# View resource usage for specific pods
kubectl top pod -l app=nginx -n default

# View resource usage for nodes
kubectl top node
# Sample output:
# NAME           CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
# node-1         368m         18%    1726Mi          45%
# node-2         132m         6%     1056Mi          27%

# Get more detailed resource information for a pod
kubectl describe pod <pod-name> | grep -A 15 Resources:

These metrics are provided by the Metrics Server, which must be installed in the cluster. For more comprehensive monitoring, consider solutions like:

• Prometheus + Grafana for detailed metrics collection and visualization
• Kubernetes Dashboard for a web UI showing cluster resources
• Vertical Pod Autoscaler in recommendation mode for resource optimization insights

Custom Metrics Autoscaling

Scaling based on CPU and memory might not always reflect your application's actual load. Custom metrics autoscaling allows you to scale based on application-specific metrics like queue depth, request latency, or any other business metric.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: queue-processor
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: queue-processor
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: External          # External metric from monitoring system
    external:
      metric:
        name: queue_messages_ready   # Metric name in monitoring system
        selector:
          matchLabels:
            queue: "worker_tasks"    # Additional labels to filter metric
      target:
        type: AverageValue           # How to interpret the metric
        averageValue: 30             # Target 30 messages per pod
  - type: Pods              # Example of pod metric (from Prometheus)
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: 1000
  - type: Object            # Example of object metric (e.g., ingress)
    object:
      metric:
        name: requests_per_second
      describedObject:
        apiVersion: networking.k8s.io/v1
        kind: Ingress
        name: main-ingress
      target:
        type: Value
        value: 10000

Best Practices

Autoscaling Strategies

Implementing effective autoscaling goes beyond simply enabling the autoscalers. Strategic approaches can significantly improve application performance and resource efficiency.

Advanced scaling implementations:

1. Multi-dimensional scaling: Using combinations of metrics (CPU, memory, custom)
2. Schedule-based scaling: Setting different min/max replicas based on time of day
3. Cascading autoscaling: Coordinating scaling across multiple dependent components
4. Controlled rollouts: Gradually increasing capacity to validate system behavior
5. Circuit breaking: Automatically degrading non-critical features during peak loads

Advanced Configuration

# HPA with behavior configuration
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 30
      - type: Pods
        value: 4
        periodSeconds: 30
      selectPolicy: Max

Troubleshooting Autoscaling

Resource Optimization

Implementation Checklist

Remember that autoscaling is an iterative process. Begin with conservative settings, monitor behavior in production, and adjust as you learn more about your application's actual needs and performance characteristics.