Kubernetes Service Account Token Volume Projection

Advanced authentication and authorization with Service Account Token Volume Projection in Kubernetes

Introduction to Service Account Token Volume Projection

Service Account Token Volume Projection is an advanced security feature in Kubernetes that enhances the way pods authenticate to the Kubernetes API and other services. This mechanism provides fine-grained control over token properties, improves security posture, and enables sophisticated authentication workflows for applications running in Kubernetes clusters.

Introduced as an alpha feature in Kubernetes 1.12 and graduating to stable in 1.20, Service Account Token Volume Projection replaces the legacy method of mounting service account tokens. It addresses several security concerns with the traditional approach and aligns with modern security best practices for containerized applications.

Service Account Authentication Evolution

Understanding the evolution of service account authentication in Kubernetes provides context for the importance of token volume projection:

Legacy Token Mounting

Traditional approach before Kubernetes 1.12
Long-lived tokens with no expiration
Automatically mounted at /var/run/secrets/kubernetes.io/serviceaccount/token
Same token for all containers in a pod
No audience binding or scope limitations
Potential security risk if token is compromised

Example of legacy token mounting:

apiVersion: v1
kind: Pod
metadata:
  name: legacy-token-pod
spec:
  serviceAccountName: my-service-account
  # Token automatically mounted unless explicitly disabled
  # automountServiceAccountToken: false
  containers:
  - name: app
    image: app:1.0
    # Token available at /var/run/secrets/kubernetes.io/serviceaccount/token

TokenRequest API

Introduced in Kubernetes 1.10
Enables programmatic token requests with specific properties
Supports bound audiences, time-bound validity
Foundation for projected service account tokens
Example token request:
kubectl create token my-service-account \ --audience="https://kubernetes.default.svc" \ --duration=1h

Projected Service Account Tokens

Introduced in Kubernetes 1.12 (alpha)
Stable in Kubernetes 1.20
Short-lived tokens with automatic rotation
Audience-bound to prevent token misuse
Independent token per container possible
Support for additional properties like expiration path

Example of projected token configuration:

apiVersion: v1
kind: Pod
metadata:
  name: projected-token-pod
spec:
  serviceAccountName: my-service-account
  containers:
  - name: app
    image: app:1.0
  volumes:
  - name: token-volume
    projected:
      sources:
      - serviceAccountToken:
          path: token
          expirationSeconds: 3600
          audience: "https://kubernetes.default.svc"

BoundServiceAccountTokenVolume Feature

Enabled by default in Kubernetes 1.21+
Makes projected tokens the default mechanism
Legacy tokens still available but deprecated
Automatic projection with secure defaults
Migration path for existing workloads

How Service Account Token Volume Projection Works

The projected service account token system involves several components working together:

TokenRequest API: The kubelet uses this API to request tokens with specific properties
Token Controller: Manages token creation and signing using the cluster's signing key
Volume Projection: Mounts the token into the pod's filesystem
Kubelet: Automatically refreshes tokens before expiration

The process flow is as follows:

sequenceDiagram
    participant Pod
    participant Kubelet
    participant API as Kubernetes API Server
    participant KAS as Kube-ApiServer
    
    Kubelet->>API: TokenRequest for Pod
    API->>KAS: Generate and sign token
    KAS-->>API: Signed token with expiry
    API-->>Kubelet: Return token
    Kubelet->>Pod: Mount token to specified path
    
    Note over Kubelet,Pod: Token approaches expiration
    
    Kubelet->>API: Request token renewal
    API->>KAS: Generate new token
    KAS-->>API: New signed token
    API-->>Kubelet: Return new token
    Kubelet->>Pod: Update token file

Basic Implementation

To implement service account token volume projection in your pods, you need to configure the projected volume and mount it in your containers:

apiVersion: v1
kind: Pod
metadata:
  name: token-projection-example
spec:
  serviceAccountName: my-service-account
  containers:
  - name: main-app
    image: myapp:1.0
    volumeMounts:
    - name: service-token
      mountPath: /var/run/secrets/tokens
      readOnly: true
  volumes:
  - name: service-token
    projected:
      sources:
      - serviceAccountToken:
          audience: "https://kubernetes.default.svc"
          expirationSeconds: 3600
          path: token

This configuration:

Creates a projected volume named service-token
Requests a token bound to the audience https://kubernetes.default.svc
Sets a 1-hour expiration time
Mounts the token at the path /var/run/secrets/tokens/token

Advanced Configuration Options

Service account token projection offers several advanced configuration options:

Multiple Audiences

volumes:
- name: service-tokens
  projected:
    sources:
    - serviceAccountToken:
        audience: "https://kubernetes.default.svc"
        expirationSeconds: 3600
        path: k8s-token
    - serviceAccountToken:
        audience: "https://vault.example.com"
        expirationSeconds: 7200
        path: vault-token

This configuration provides two different tokens bound to different audiences.

Custom Expiration Times

volumes:
- name: service-tokens
  projected:
    sources:
    - serviceAccountToken:
        audience: "https://app.example.com"
        expirationSeconds: 86400  # 24 hours
        path: long-lived-token
    - serviceAccountToken:
        audience: "https://api.example.com"
        expirationSeconds: 300    # 5 minutes
        path: short-lived-token

Different expiration times can be set based on security requirements.

Combining with Other Projected Sources

volumes:
- name: all-tokens
  projected:
    sources:
    - serviceAccountToken:
        audience: "https://kubernetes.default.svc"
        expirationSeconds: 3600
        path: k8s-token
    - configMap:
        name: trusted-ca
        items:
          - key: ca.crt
            path: ca.crt
    - secret:
        name: api-credentials
        items:
          - key: api-key
            path: api-key

Projected volumes can combine service account tokens with other sources like ConfigMaps and Secrets.

Multiple Containers with Different Tokens

spec:
  containers:
  - name: frontend
    image: frontend:1.0
    volumeMounts:
    - name: frontend-token
      mountPath: /var/run/secrets/tokens
  - name: backend
    image: backend:1.0
    volumeMounts:
    - name: backend-token
      mountPath: /var/run/secrets/tokens
  volumes:
  - name: frontend-token
    projected:
      sources:
      - serviceAccountToken:
          audience: "https://frontend.example.com"
          expirationSeconds: 3600
          path: token
  - name: backend-token
    projected:
      sources:
      - serviceAccountToken:
          audience: "https://backend.example.com"
          expirationSeconds: 3600
          path: token

Each container can have its own token with specific properties.

Security Benefits

Service account token volume projection provides significant security benefits:

Time-bound Tokens

Tokens automatically expire after the specified period
Limits the exposure window if tokens are compromised
Provides defense-in-depth for pod security
Example configuration with short expiration:
serviceAccountToken: expirationSeconds: 900 # 15 minutes

Audience Binding

Tokens are bound to specific audiences (recipients)
Prevents token reuse for unintended services
Implements the OAuth 2.0 audience restriction pattern
Example with explicit audience:
serviceAccountToken: audience: "https://payment-processor.example.com"

Automatic Rotation

Kubelet refreshes tokens before expiration
No application changes needed to use fresh tokens
Maintains token validity without intervention
Typically refreshed when 80% of lifetime has elapsed

Principle of Least Privilege

Different tokens with different permissions per container
Fine-grained access control with audience restrictions
Minimizes the impact of token compromise

Example of least privilege implementation:

containers:
- name: metrics
  image: metrics:1.0
  volumeMounts:
  - name: metrics-token
    mountPath: /var/run/secrets/tokens
volumes:
- name: metrics-token
  projected:
    sources:
    - serviceAccountToken:
        audience: "https://monitoring.example.com"
        expirationSeconds: 3600
        path: token

Integration with External Systems

One of the most powerful aspects of service account token projection is integration with external authentication systems:

HashiCorp Vault Integration
volumes: - name: vault-token projected: sources: - serviceAccountToken: audience: "vault.example.com" expirationSeconds: 3600 path: token
This token can be used with Vault's Kubernetes authentication method.

OAuth2 Proxy Authentication

containers:
- name: oauth-proxy
  image: oauth2-proxy:latest
  args:
  - --provider=oidc
  - --oidc-issuer-url=https://kubernetes.default.svc
  - --client-id=kubernetes
  - --token-path=/var/run/secrets/tokens/token
  volumeMounts:
  - name: k8s-token
    mountPath: /var/run/secrets/tokens
volumes:
- name: k8s-token
  projected:
    sources:
    - serviceAccountToken:
        audience: "kubernetes"
        expirationSeconds: 3600
        path: token

Custom API Authentication

containers:
- name: api-client
  image: api-client:latest
  env:
  - name: TOKEN_PATH
    value: /var/run/secrets/tokens/api-token
  volumeMounts:
  - name: api-token
    mountPath: /var/run/secrets/tokens
volumes:
- name: api-token
  projected:
    sources:
    - serviceAccountToken:
        audience: "https://api.example.com"
        expirationSeconds: 3600
        path: api-token

Service Mesh Authentication

containers:
- name: istio-proxy
  image: istio/proxyv2:latest
  volumeMounts:
  - name: istio-token
    mountPath: /var/run/secrets/tokens
volumes:
- name: istio-token
  projected:
    sources:
    - serviceAccountToken:
        audience: "istio-ca"
        expirationSeconds: 43200  # 12 hours
        path: istio-token

Working with Token Volumes in Applications

Applications need to be aware of how to use the projected tokens:

Reading Token Files
// Go example tokenBytes, err := os.ReadFile("/var/run/secrets/tokens/token") if err != nil { log.Fatalf("Failed to read token: %v", err) } token := string(tokenBytes)
Handling Token Expiration The kubelet refreshes tokens automatically, so applications can simply read the current token file when needed:
# Python example def get_current_token(): with open("/var/run/secrets/tokens/token", "r") as f: return f.read().strip() # Use this function whenever you need the token # No need to track expiration or refresh manually

Making Authenticated API Calls

// JavaScript example
const fs = require('fs');

async function callAPI() {
  const token = fs.readFileSync('/var/run/secrets/tokens/token', 'utf8');
  const response = await fetch('https://api.example.com/data', {
    headers: {
      'Authorization': `Bearer ${token}`
    }
  });
  return response.json();
}

Adding Token to gRPC Calls

// Go gRPC example
func getGRPCClient() (pb.ServiceClient, error) {
    tokenBytes, err := os.ReadFile("/var/run/secrets/tokens/token")
    if err != nil {
        return nil, fmt.Errorf("failed to read token: %v", err)
    }
    
    creds := oauth.NewOauthAccess(&oauth2.Token{
        AccessToken: string(tokenBytes),
        TokenType:   "Bearer",
    })
    
    conn, err := grpc.Dial(
        "service.example.com:443",
        grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{})),
        grpc.WithPerRPCCredentials(creds),
    )
    if err != nil {
        return nil, fmt.Errorf("failed to dial: %v", err)
    }
    
    return pb.NewServiceClient(conn), nil
}

Advanced Usage Patterns

Service account token projection enables several advanced authentication patterns:

Multi-service Authentication

Single pod authenticating to multiple services
Different tokens with different audiences

Example configuration:

volumes:
- name: tokens
  projected:
    sources:
    - serviceAccountToken:
        audience: "https://database.example.com"
        expirationSeconds: 3600
        path: db-token
    - serviceAccountToken:
        audience: "https://cache.example.com"
        expirationSeconds: 1800
        path: cache-token
    - serviceAccountToken:
        audience: "https://logging.example.com"
        expirationSeconds: 900
        path: logging-token

Sidecar Auth Proxy

Dedicated sidecar container for authentication
Centralizes auth logic for the main application
Provides authentication services to the main container

Example implementation:

containers:
- name: main-app
  image: main-app:1.0
  env:
  - name: AUTH_PROXY_URL
    value: "http://localhost:8181/token"
- name: auth-proxy
  image: auth-proxy:1.0
  ports:
  - containerPort: 8181
  volumeMounts:
  - name: service-token
    mountPath: /var/run/secrets/tokens
    readOnly: true
volumes:
- name: service-token
  projected:
    sources:
    - serviceAccountToken:
        audience: "https://api.example.com"
        expirationSeconds: 3600
        path: token

Token Exchange Service

Exchange Kubernetes tokens for external service tokens
Implement OAuth2 token exchange flows
Maintain token lifecycle management

Example configuration:

containers:
- name: token-exchange
  image: token-exchange:1.0
  volumeMounts:
  - name: k8s-token
    mountPath: /var/run/secrets/tokens
  env:
  - name: TOKEN_EXCHANGE_URL
    value: "https://auth.example.com/token"
  - name: K8S_TOKEN_PATH
    value: "/var/run/secrets/tokens/token"
volumes:
- name: k8s-token
  projected:
    sources:
    - serviceAccountToken:
        audience: "https://auth.example.com"
        expirationSeconds: 3600
        path: token

Zero-trust Architecture

Short-lived tokens with minimal permissions
Regular re-authentication for all service communications
Defense in depth with multiple authentication layers
Example zero-trust configuration:
volumes: - name: auth-token projected: sources: - serviceAccountToken: audience: "https://zero-trust-gateway.example.com" expirationSeconds: 300 # 5 minutes path: token

Troubleshooting Token Projection

Common issues and troubleshooting steps for service account token projection:

Token Not Found
- Verify volume and volumeMount paths match
- Check for typos in the path configuration
- Ensure the pod has a service account assigned
- Example debug commands:
  # Check if token exists in the container kubectl exec -it <pod-name> -- ls -la /var/run/secrets/tokens/ # Verify service account assignment kubectl get pod <pod-name> -o jsonpath='{.spec.serviceAccountName}'
Token Rejected by Service
- Verify the audience matches the service's expected audience
- Check token expiration time
- Inspect token contents to verify claims
- Example debugging steps:
  # Extract token from pod TOKEN=$(kubectl exec -it <pod-name> -- cat /var/run/secrets/tokens/token) # Decode JWT token to inspect claims echo $TOKEN | base64 --decode | jq . # Validate audience claim echo $TOKEN | base64 --decode | jq -r '.aud'
Permission Issues
- Ensure the service account has necessary RBAC permissions
- Check for proper role bindings
- Verify namespace restrictions
- Example RBAC verification:
  # Check service account roles and bindings kubectl get rolebindings,clusterrolebindings --all-namespaces \ -o jsonpath='{range .items[?(@.subjects[0].name=="<service-account-name>")]}[{.roleRef.kind},{.roleRef.name}]{end}'
Token Refresh Issues
- Check kubelet logs for token refresh errors
- Verify Kubernetes API server is accessible
- Ensure kubelet has appropriate permissions
- Example kubelet log check:
  # Check kubelet logs for token-related errors sudo journalctl -u kubelet | grep -i token

Best Practices

When implementing service account token volume projection, follow these best practices:

Use Short Expiration Times
- Set the shortest practical expiration time for your use case
- Balance security needs with API server load
- Consider workload type and security requirements
- Example for different security levels:
  # High security (e.g., financial transactions) expirationSeconds: 300 # 5 minutes # Medium security (e.g., internal services) expirationSeconds: 3600 # 1 hour # Low security (e.g., read-only public data) expirationSeconds: 43200 # 12 hours
Specify Explicit Audiences
- Always set an explicit audience that matches the service
- Avoid generic audiences when possible
- Use DNS names or URIs as audience values
- Example best practice:
  # Good: specific service audience audience: "https://payment-api.example.com" # Avoid: generic audience audience: "kubernetes"

Implement Minimal Access

Create service accounts with minimal permissions
Use different service accounts for different functions
Apply RBAC policies that follow least privilege

Example RBAC configuration:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: ServiceAccount
  name: my-service-account
  namespace: default
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Properly Mount Token Volumes
- Mount volumes as read-only
- Use specific paths that are not easily guessable
- Ensure proper file permissions within containers
- Example secure mounting:
  volumeMounts: - name: token-volume mountPath: /var/run/secrets/tokens readOnly: true
Monitor Token Usage
- Implement logging for token access
- Set up alerts for unusual token requests
- Track token usage patterns
- Example monitoring configuration:
  # Prometheus alert for high token request rate alert: HighTokenRequestRate expr: rate(serviceaccount_token_requests_total[5m]) > 10 for: 10m labels: severity: warning annotations: summary: "High token request rate detected" description: "Service account token requests exceeding normal threshold"

Migration from Legacy Tokens

If you're migrating from legacy token mounting to projected tokens, follow these steps:

Identify Workloads Using Legacy Tokens
# Find pods using the legacy token mount kubectl get pods --all-namespaces -o json | jq '.items[] | select(.spec.automountServiceAccountToken == null or .spec.automountServiceAccountToken == true) | .metadata.name + " in namespace " + .metadata.namespace'

Update Application Code

Modify code to read from the new token path
Consider supporting both paths during transition
Example updated code:

def get_token():
    # Try projected token first, fall back to legacy path
    token_paths = [
        "/var/run/secrets/tokens/token",  # Projected token
        "/var/run/secrets/kubernetes.io/serviceaccount/token"  # Legacy token
    ]
    
    for path in token_paths:
        try:
            with open(path, "r") as f:
                return f.read().strip()
        except FileNotFoundError:
            continue
            
    raise Exception("No valid token found")

Update Deployment Manifests

Add projected token volume configuration
Set automountServiceAccountToken: false to disable legacy token
Deploy updated configuration
Example migration configuration:

apiVersion: v1
kind: Pod
metadata:
  name: migrated-pod
spec:
  serviceAccountName: my-service-account
  automountServiceAccountToken: false  # Disable legacy token
  containers:
  - name: app
    image: app:1.0
    volumeMounts:
    - name: token-volume
      mountPath: /var/run/secrets/tokens
  volumes:
  - name: token-volume
    projected:
      sources:
      - serviceAccountToken:
          audience: "https://kubernetes.default.svc"
          expirationSeconds: 3600
          path: token

Test Thoroughly
- Verify application functionality with projected tokens
- Check for authentication errors or token handling issues
- Validate token renewal behavior

Service Account Token Volume Projection in Kubernetes v1.27+

Recent Kubernetes versions have introduced additional capabilities:

ImplicitServiceAccountToken Feature Gate
- Controls automatic token mounting behavior
- Setting to false requires explicit projection
- Important for security-focused clusters
- Example cluster configuration:
apiVersion: kubeadm.k8s.io/v1beta3 kind: ClusterConfiguration apiServer: extraArgs: feature-gates: "ImplicitServiceAccountToken=false"
TokenRequestProjection Stable API
- Fully supported stable API for token projection
- Compatible with all modern Kubernetes distributions
- No feature gate required in recent versions
Extended Token Attributes
- Support for additional claims in projected tokens
- Enhanced binding to specific resources or operations
- Example with extended attributes:
apiVersion: v1 kind: Pod metadata: name: extended-token-pod spec: serviceAccountName: my-service-account volumes: - name: token-volume projected: sources: - serviceAccountToken: audience: "https://kubernetes.default.svc" expirationSeconds: 3600 path: token

Conclusion

Service Account Token Volume Projection represents a significant advancement in Kubernetes security practices. By providing time-bound, audience-restricted tokens that are automatically rotated, this feature addresses many of the security concerns associated with the legacy token mounting system.

Key takeaways about service account token projection:

It provides enhanced security through time-limited, audience-bound tokens
It supports complex authentication scenarios with multiple tokens and audiences
It integrates well with external authentication systems and service meshes
It requires minimal application changes when migrating from legacy tokens
It follows modern security best practices for containerized applications

As Kubernetes environments face increasing security challenges, implementing service account token projection should be considered a standard practice for securing application authentication and maintaining a strong security posture in your cluster.

Edit this page

Kubernetes Topology Spread Constraints

Advanced pod scheduling across topology domains using Topology Spread Constraints for high availability and balanced workload distribution

Welcome to From Docker to Kubernetes - Your Free Learning Journey Begins

Introducing our open-source platform for mastering containerization and orchestration.