Welcome to from-docker-to-kubernetes

Service Mesh & Ingress

Understanding Kubernetes service mesh architecture and ingress controllers

Service Mesh

A service mesh provides an infrastructure layer for service-to-service communication with additional capabilities like traffic management, security, and observability. Unlike traditional networking approaches, a service mesh decouples the application logic from the network communication logic by implementing a dedicated infrastructure layer.

Service meshes solve complex microservices challenges by providing:

  • Uniform network observability across services
  • Consistent security policies with mutual TLS (mTLS)
  • Intelligent traffic management without code changes
  • Built-in resilience patterns like circuit breaking and retries

Service Mesh Components

Control Plane

  • Configuration management: Centralized configuration for all mesh policies
  • Policy enforcement: Implementing security, traffic, and access policies
  • Certificate management: Handling TLS certificates for secure communication
  • API integration: Exposing APIs for mesh configuration and monitoring
  • Service discovery: Maintaining a registry of available services and endpoints

Data Plane

  • Proxies (sidecars): Lightweight network proxies (like Envoy) deployed alongside each service
  • Traffic routing: Dynamic request routing based on various attributes like headers, paths
  • Load balancing: Advanced load balancing with support for various algorithms and health checking
  • Circuit breaking: Preventing cascading failures by detecting and isolating failing services
  • Telemetry collection: Gathering detailed metrics, logs, and traces for each request

The control plane and data plane work together in a complementary fashion:

  1. The control plane defines policies and configurations
  2. These configurations are distributed to all sidecar proxies
  3. The data plane sidecars enforce these policies for every request
  4. Telemetry data flows back to the control plane for monitoring and analysis

Istio

  • Comprehensive platform: Complete solution with extensive features
  • Built on Envoy proxy: Leverages Envoy's powerful capabilities as sidecar
  • Strong traffic management: Advanced routing, splitting, and mirroring capabilities
  • Robust security features: Built-in mTLS, RBAC, and certificate management
  • Deep observability: Integrated with Prometheus, Grafana, Jaeger, and Kiali
  • Enterprise adoption: Backed by Google and widely used in production environments

Linkerd

  • Lightweight and simple: Smaller resource footprint than alternatives
  • Minimal overhead: <1ms p99 latency impact on service calls
  • Fast data plane: Written in Rust for performance and memory safety
  • Focused on simplicity: Easy installation and intuitive operation
  • Strong security defaults: Automatic mTLS without complex configuration
  • CNCF graduated project: Industry-recognized maturity and stability

Consul

  • Service discovery: Robust service registry with health checking
  • Multi-platform: Works in Kubernetes, VMs, and bare metal environments
  • Key-value store: Built-in distributed key-value storage
  • ACL support: Fine-grained access control system
  • Network segmentation: Service-to-service authorization with intentions
  • HashiCorp ecosystem: Integrates with Vault, Nomad, and other HashiCorp tools

When selecting a service mesh, consider:

  • Complexity vs. simplicity needs
  • Resource overhead constraints
  • Integration with existing tools
  • Team familiarity with technology
  • Specific feature requirements

Istio Example

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews-route
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1

Traffic Management

These traffic management capabilities allow teams to:

  • Deploy new versions with minimal risk
  • Test in production safely
  • Build resilient applications
  • Troubleshoot complex issues
  • Optimize user experience

Ingress Controllers

Ingress controllers expose HTTP and HTTPS routes from outside the cluster to services within the cluster. They serve as the entry point for external traffic and provide essential capabilities for managing that traffic.

Unlike basic Kubernetes Services of type LoadBalancer or NodePort, Ingress controllers offer:

  • Path-based routing to different backend services
  • Host-based virtual hosting for multiple domains
  • TLS/SSL termination
  • URL rewriting and redirection
  • Authentication and authorization
  • Rate limiting and traffic control

In the Kubernetes architecture, an Ingress controller consists of:

  1. A reverse proxy implementation (NGINX, HAProxy, Traefik, etc.)
  2. Controller logic that watches Kubernetes API for Ingress resources
  3. Additional configuration for the specific proxy being used

Ingress Resource

The Ingress resource is a Kubernetes API object that defines how external HTTP/HTTPS traffic should be routed to services within the cluster.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    # Controller-specific configuration
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  # Specifies which Ingress controller to use
  ingressClassName: nginx
  rules:
  # First rule: Routes for example.com
  - host: example.com
    http:
      paths:
      - path: /app
        pathType: Prefix    # Other options: Exact, ImplementationSpecific
        backend:
          service:
            name: app-service
            port:
              number: 80
  # Second rule: Routes for api.example.com
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080

Key components of the Ingress resource:

  • metadata.annotations: Controller-specific configuration options
  • spec.ingressClassName: Specifies which Ingress controller implementation to use
  • spec.rules: List of host and path rules for routing
  • spec.rules.host: Domain name for virtual hosting
  • spec.rules.http.paths: List of URL paths and their backend services
  • pathType: How to match the path (Prefix, Exact, ImplementationSpecific)

NGINX Ingress Controller

  • Most common: Widely adopted in the Kubernetes ecosystem
  • Production-grade: Battle-tested in large-scale deployments
  • HTTP load balancing: Advanced load balancing algorithms and features
  • Path-based routing: Flexible routing based on URL paths and hosts
  • TLS termination: Efficient SSL/TLS handling with optional passthrough
  • Extensive annotations: Rich set of configuration options via annotations
  • Commercial support: Available through NGINX, Inc. (F5)

Traefik

  • Dynamic configuration: Auto-detects changes and reconfigures without restarts
  • Automatic Let's Encrypt: Built-in ACME support for free TLS certificates
  • Dashboard UI: Visual management interface for monitoring and configuration
  • Middleware support: Pluggable middleware chain for request processing
  • Modern architecture: Built for cloud-native and microservices environments
  • Multiple protocols: Supports HTTP, HTTPS, TCP, gRPC, and WebSocket
  • Canary deployments: Native support for traffic splitting and canary releases

HAProxy

  • High performance: Optimized for speed and low latency
  • Advanced load balancing: Sophisticated algorithms and health checking
  • Detailed metrics: Comprehensive statistics and monitoring capabilities
  • Connection management: Fine-grained control over connection pools
  • Custom rules: Powerful ACLs and routing rules
  • Enterprise features: Rate limiting, circuit breaking, and DDoS protection
  • Low resource usage: Efficient memory and CPU utilization

Other notable Ingress controllers:

  • Ambassador/Emissary: API Gateway built on Envoy with strong developer experience
  • Kong: API Gateway with extensive plugin ecosystem
  • Contour: High-performance Ingress controller based on Envoy
  • AWS ALB Controller: Native integration with AWS Application Load Balancers
  • GKE Ingress Controller: Native integration with Google Cloud Load Balancers

Secure Ingress with TLS

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-example-ingress
spec:
  tls:
  - hosts:
      - secure.example.com
    secretName: example-tls
  rules:
  - host: secure.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80

Service Mesh and Ingress Integration

Istio Gateway

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "bookinfo.example.com"

Istio Virtual Service

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "bookinfo.example.com"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        prefix: /reviews
    route:
    - destination:
        host: reviews
        port:
          number: 9080

Advanced Ingress Patterns

Observability

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: grafana-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "grafana.example.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: grafana-vs
  namespace: istio-system
spec:
  hosts:
  - "grafana.example.com"
  gateways:
  - grafana-gateway
  http:
  - route:
    - destination:
        host: grafana
        port:
          number: 3000

Service Mesh Best Practices

Performance Considerations

  • Sidecar resource requirements
  • Control plane sizing
  • Selective mesh inclusion
  • Monitoring overhead
  • Proxy latency impact

Security Implementation

  • mTLS configuration
  • Authorization policies
  • Certificate management
  • Network policies
  • Service-to-service authentication

Ingress Best Practices

Troubleshooting

Service Mesh Issues

  • Sidecar injection failures:
    # Check if automatic injection is enabled
    kubectl get namespace -L istio-injection
    
    # Verify pod has sidecar injected
    kubectl get pod <pod-name> -o jsonpath='{.spec.containers[*].name}'
    
  • mTLS configuration problems:
    # Verify mTLS policy
    kubectl get peerauthentication -A
    
    # Test connection between services
    istioctl x describe pod <pod-name>
    
  • Routing misconfiguration:
    # Validate virtual service
    istioctl analyze virtualservice/<name>
    
    # Check if routes are applied
    istioctl proxy-config routes <pod-name>.<namespace>
    
  • Control plane connectivity:
    # Check control plane pods
    kubectl get pods -n istio-system
    
    # Verify istiod connectivity
    istioctl proxy-status
    
  • Telemetry collection issues:
    # Check if Prometheus is scraping metrics
    kubectl port-forward -n istio-system svc/prometheus 9090:9090
    # Then open http://localhost:9090 in browser
    
    # Verify traces are being collected
    kubectl port-forward -n istio-system svc/jaeger 16686:16686
    # Then open http://localhost:16686 in browser
    

Ingress Issues

  • Certificate problems:
    # Check TLS secret
    kubectl get secret <tls-secret-name> -o yaml
    
    # Verify certificate is valid
    kubectl get secret <tls-secret-name> -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout
    
  • Routing errors:
    # Validate Ingress resource
    kubectl describe ingress <ingress-name>
    
    # Check Ingress controller logs
    kubectl logs -n ingress-nginx deploy/ingress-nginx-controller
    
  • Backend connectivity:
    # Test service connectivity from inside cluster
    kubectl run -it --rm debug --image=curlimages/curl -- curl http://<service-name>.<namespace>.svc.cluster.local:<port>
    
    # Check endpoints for service
    kubectl get endpoints <service-name>
    
  • Configuration validation:
    # Verify Ingress configuration
    kubectl get ingress <ingress-name> -o yaml
    
    # Check for controller-specific issues
    kubectl get events --field-selector involvedObject.kind=Ingress
    
  • Resource constraints:
    # Check resource usage
    kubectl top pods -n ingress-nginx
    
    # Verify resource requests and limits
    kubectl describe deploy -n ingress-nginx ingress-nginx-controller
    

Common debugging techniques:

  1. Enable debug logging in Ingress controllers or mesh components
  2. Use port-forwarding to access internal dashboards and UIs
  3. Deploy test pods to verify network connectivity from inside the cluster
  4. Check controller-specific documentation for troubleshooting guides
  5. Review Kubernetes events for clues about failures

Production Deployment Checklist

Additional production considerations:

  • Documentation for operations teams
  • Backup and disaster recovery planning
  • Performance benchmarking under expected load
  • Integration with existing CI/CD pipelines
  • Training for development and operations teams