KEDA (Kubernetes Event-Driven Autoscaling)

Event-driven autoscaler for Kubernetes workloads. KEDA enables Kubernetes to scale applications based on external metrics and events, going beyond standard CPU/memory-based scaling.

Features

Event-Driven Scaling

• External Metrics: Scale based on queue length, database rows, HTTP traffic, etc.
• Scale to Zero: Scale down to zero pods when no events, saving costs
• Multiple Scalers: 50+ built-in scalers for popular services
• Custom Scalers: Build your own scalers for any metric source
• Composite Scalers: Combine multiple metrics for scaling decisions

Built-in Scalers

Message Queues

• Azure Service Bus
• RabbitMQ
• Apache Kafka
• AWS SQS
• Google Pub/Sub
• NATS
• Redis Lists/Streams

Databases

• PostgreSQL
• MySQL
• MongoDB
• Cosmos DB
• SQL Server

Cloud Services

• Azure Monitor
• AWS CloudWatch
• Google Cloud Monitoring
• Prometheus

Other

• HTTP endpoints
• Cron schedules
• CPU/Memory metrics
• Custom metrics servers

Kubernetes Native

• CRDs: Extends Kubernetes with custom resources
• HPA Integration: Works with Horizontal Pod Autoscaler
• RBAC: Secure with Kubernetes RBAC
• Multi-cluster: Deploy across multiple clusters

How It Works

1. Define a ScaledObject pointing to your deployment
2. Configure one or more scalers with trigger metrics
3. KEDA monitors the metrics
4. Scales deployment based on thresholds
5. Can scale to zero when idle

Example Configuration

Azure Service Bus Scaler

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor-scaler
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 0
  maxReplicaCount: 100
  triggers:
  - type: azure-servicebus
    metadata:
      queueName: orders
      messageCount: "5"
      connectionFromEnv: SERVICEBUS_CONNECTION

Kafka Scaler

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: kafka-consumer-scaler
spec:
  scaleTargetRef:
    name: kafka-consumer
  triggers:
  - type: kafka
    metadata:
      bootstrapServers: kafka:9092
      consumerGroup: my-group
      topic: events
      lagThreshold: "10"

HTTP Scaler

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: api-scaler
spec:
  scaleTargetRef:
    name: api-deployment
  triggers:
  - type: prometheus
    metadata:
      serverAddress: http://prometheus:9090
      metricName: http_requests_total
      threshold: "100"
      query: sum(rate(http_requests_total[1m]))

Use Cases

Event Processing

• Process messages from queues as they arrive
• Scale based on queue depth
• Handle bursts efficiently
• Scale to zero when no messages

Scheduled Jobs

• Run batch jobs on schedule
• Process data at specific times
• Combine with cron triggers
• Clean up resources when done

API Services

• Scale APIs based on request rate
• Handle traffic spikes automatically
• Reduce costs during low traffic
• Maintain performance SLAs

Stream Processing

• Process Kafka/Event Hub streams
• Scale based on consumer lag
• Maintain throughput guarantees
• Optimize resource usage

Benefits

Cost Optimization

• Scale to Zero: No pods running when idle
• Right-sizing: Match resources to actual demand
• Burst Handling: Add capacity only when needed
• Spot Instances: Compatible with spot/preemptible instances

Performance

• Responsive: Quick scaling decisions
• Fine-grained: Scale based on actual metrics
• Predictable: Deterministic scaling behavior
• Efficient: Minimal overhead

Developer Experience

• Simple Configuration: YAML-based setup
• Kubernetes Native: Works with existing tools
• Flexible: Supports many event sources
• Observable: Integrates with monitoring tools

Installation

Helm

helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda --namespace keda --create-namespace

YAML

kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.12.0/keda-2.12.0.yaml

Best Practices

• Start with conservative scaling thresholds
• Monitor scaling behavior and adjust
• Use scale to zero for development environments
• Combine multiple triggers for robust scaling
• Set appropriate min/max replica counts
• Test scaling behavior under load
• Use authentication for secure metric access

vs Kubernetes HPA

Standard HPA

• CPU/memory metrics only
• Can't scale to zero
• Simple but limited

KEDA

• Any external metric
• Scales to zero
• Event-driven architecture
• More complex but powerful

Best For

• Event-driven architectures
• Queue-based processing systems
• Cost-sensitive workloads
• Bursty traffic patterns
• Scheduled batch jobs
• Serverless-style deployments on Kubernetes

Ecosystem

• CNCF Project: Part of Cloud Native Computing Foundation
• Active Community: Regular updates and new scalers
• Enterprise Support: Multiple vendors offer support
• Integration: Works with service meshes, Istio, etc.

KEDA transforms Kubernetes into an event-driven platform, enabling true serverless-style scaling with the flexibility to scale based on any metric that matters to your application.