Kubernetes Autoscaling

Drivers

🤷 Reasons for Scaling

Scale Up

1. Latency.
2. Availability.
3. Throughput.

Scale Down

1. Costs.
2. Density.
3. Sharing.

👨‍👧‍👦 Autoscaling Maturity Model

| Level            | Monitoring       | Scaling                   | Benchmarking                |
| ---------------- | ---------------- | ------------------------- | --------------------------- |
|  0 - Static      | No observability | Best guess provisioning   | No performance/load testing |
|  1 - Coarse      | CPU/Memory       | Based on CPU/Memory       | Manual load tests           |
|  2 - Qualitative | Calls/Latency    | Based on calls/latency    | Automatic but periodic      |
|  3 - Optimising  | Tracing          | Adaptive                  | Automatic per commit        |

Level 0 - Static

👨‍👧‍👦 Autoscaling Maturity Model

• No observability.
  • pod restarts only indicator of workload behaviour.
• Best guess provisioning.
  • example 3 replicas for HA.
• No performance/load testing.
  • high value, low volume usage might not be high priority (e.g. 10s of RPS).

Level 1 - Coarse

👨‍👧‍👦 Autoscaling Maturity Model

• CPU/Memory monitoring.
  • metrics-server process and host CPU/Memory utilisation.
• Scaling based on course metrics.
  • scale with what you've got.
• Manual load tests.
  • one off or per release go/no-go.

Level 2 - Qualitative

👨‍👧‍👦 Autoscaling Maturity Model

• Call/Latency Monitoring.
  • RPS, Latency by Call, etc.
• Scaling based on user experience.
  • RPS is a good basis to scale on.
• Automated periodic load tests.
  • nightly runs, release tagging, etc.

Level 3 - Optimising

👨‍👧‍👦 Autoscaling Maturity Model

• Adaptive load management.
  • circuit breakers, load-shedding, retry limits, exponential backoff, partial degradation.
• Automated load tests per commit.
• Distributed Tracing
  • useful for identifying scaling relationships between components in microservices.

📈 Scaling in Kubernetes

Scaling Maths 101

📈 Scaling in Kubernetes

desiredReplicas = ceil(currentReplicas * (currentMetric/desiredMetric))

    Given a target utilization of 60%
    And a replica count of 4 pods
    And an average utilization of 80%
    When the HPA evaluates the metrics
    Then it should scale to 6 pods.

 ceil(4 * (100/60)).

Level 0 - Static

📈 Scaling in Kubernetes

kubectl scale --replicas=2 -n instana-dev deployment/fizzbuzz 

Level 1 - Coarse

📈 Scaling in Kubernetes

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: fizzbuzz
  namespace: instana-dev
spec:
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - resource:
      name: cpu
      target:
        averageUtilization: 60
        type: Utilization
    type: Resource
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: fizzbuzz

Level 2 - Qualitative - Landscape

📈 Scaling in Kubernetes

• Azure Adapter.
• KEDA (K8s event driven autoscaling component)
• Kube metrics adapter.
• Prometheus adapter.
• Instana adapter - coming soon!

Thank you