Autoscaling

Setting up Auto Scaling

Horizontal Pod Autoscaler

Prerequisite

Make sure Metrics Server is installed on the cluster

You can check this using:

kubectl top pods -n kube-system

To install, apply the metrics server manifest:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

This can be verified using:

kubectl get deployment metrics-server -n kube-system

Step 1

Create a HPA manifest

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
    name: uplevyl-api
spec:
    scaleTargetRef:
        apiVersion: apps/v1
        kind: Deployment
        name: uplevyl-api
    minReplicas: 1
    maxReplicas: 10
    metrics:
        - type: Resource
          resource:
              name: cpu
              target:
                  type: Utilization
                  averageUtilization: 75
        - type: Resource
          resource:
              name: memory
              target:
                  type: AverageValue
                  averageValue: 1Gi

Apply this using:

kubectl apply -f kube/hpa.yaml

Step 2

Verify the HPA is running correctly using

kubectl get hpa -w

Cluster Autoscaler

Cluster Autoscaling scales the Node Group the nodes in the cluster run on. Each cluster has at least one Node Group, which defines a template for nodes. When we scale up, we spin up more Nodes (ec2 instances) inside that Node Group.

Prerequisite

Make sure you have created a Node Group via eksctl with an Auto Scaling Group.

Use the --asg-access flag or add this to the nodegroup config:

  iam:
      withAddonPolicies:
          autoScaler: true

Step 1

Create a new IAM policy (if eksctl has not already) called AmazonEKSClusterAutoscalerPolicy.

Give it the policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:DescribeLaunchConfigurations",
                "autoscaling:DescribeTags",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup",
                "ec2:DescribeLaunchTemplateVersions"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

Check policy for updates here https://docs.aws.amazon.com/eks/latest/userguide/autoscaling.html#cluster-autoscaler

Step 2

Create an IAM role using eksctl and attach the policy from Step 1

eksctl create iamserviceaccount \
  --cluster=<CLUSTER_NAME> \
  --profile=<PROFILE_NAME> \
  --namespace=kube-system \
  --name=cluster-autoscaler \
  --attach-policy-arn=<POLICY_ARN> \
  --override-existing-serviceaccounts \
  --approve \
  --region=<REGION>

Step 3

Download the autoscaler from the source if you do not have it already.

Replace all instances of <YOUR CLUSTER NAME> with the name of the cluster.

Apply the config

kubectl apply -f cluster-autoscaler-autodiscover.yaml

Load Testing

To test the autoscaling capabilities, you can use Artillery. This is an npm tool that can run a test plan.

This can be run locally, but may be constrained by local networking resources.

A better option is to run it on an EC2 instance, or on the Kubernetes cluster itself.

SSH into a node image on the cluster:

kubectl run -i --tty load-generator --rm --image=node --restart=Never -- /bin/sh

Install Artillery globally:

npm i -g artillery

Create a test plan on the image:

# cat > test.yaml
config:
    target: http://uplevyl-api-service:4000
    phases:
        - duration: 500
          arrivalRate: 1000
scenarios:
    - flow:
          - post:
                url: '/graphql'

Ensure the target points to the service URL

This will create 1000 users every second for 500 seconds.

To run this test plan:

artillery run test.yaml