Issue: In my Azure Kubernetes Service cluster, the horizontal pod autoscaler is not working due to resource cpu on pods is unknown.

Horizontal Pod Autoscaling increases (and decreases) the number of replicas (or pods) on deployments based on cpu usage. I have come across this issue a number of times when setting up HPA.

HPA not working – Events:

  Type     Reason               Age                     From                       Message
  —-     ——               —-                    —-                       ——-
Warning  FailedGetPodsMetric  3m46s (x3719 over 15h)  horizontal-pod-autoscaler  unable to get metric packets-per-second: unable to fetch metrics from custom metrics API: no custom metrics API (custom.metrics.k8s.io) registered

Ignore the get metrics packets-per second metric as that is not relevant for basic HPA.

Check that the metrics server is running. It is installed by default with an AKS cluster deployment.

kubectl get deploy,svc -n kube-system | egrep metrics-server

kubectl get –raw “/apis/metrics.k8s.io/v1beta1/nodes” 

Data is returned from all nodes, so metrics server is working 

One observation is that the k8s deployment does not have any resources defined under containers. Resources 

Resolution 

Ensure to include 

spec: 
      volumes: 
        - name: tmp 
          emptyDir: {} 
      containers: 
        - name: productpage 
          image: 'docker.io/istio/examples-bookinfo-productpage-v1:1.15.0' 
          ports: 
            - containerPort: 9080 
              protocol: TCP 
          resources: 
            limits: 
              cpu: 100m 
              memory: 128Mi 
            requests: 
              cpu: 50m 
              memory: 64Mi

Verify: