Objective: To provide key configuration parameters for the beginner when creating the Azure Kubernetes Service with Azure CLI.

My design with related az aks create command configuration parameters.

You can find the az aks create command’s documentation provides a list of about 60 parameters.

az aks create --name
              --resource-group
              [--aad-admin-group-object-ids]
              [--aad-client-app-id]
              [--aad-server-app-id]
              [--aad-server-app-secret]
              [--aad-tenant-id]
              [--admin-username]
              [--api-server-authorized-ip-ranges]
              [--attach-acr]
              [--ca-profile]
              [--client-secret]
              [--disable-rbac]
              [--dns-name-prefix]
              [--dns-service-ip]
              [--docker-bridge-address]
              [--enable-aad]
              [--enable-addons]
              [--enable-cluster-autoscaler]
              [--enable-managed-identity]
              [--enable-node-public-ip]
              [--enable-private-cluster]
              [--enable-rbac]
              [--generate-ssh-keys]
              [--kubernetes-version]
              [--load-balancer-idle-timeout]
              [--load-balancer-managed-outbound-ip-count]
              [--load-balancer-outbound-ip-prefixes]
              [--load-balancer-outbound-ips]
              [--load-balancer-outbound-ports]
              [--load-balancer-sku]
              [--location]
              [--max-count]
              [--max-pods]
              [--min-count]
              [--network-plugin {azure, kubenet}]
              [--network-policy]
              [--no-ssh-key]
              [--no-wait]
              [--node-count]
              [--node-osdisk-diskencryptionset-id]
              [--node-osdisk-size]
              [--node-vm-size]
              [--nodepool-labels]
              [--nodepool-name]
              [--nodepool-tags]
              [--outbound-type {loadBalancer, userDefinedRouting}]
              [--pod-cidr]
              [--service-cidr]
              [--service-principal]
              [--skip-subnet-role-assignment]
              [--ssh-key-value]
              [--subscription]
              [--tags]
              [--uptime-sla]
              [--vm-set-type]
              [--vnet-subnet-id]
              [--windows-admin-password]
              [--windows-admin-username]
              [--workspace-resource-id]
              [--zones {1, 2, 3}]

I will explain in detail 16 of my recommended parameters and its values for starting out a development environment with the intent of leaning towards a production design. Hope this can help kick start your journey.

Here is my bash script:

rgName='linux-demos' 
aksName='rkaks-vsent'
location='canadacentral'

# create AKS cluster
az group create -l $location -n $rgName --subscription $appSubId

# Azure Container Registry
# set this to the name of your Azure Container Registry.  It must be globally unique
acrName=rkimacrVSEnt 
# Note: acr in enterprise rg in Enterprise subscription
az acr create --name $acrName --resource-group $rgName -l $location --sku Basic
acrResourceId=$(az acr show --name $acrName --resource-group $rgName --query "id" --output tsv)

# create vnet and subnet
az network vnet create -g $rgName -n aksVnet  -l $location --address-prefix 10.0.0.0/24 \
    --subnet-name akssubnet --subnet-prefix 10.0.0.0/25 
subnetId=$(az network vnet subnet show --resource-group $rgName --vnet-name aksVnet --name akssubnet --query id -o tsv)
echo $subnetId

# OR get existing VNET subnets
az network vnet subnet list --resource-group $rgName --vnet-name rkLinuxVNET -o tsv
subnetId=$(az network vnet subnet show --resource-group $rgName --vnet-name rkLinuxVNET --name aks --query id -o tsv)
# VM SKUS 
az vm list-skus --location $location -o table
# versions
az aks get-versions --location $location --output table

az aks create --resource-group $rgName --name $aksName \
    --kubernetes-version 1.19.0 \
    --location $location \
    --node-vm-size Standard_D2_v3 \
    --vm-set-type VirtualMachineScaleSets \
    --node-osdisk-size 30 \
    --node-count 1 --max-pods 30 \
    --network-plugin azure 
    --vnet-subnet-id $subnetId \
    --load-balancer-sku Basic \
    --generate-ssh-keys \
    --enable-cluster-autoscaler --min-count 1 --max-count 5 \
    --enable-aad \
    --enable-managed-identity \
    --attach-acr $acrResourceId

My preferred configuration values

–node-count 1

The number of Azure Virtual machines upon creating the cluster. For a dev environment and keep costs down, I generally like to start with 1 node. For production, the recommendation is 3 nodes and that is the default. One can easily manually scale to 3 nodes in the Azure Portal.

–kubernetes-version 1.19.0

The specific Kubernetes version. To find the list of available, run az aks get-versions –location $location –output table

–location canadacentral

The location of this resource by region. Different regions provide slightly different options. For example, for the Kubernetes version you want to check available options by running
az aks get-versions –location $location –output table

–vm-set-type VirtualMachineScaleSets

This is the Agent pool vm set type that is either VirtualMachineScaleSets or AvailabilitySet. I recommend the VirtualMachineScaleSet as they are intended to be identical set of VMs and designed for automatic scaling. Availability Set are a manual group of VMs and don’t have to be identical yet still distributed in the data center for availability. You can manually scale the VMs.

–node-osdisk-size 30

As a cost saving measure in a dev environment, I like to set the OS Disk size to the minimum of 30 GB, otherwise it will be 128GB.

–node-vm-size Standard_D2s_v3

The VM size of each node (in the initial node pool). This is set to 2 vCPU and 8GB of memory which is starting point for very small workloads in a development environment.

–network-plugin azure

The network-plugin has two options: Kubenet (default) and Azure CNI. I recommend Azure CNI which has more capabilities, better network performance. Know that with Azure CNI, every pod gets an IP in the Virtual Network address space. This is a deep topic and so I suggest further reading at

• https://kubernets.io/docs/concepts/cluster-administration/networking/#azure-cni-for-kubernetes
• https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/
• https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni

–vnet-subnet-id $subnetId

Pass in the subnetId of an existing VNet Subnet. Although a VNET is created by default in creating an AKS cluster, I highly recommend to create a separate VNET and associate the AKS cluster to that. This will be practice going into production and might as well understand how this is setup. Remember the subnet address space needs to big enough to have all the possible pod IP addresses.

Here’s a script to create create a VNET and find its subnet ID

az network vnet create -g $rgName -n aksVnet  -l $location –address-prefix 10.0.0.0/24 \

    –subnet-name akssubnet –subnet-prefix 10.0.0.0/25

subnetId=$(az network vnet subnet show –resource-group $rgName –vnet-name aksVnet –name akssubnet –query id -o tsv)

–max-pods 30

The maximum nodes deployable to a node. The default in Azure CNI is 30 and max is 250. Note that with Azure CNI network plugin, there will be 30 NICs created * num of nodes in the subnet as connected devices. This is so that every pod and potentially created pod has its own IP address on the subnet.

–load-balancer-sku Basic

Azure Load Balancer SKU selection for your cluster. Basic or Standard. For dev purposes, Basic is sufficient, but recommend Standard for production environment and the potential to leverage more capabilities. For further the differences, read https://docs.microsoft.com/en-us/azure/load-balancer/skus#skus

–generate-ssh-keys

SSH keys will be stored in the ~/.ssh directory. You use these keys to connect to the Linux VM Nodes. https://docs.microsoft.com/en-us/azure/virtual-machines/linux/mac-create-ssh-keys
I usually don’t connect into the Linux VM nodes but it good to be aware for any troubleshooting.

–enable-managed-identity

The AKS cluster can get a system assigned managed identity. This is beneficial when using Azure AD Pod Identity to access other Azure resources without any passwords or client secrets. For example, accessing an Azure SQL Database without a database password in the SQL connection string.

–enable-cluster-autoscaler –min-count 1 –max-count 5

It is beneficial to take advantage of autoscaling, but you need VirtualMachineScaleSets. You can set the min and max count. You can easily disable the auto scaler and manually set the number of nodes for better cost control in your dev environments. For production it is recommended to keep min-count to 3. You can read more at https://docs.microsoft.com/en-us/azure/aks/cluster-autoscaler

–enable-aad

Highly recommend enabling Azure AD Integration as you no longer need to configure Azure AD client and server apps. This help support Role Based Access Security where you can assign permissions to an Azure AD user or Azure AD group. For example assign an Azure AD group as the AKS admins with admin privileges.

You can read more at https://docs.microsoft.com/en-us/azure/aks/managed-aad

–enable-managed-identity

Enabling managed identity creates a system assigned identity for the AKS cluster. This is used for scenarios where you can set up Azure AD pod identity so that pods (using the managed identity) can access other Azure services such as Azure SQL in a ‘passwordless’ approach.

–attach-acr $acrResourceId

You will definitely use Azure Container Registry to store your application’s docker images. By attaching the ACR you grant the ‘acrpull’ role assignment to the ACR so that docker images can be deployed into the AKS cluster via Kubernetes manifest files.
Here is the script to create an ACR and find its resource ID
az acr create –name $acrName –resource-group $rgName -l $location –sku Basic
acrResourceId=$(az acr show –name $acrName –resource-group $rgName –query “id” –output tsv)

Conclusion

I hope I’ve explained the creation and design of an AKS story from the point of view of a use case. That is baking a cake than just presenting a recipe.

References

• https://docs.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az_aks_create
• https://docs.microsoft.com/en-us/azure/aks/managed-aad
• https://docs.microsoft.com/en-us/azure/aks/azure-ad-rbac