The previous part 1 blog post went over fundamental concepts of ingress and ingress controller. This part 2 post will build on this concept and give a review of the App Gateway Ingress Controller (AGIC)
First of all, what happens when you deploy AKS with its default settings?
The default AKS deployment when going through the Azure Portal has network related configuration set with HTTP application routing as disabled.
Http Application Routing add-on makes it really easy to setup an ingress controller so that you can access your deployed apps in AKS. However, it is not recommended for production deployments.
Essentially, there is no real ingress controller set up upon default AKS deployment. So you have to install an ingress controller. A popular approach is with NGINX (i.e. in the cluster), you can read about it in Create an ingress controller in Azure Kubernetes Service (AKS). This will work with a public Azure Load Balancer (that is automatically deployed with AKS) to support ingress traffic outside of the AKS Virtual Network. Not that a Public IP that is also configured for this load balancer’s front end listener, by default. Note that there is the ability to setup an internal Azure Load Balancer for internal private traffic scenarios.
For a more sophisticated option, you can use the Azure Application Gateway (with Web Application Firewall feature). The most notable features are
- URL routing
- Cookie-based affinity
- Secure Sockets Layer (SSL) termination
- End-to-end SSL
- Support for public, private, and hybrid web sites
- Integrated web application firewall
For me, the web application firewall is very useful for detecting or preventing web attacks as it leverages the OWASP ModeSecurity Core Rule Set. For example, protect from cross site scripting and SQL injection attacks onto your web applications.
The App Gateway Ingress Controller architecture looks as follows. The Ingress Resource as defined in a YAML file specifies routing rules. The App Gateway Ingress Controller runs as a pod that takes the ingress resource and configures the Azure App Gateway so that ingress traffic can go to the appropriate application and pods. The AGIC needs to have the appropriate permission to configure the App Gateway.
I later learned that the AG Ingress Controller works only with the App Gateway pricing tier of Standard_v2 and WAF_v2 SKUs. This means you can’t use the cheaper and older App Gateway Standard and WAF tiers. I believe where technical requirements are sufficient for the older WAF tiers, it would be a much cheaper option. But I anticipate those older pricing tiers would perhaps be deprecated anyways. This is just a guess.
The Application Gateway Ingress Controller can achieve up to 50 percent lower network latency than in-cluster ingress controllers. You can read further about it here.
You can share the App Gateway with other Azure resources such as a VM and/or Azure App Service. Therefore, making it more cost effective as it is being shared.
My Review, Opinions and Tips from Experience
The AGIC controller manages the configuration of the Azure App Gateway based on the specifications in the ingress resource. Therefore, do no touch those configuration or else AGIC controller may just override your settings by correcting it.
For Dev/UT environments, I usually turn off the App Gateway to save on costs as it is an expensive service. The only way to do this is at the moment is through Azure PowerShell.
$AppGw = Get-AzApplicationGateway -Name <AG name> -ResourceGroupName <rg name>
Stop-AzApplicationGateway -ApplicationGateway $AppGw
Note that the running AGIC container will notice that the App Gateway is off and will eventually turn it back on. I believe this is by design for resiliency. So what I normally do is shut down the node pool VMs and then run the PowerShell cmds to turn off the App Gateway.
Use the App Gateway to leverage the Layer 7 features and share them across multiple AKS clusters and even other Azure services such as Azure Web Apps and VMs.
My hope is that I have been able to explain the roles and relationships between the ingress, ingress controller and how the App Gateway is a sophisticated approach to support the ingress traffic into the AKS cluster to access the hosted applications. This is something that has been a challenge to wrap my head around and hope to make it a smoother learning curve for you.
References
Roy Kim on Azure and Microsoft 365 
Pingback: Understanding Ingress Controllers and Azure App Gateway for Azure Kubernetes Part 1: Intro – Roy Kim on Azure, Office 365 and SharePoint
Reblogged this on El Bruno.
I’m struggling to understand how exactly the traffic is routed through the Application Gateway to the pods. I’m assuming the AGIC sets up pools on the App Gateway with pods as pool members?
My struggle is I’d like to keep all internet traffic coming through my 3rd party firewall today and avoid a second Ingress point through the app Gateway. Currently this is done through user defined routes pointing to my network virtual appliance (firewall) Is this possible? Or do you have to expose the App Gateway controlled by AGIC directly to the internet and just secure with the WAF? I
Yes the pods are pool members since they get a private IP on the VNET subnet. I believe the later is the way to go. Or you find some open source ingress controller that can work with your 3rd party firewall. Does this make sense?