Category: Kubernetes

Exploring Azure Verified Module for Azure Kubernetes Service

On By Roy Kim (MVP)In Azure, Kubernetes, TerraformLeave a comment

I have been testing out the Azure Verified Module for Azure Kubernetes Service that can be found in the Terraform Registry at https://registry.terraform.io/modules/Azure/avm-res-containerservice-managedcluster. This module came out in October 2024 so its fairly new. This module is suitable for enterprise-grade production environments, applies Microsoft best practices, features RBAC, complex monitoring. Also it is supported by …

Continue reading Exploring Azure Verified Module for Azure Kubernetes Service

Well Architected Framework With The Azure Verified Module For Azure Kubernetes

On By Roy Kim (MVP)In Azure, Kubernetes, TerraformLeave a comment

I came across the Azure Verified Module for Azure Kubernetes Service and in its Github repo I found a Well Architected Framework (WAF) Aligned example for deploying this Terraform module. So asked myself, "What exactly makes this example of deploying AKS WAF Aligned?" Before I get into that, let me explain what is WAF.  It …

Continue reading Well Architected Framework With The Azure Verified Module For Azure Kubernetes

Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 4: Evaluation

On By Roy Kim (MVP)In AI, Azure, Kubernetes3 Comments

In the previous article Part 3, I have shown deploy the fine-tuned model on Azure Kubernetes Service with the Kaito add-on. In this article, I will show manual evaluation with a series of prompts taken from the fine-tuning dataset. This blog post is part of a series.Part 1: Intro and overview of the KAITO fine-tuning …

Continue reading Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 4: Evaluation

Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 3: Deploying the FT Model

On By Roy Kim (MVP)In AI, Azure, Kubernetes4 Comments

Now that I have fine-tuned a model in Part 2, next is to deploy the fine tuned model into a new Kaito workspace. This blog post is part of a series.Part 1: Intro and overview of the KAITO fine-tuning workspace yamlPart 2: Executing the Training Kubernetes Training JobPart 3: Deploying the Fine-Tuned ModelPart 4: Evaluating …

Continue reading Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 3: Deploying the FT Model

Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 2: Execution

On By Roy Kim (MVP)In AI, Azure, Kubernetes3 Comments

I will continue from the Part 1 to execute the deployment of the fine-tuning workspace job. This blog post is part of a series.Part 1: Intro and overview of the KAITO fine-tuning workspace yamlPart 2: Executing the Training Kubernetes Training JobPart 3: Deploying the Fine-Tuned ModelPart 4: Evaluating the Fine-Tuned Model Let' start the fine …

Continue reading Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 2: Execution

Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 1: Intro

On By Roy Kim (MVP)In AI, Azure, Kubernetes4 Comments

I uncover the process and workings of fine-tuning a large language model on a dataset. Scenario for fine-tuning a model can be on corporate data such as a knowledge base, product information, human resources. I walk you through the process and share my detailed observations. What is Fine-tuning a large language model? Involves adjusting a …

Continue reading Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 1: Intro

Effortlessly Setup Kaito v0.3.1 on Azure Kubernetes Service To Deploy A Large Language Model

On By Roy Kim (MVP)In AI, Azure, Kubernetes4 Comments

KAITO simplifies the deployment of large language models (LLMs) in Azure Kubernetes Service (AKS) environments with preset GPU configurations. This tool automates the setup process, including node provisioning and identity management, essential for data experiments while ensuring security compliance. It enhances efficiency, allowing engineers to focus on AI/ML model experimentation. #azure #kubernetes #AI #genAI #mvpbuzz

Did My Apps Go Down During AKS Upgrade?

On By Roy Kim (MVP)In Azure, Kubernetes1 Comment

I did an experiment of running a load test against 4 applications and while executing a Azure Kubernetes Cluster Upgrade to see the affects and any downtime. The applications hosted in my AKS cluster are AKS Helloworld - super simple application Voting App - simple application with Redis backend Bookinfo Istio demo - microservices architecture …

Continue reading Did My Apps Go Down During AKS Upgrade?

Fixing A Failed State AKS Cluster

On By Roy Kim (MVP)In Azure, Kubernetes1 Comment

If you ever encounter the following error, I'll show you one way that helped me resolve my AKS cluster. The cluster is in a failed state. If you didn't do an operation, AKS may resolve the provisioning status automatically if your cluster applications continue to run. To further troubleshoot, you can refer to https://github.com/MicrosoftDocs/SupportArticles-docs/blob/main/support/azure/azure-kubernetes/cluster-node-virtual-machine-failed-state.md For …

Continue reading Fixing A Failed State AKS Cluster