I built a ragengine-ingest-docs CLI tool simplifies managing documents in the AKS KAITO Retrieval Augmented Generation Engine. It supports creating, updating, listing, and querying documents, enhancing demos and workshops by clearly demonstrating document indexing and retrieval behaviors.
The post outlines the installation process for the RAG Engine, detailing prerequisites like AKS and GPU provisioning. It guides users through creating an AKS cluster, installing the KAITO Workspace for model inference, and configuring the RAG Engine with a specific model. Finally, it demonstrates indexing documents and querying with RAG Engine.
The Kubernetes AI Toolchaining Operator (AKS) features a RAG engine that enables users to interact with private documents using a hosted language model, like Phi-4. This tool allows for grounded AI responses by indexing and retrieving relevant data. This is an AI platform offering management control and scalability supporting many Gen AI applications.
This blog post discusses setting up a chatbot UI using Streamlit alongside a deployed language model inference service in Azure Kubernetes. It details the process of testing the inference service with curl commands, implementing a Streamlit app, and configuring ingress rules for external access, highlighting Streamlit's user-friendly capabilities for chatbot development.
The Kubernetes AI Toolchain Operator (KAITO) automates AI/ML model processes in Kubernetes, enhancing control and security. This setup involves creating an AKS cluster, installing KAITO, and configuring an Nginx ingress for public model access, showcasing actionable AI solutions.
The Azure Kubernetes Service AI toolchain operator facilitates language model deployment by automating GPU provisioning and inference setup. Different models offer unique capabilities for various applications, with detailed cost and configuration information based on virtual machine types for optimal usage and testing insights.
KAITO is an AI toolchain operator designed for deploying language models in Kubernetes. It features various model families, including DeepSeek for advanced reasoning, Falcon for custom fine-tuning, Llama for general assistance, Mistral for efficiency, Phi for cost-sensitive tasks, and Qwen for programming. Open-weight models ensure privacy and customization options, making them suitable for enterprise workloads while allowing fine-tuning and governance.
The Azure Naming terraform module provides a consistent way to generate resource names that adhere to established conventions in Azure deployments. This tool facilitates unique naming for resources, particularly useful for ensuring no conflicts in various subscriptions or regions, enhancing organization and efficiency in Terraform projects.
I have been testing out the Azure Verified Module for Azure Kubernetes Service that can be found in the Terraform Registry at https://registry.terraform.io/modules/Azure/avm-res-containerservice-managedcluster. This module came out in October 2024 so its fairly new. This module is suitable for enterprise-grade production environments, applies Microsoft best practices, features RBAC, complex monitoring. Also it is supported by …
Continue reading Exploring Azure Verified Module for Azure Kubernetes Service
I came across the Azure Verified Module for Azure Kubernetes Service and in its Github repo I found a Well Architected Framework (WAF) Aligned example for deploying this Terraform module. So asked myself, "What exactly makes this example of deploying AKS WAF Aligned?" Before I get into that, let me explain what is WAF. It …
Continue reading Well Architected Framework With The Azure Verified Module For Azure Kubernetes
Roy Kim on Azure and AI 