The post outlines the installation process for the RAG Engine, detailing prerequisites like AKS and GPU provisioning. It guides users through creating an AKS cluster, installing the KAITO Workspace for model inference, and configuring the RAG Engine with a specific model. Finally, it demonstrates indexing documents and querying with RAG Engine.
The Kubernetes AI Toolchaining Operator (AKS) features a RAG engine that enables users to interact with private documents using a hosted language model, like Phi-4. This tool allows for grounded AI responses by indexing and retrieving relevant data. This is an AI platform offering management control and scalability supporting many Gen AI applications.
This blog post discusses setting up a chatbot UI using Streamlit alongside a deployed language model inference service in Azure Kubernetes. It details the process of testing the inference service with curl commands, implementing a Streamlit app, and configuring ingress rules for external access, highlighting Streamlit's user-friendly capabilities for chatbot development.
The Kubernetes AI Toolchain Operator (KAITO) automates AI/ML model processes in Kubernetes, enhancing control and security. This setup involves creating an AKS cluster, installing KAITO, and configuring an Nginx ingress for public model access, showcasing actionable AI solutions.
As I was starting to try out Azure Foundry Safety and Security feature, I confronted with the error "Your account does not have access to this resource, please contact your resource owner to get access". And so I went to the Management Center, to check user permissions and yet I have owner permissions at the …
Continue reading Permissions with Azure AI Foundry: Safety And Security
Now that I have fine-tuned a model in Part 2, next is to deploy the fine tuned model into a new Kaito workspace. This blog post is part of a series.Part 1: Intro and overview of the KAITO fine-tuning workspace yamlPart 2: Executing the Training Kubernetes Training JobPart 3: Deploying the Fine-Tuned ModelPart 4: Evaluating …
I will continue from the Part 1 to execute the deployment of the fine-tuning workspace job. This blog post is part of a series.Part 1: Intro and overview of the KAITO fine-tuning workspace yamlPart 2: Executing the Training Kubernetes Training JobPart 3: Deploying the Fine-Tuned ModelPart 4: Evaluating the Fine-Tuned Model Let' start the fine …
Continue reading Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 2: Execution
I uncover the process and workings of fine-tuning a large language model on a dataset. Scenario for fine-tuning a model can be on corporate data such as a knowledge base, product information, human resources. I walk you through the process and share my detailed observations. What is Fine-tuning a large language model? Involves adjusting a …
Continue reading Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 1: Intro
I have been exploring some in depth tutorials in building a chat application. It implements Retrieval Augmentation Generation (RAG) on a product database in Azure AI Search. The scenario is for a retail customer to ask product recommendations on camping gear. This solution employs Azure AI Foundry for LLM models and interfacing. My goal of …
Continue reading Building a Chat App using Azure AI Foundry SDK and AI Search
The Azure AI Foundry portal is a cohesive environment for developing AI solutions, enabling users to create Hubs and Projects that facilitate collaboration among data scientists, developers, and engineers. It integrates services like Azure Open AI, offering shared resources and role-based access management while streamlining AI services' deployment and management processes.
Roy Kim on Azure and AI 