Load Balancing for LLM models using API Management

This is how to overcome the limitations of number of tokens per subscription and per region. Just by creating multiple instances of LLM in the same or multiple locations. But, now, how to consume these models ? Well, there is where Azure API Management comes into play.

Disclaimer: This video…

14/01/2025 Video Bloggers Houssem Dellai

Share:

You may be interested in

Kubecon EU in Paris
I attended Kubecon Europe in Paris and made some interviews. Here is the final video. Follow me on Twitter for more content: https://twitter.com/houssemdellai
27/03/2024
AKS and API Management: better together
You can use an #IngressController in #AKS to securely expose your apps. But if you are required to use #Azure #API Management as a central hub to your apps, then…
18/07/2024
Running Deepseek R1 LLM on Azure Container Apps
In this video, you'll learn how to deploy and run the Deepseek R1 Large Language Model (LLM) on Azure Container Apps. We'll guide you through downloading the model from Hugging…
08/02/2025
Introduction to Azure Firewall
Learn in this video the why and how of an #Azure #Firewall. Hint: it is all about #security ! If you're interested in learning more, I invite you to join…
22/01/2024
App Routing, the best AKS feature ?
When you expose your AKS applications, you typically use ingress. With ingress, you will need to manage: 1) Private and public Ingress Controllers 2) DNS custom domain names 3) TLS…
19/04/2024
Semantic Caching for LLM models
This is how to enhance the performance of intelligent applications by implementing cache. For the case of LLMs, it is a bit different as we are dealing with user prompts.…
15/01/2025
The Azure services that resolves DNS complications
I saw many recent architectures using this new Azure service. It solves one of the headaches for cloud architects: #DNS. I got interested in the topic. I spent long hours…
16/02/2024
Intro and demo for App Gateway with Ingress Controller for AKS
This is how you can setup App Gateway with an Ingress Controller to manage ingress traffic in AKS. This is an alternative to using AGIC. Disclaimer: This video is part…
28/07/2025
#Azure #LandingZone for #ContainerApps

02/10/2023
Getting Container Apps logs in Grafana
Grafana is very popular with Kubernetes. Does it work with anything else ? Grafana is very popular with collecting metrics. Does it work with logs ? Yes it do! You…
16/04/2024
Why Azure Load Balancer

20/11/2023
Preserving client IP address in Kubernetes
Learn how you can preserve and retrieve real client IP address in Kubernetes. This video is part of my course on Udemy: https://www.udemy.com/course/learn-aks-network-security Follow me on Twitter for more content:…
07/12/2024