Using WebRTC with the OpenAI Realtime API

In October 2024, OpenAI introduced the Realtime API. It enables developers to integrate low-latency, multimodal conversational experiences into their applications. It supports both text and audio inputs and outputs, facilitating natural speech-to-speech interactions without the need for multiple models.

It addresses the following problems:

Simplified Integration: Combines speech…

19/12/2024 Article Bloggers Geert Baeke

Share:

You may be interested in

Finding images with text and image queries with the help of GPT-4 Vision
With the gpt-4-vision-preview model available at OpenAI, it was time to build something with it. I decided to use it as part of a quick solution that can search for…
29/11/2023
Using WebRTC with the OpenAI Realtime API
In October 2024, OpenAI introduced the Realtime API. It enables developers to integrate low-latency, multimodal conversational experiences into their applications. It supports both text and audio inputs and outputs, facilitating…
19/12/2024
Fast chat bot creation with the OpenAI Assistants API and the Microsoft Bot Framework SDK
This post is part of a series of blog posts about the Azure OpenAI Assistants API. Here are the previous posts: Part 1: introduction Part 2: using tools Part 3:…
11/02/2024
Creating a Copilot declarative agent with VS Code and the Teams Toolkit
If you are a Microsoft 365 Copilot user, you have probably seen that the words “agent” and “Copilot agent” are popping up here and there. For example, if you chat…
31/10/2024
Writing your first flow with Prompt Flow in Visual Studio Code
In this blog post, we will create a flow with Prompt Flow in Visual Studio Code. Prompt Flow is a suite of development tools to build LLM-based AI applications. It…
12/12/2023
Load balancing OpenAI API calls with LiteLLM
If you have ever created an application that makes calls to Azure OpenAI models, you know there are limits to the amount of calls you can make per minute. Take…
16/04/2024
Kubernetes Blue-Green deployments with Argo Rollouts
In this post, we will take a look at / blue-green deployments in Kubernetes. With blue-green deployments, you deploy a new version of an application or service next to the…
01/11/2021
AKS Workload Identity Revisited
A while ago, I blogged about Workload Identity. Since then, Microsoft simplified the configuration steps and enabled Managed Identity, in addition to app registrations. But first, let’s take a step…
24/11/2022
DNS Options for Private Azure Kubernetes Service
When you deploy Azure Kubernetes Service (AKS), by default the API server is publicly made available. That means it has a public IP address and an Azure-assigned name that’s resolvable…
01/07/2021
Approving a private endpoint connection with Azure CLI
In my previous post, I wrote about App Services with Private Link and used Azure Front Door to publish the web app. Azure Front Door Premium (in preview), can create…
24/06/2021
Use Azure OpenAI on your data with Semantic Kernel
I have written before about Azure OpenAI on your data. For a refresher, see Microsoft Learn. In short, Azure OpenAI on your data tries to make it easy to create…
24/03/2024
Create a Copilot declarative agent that calls an API with authentication
In a previous post, we looked at creating a Copilot declarative agent. The agent had one custom action that called the JSONPlaceholder API. Check that post for an introduction to…
05/11/2024