Skip to content
HardwareLab
Voice Assistants & CompanionsIntermediate · Our suggestion

OpenAI Realtime on ESP32

Espressif's WebRTC voice chat demo.

Full build guide

OpenAI Realtime on ESP32
No photo in the source; GitHub repository card, served from HardwareLab.

By Espressif

View original on GitHub View another source on GitHub View original on youcanjustbuildthings.openai.chatgpt.site
Posted
GitHub stars
410 (GitHub)

What you need

Difficulty
Intermediate · Our suggestion
License
Demo code: Public Domain or CC0 (source)

Parts

  • ESP32-S3-Korvo-2 (default board; others via esp_board_manager) (board)
AI model or service
OpenAI Realtime API (GA WebRTC flow, per Espressif's README)

Overview

Espressif's openai_demo connects an ESP32 board to OpenAI for real-time chat. It is part of the esp-webrtc-solution repo, uses esp_webrtc and runs by default on the ESP32-S3-Korvo-2. Espressif's README lists OPUS encoding and decoding and acoustic echo cancellation, and predefined function calls let voice commands switch a light on and off, change its colour, set the speaker volume and open a door. The demo source files are public domain or CC0.

How it works

Per Espressif's README, the demo uses the OpenAI Realtime GA WebRTC ephemeral-token flow: the board uses your API key to request a short-lived client secret from /v1/realtime/client_secrets, then exchanges SDP with /v1/realtime/calls using that token; no STUN or TURN server is used. To build, install ESP-IDF (release v5.4 or master), set Wi-Fi in main/settings.h, export OPENAI_API_KEY, pick your board with esp_board_manager (idf.py gen-bmgr-config) and flash. After boot it joins Wi-Fi and connects to OpenAI on its own; console commands start and stop the chat.

Hardware

  • ESP32-S3-Korvo-2 (default board; others via esp_board_manager)

More ESP32 builds

Like this build? Get the weekly builds

Every Thursday: the 5 best gadget hacks you can copy. Free.

Built something? Submit a build

Credits and links

This page is a short summary written by HardwareLab. Build steps, code and photos belong to the creator; follow “View original” for the full project.

Featured? Add this badge

Built this? Paste the badge into your README. It links back to this page.

Featured on HardwareLab

Markdown

[![Featured on HardwareLab](https://hardwarelab.dev/badge.svg)](https://hardwarelab.dev/projects/openai-realtime-esp32/?utm_source=badge&utm_medium=referral&utm_campaign=launch_2w)

HTML

<a href="https://hardwarelab.dev/projects/openai-realtime-esp32/?utm_source=badge&utm_medium=referral&utm_campaign=launch_2w"><img src="https://hardwarelab.dev/badge.svg" alt="Featured on HardwareLab" width="188" height="20"></a>

Related builds

All 44 in Voice Assistants & Companions