Esp32 Llm
A real LLM running on an ESP32.
By DaveBben
- Est. cost
- Cost unknown
- Posted
- GitHub stars
- 643 (GitHub)
Overview
Proof that a language model can run on an ESP32: a 260K-parameter TinyLlamas model trained on TinyStories generating text at about 19 tokens per second.
How it works
It uses llama2.c on an ESP32-S3FH4R2, whose 2 MB of PSRAM fits the roughly 1 MB model. Speed-ups include using both cores, ESP-DSP SIMD dot-product functions, a 240 MHz CPU and 80 MHz PSRAM, and a larger instruction cache. It builds with the ESP-IDF toolchain.
Hardware
- ESP32-S3 with PSRAM
This page is a short summary written by HardwareLab. Build steps, code and photos belong to the creator; follow “View original” for the full project.
More in Local LLM Rigs & Edge Agents
See all 17Esp32 Iphone Agent
A $7 ESP32 lets an AI drive a real iPhone via AirPlay and USB HID.
- ESP32 (USB HID)
- iPhone
IntermediateCost unknown1.4KPocket Tank
A 14M-parameter LLM keeps a virtual fish tank alive on an $8 ESP32.
- ESP32-S3
AdvancedCost unknown207Needle 2 Esp32
45M-parameter LLM offline on an ESP32-S3 turning English into valid commands.
- ESP32-S3
AdvancedCost unknown193