28.9M-parameter LLM on an ESP32-S3

Overview

Uses Gemma-style Per-Layer Embeddings: the 25M-parameter embedding table stays in flash, so a 28.9M-parameter model generates text at 9.88 tokens/s on the ESP32-S3 itself. Covered by Tom’s Hardware.

What it does

• Runs a 28.9 million parameter language model on the ESP32-S3 itself, with nothing sent to a server, showing the text on a small screen at 9.88 tokens per second. [1]
• Fits because most of the model, a 25M-parameter embedding table, stays in flash and only about 450 bytes are read per token (Per-Layer Embeddings, from Google’s Gemma 3n). [2]
• The model is 14.9 MB at 4-bit; activations and norms sit in 512 KB SRAM, the core and output head in 8 MB PSRAM. [3]
• The same chip also runs part of the fruit fly connectome (48,311 neurons) escaping a spider on screen. [4]

The creator's config

Copied word for word from the creator's repository, at the commit linked in each source.

• FQBN used by deploy.sh [5]

  Copy

  FQBN='esp32:esp32:esp32s3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=cdc,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=16M,PartitionScheme=custom,PSRAM=opi,DebugLevel=info'

• OLED wiring (display.h) [6]

  Copy

  //   DISPLAY_OLED_I2C (default) -- 128x64 I2C mono OLED. 4 wires:
  //     GND->GND, VCC->3V3, SCL->GPIO46, SDA->GPIO18. Set OLED_CONTROLLER to match
  //     the panel: 1.3" is usually SH1106, 0.96" usually SSD1306. Wrong choice ->
  //     top row correct, rest noise (SH1106-as-SSD1306) -> flip the define.

Use-case ideas

• Run the other published models on the same board: Barista (espresso question answering) or the fly connectome, with the same two commands. [7]
• A reference for Per-Layer Embeddings on microcontroller memory: the reusable architecture is in src/ and the training and quantization code in research/tinystories/. [8]

Parts and requirements

• Network: Only for fetch_model.sh, which downloads the model; deploy.sh downloads no model
• Model files from Hugging Face (slvDev/esp32-ai-tinystories), checked against a pinned SHA-256 by fetch_model.sh
• Requirements source: [13]

Setup effort

Skill level

Advanced

Two commands: one downloads and verifies the model, the other compiles and flashes it. [17]

Why it's interesting

The trick here is memory layout, not raw speed. The ESP32-S3 has 512 KB of SRAM, 8 MB of PSRAM and 16 MB of flash, so activations and norms stay in SRAM, the dense core and output head sit in PSRAM, and a 25-million-parameter embedding table stays in flash, where only about 450 bytes are pulled per token. The README credits this Per-Layer Embeddings idea to Google's Gemma 3n. The result is 9.88 tokens per second end to end, a 14.9 MB model at 4-bit, and no network connection at all. It is also clear about limits: trained on TinyStories, it writes short simple stories and will not answer questions, follow instructions or know facts. The same chip and repo also run the fruit fly connectome demo listed separately on HardwareLab.

Source: github.com/slvDev/esp32-ai#readme

Hardware

• ESP32-S3 N16R8
• OLED

More ESP32 builds

Credits and links

• slvDev (Viacheslav Sierbov): Creator

• Method and measurements (RESULTS.md)  ↗
• TinyStories model on Hugging Face  ↗
• Tom’s Hardware coverage (27 July 2026)  ↗

Sources

1. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L10-L14
2. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L38-L46
3. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L22-L35
4. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L16-L18
5. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/scripts/deploy.sh#L187
6. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/firmware/esp32_tinystories/display.h#L4-L7
7. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L108-L124
8. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L138-L144
9. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L25
10. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/firmware/esp32_tinystories/display.h#L1-L8
11. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/scripts/deploy.sh#L190-L191
12. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L130-L133
13. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L111-L133
14. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L22-L28
15. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L48-L54
16. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/firmware/esp32_tinystories/README.md?plain=1#L14-L18
17. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L119-L125
18. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L58-L63
19. github.com/slvDev/esp32-ai/blob/52ebcbbd905270592c1d415efd1128d04d0fa365/README.md?plain=1#L124-L125