Skip to content
HardwareLab

Muse Gadgets SDK

Meta’s open SDKs for connecting Muse to ESP32 boards and Raspberry Pis

  1. Get an SDK token and review the Gadget SDK Terms; every gadget needs a token to pair.
  2. Pick the ESP32 device SDK for a board, or the Linux device SDK for a spare Raspberry Pi.
  3. In the Muse app, turn on Developer mode under Settings, Devices.
  4. Pair the gadget: look for devices whose names start with MuseGadget.

Steps condensed from the creator's guide.

Muse Gadgets SDK
No photo in the source; GitHub repository card, served from HardwareLab.No demo video in the repository README.

By facebookincubator

Posted
GitHub stars
2.2K (GitHub)

Build sheet

Board
ESP32 boards
Other parts
Raspberry Pi (3B+, 4, 5 or Zero 2 W) or another Linux computer with Bluetooth LE
AI model or service
Muse, through the Muse app
Difficulty
Intermediate
Accounts needed
Muse SDK token from gadgets.muse.ai

Parts list

  • ESP32-C5 DevKitC-1 (quickest start) or another listed ESP32 board [10]
  • USB cable that carries data [11]
  • Raspberry Pi 3B+, 4, 5 or Zero 2 W, or another Linux computer with Bluetooth LE [optional] [12]
  • The Muse app on your phone [13]

Prices and notes: Parts and requirements

Steps

Skill level
Intermediate
  1. Get an SDK token and review the Gadget SDK Terms
  2. Run Muse Code in esp32/, or install ESP-IDF v6.0.1 and run idf.py build and flash
  3. In the Muse app: Settings > Devices > Developer mode, then Add Device; press BOOT when the light breathes blue [18]

Full instructions: View original

Like this build? Get the weekly builds

Get the best new AI hardware builds every Thursday. One email a week, free.

Built something? Submit a build

Overview

Meta released ESP32 and Linux device SDKs for connecting Muse to displays, buttons, sensors and actuators on ESP32 and Raspberry Pi gadgets; on a Linux gadget Muse can run commands and move files, and you can hack in your own commands for sysadmin chores or a Home Assistant setup.

What it does

  • Open source devices you build yourself: program an off-the-shelf ESP32 board or set up a Raspberry Pi, then connect Muse to displays, buttons, sensors and actuators. [1]
  • ESP32 Device SDK: add a screen for images, audio in and out, or other sensors. [2]
  • Linux Device SDK: Muse can run shell commands, read and write files and report health (uptime, load, memory, disk, temperature). [3]
  • Gadgets pair with the Muse app on iOS and Android under Settings > Devices, in Developer mode. [4]

The creator's prompts

Copied word for word from the creator's repository, at the commit linked in each source.

  • First prompt for Muse Code (ESP32) [5]

    Build this firmware for my ESP32-C5 DevKitC-1 and flash it.
  • Follow-up prompts for Muse Code (ESP32) [6]

    Watch the serial log and tell me when it's ready to pair.
    
    I have a Waveshare ESP32-S3 AMOLED board. Build the UI for it.
    
    Add support for my board. It's an ESP32-S3 with 16 MB flash, a button on
    GPIO 0 and no PSRAM.
    
    Make the status light half as bright.
  • Things to ask Muse on a Linux gadget [7]

    What's using all the disk space on my Pi?
    
    Install Home Assistant on my Pi and tell me how to open it.
    
    Every morning at 7, check if my Pi's backups ran and tell me if they didn't.

Use-case ideas

  • Let Muse handle sysadmin chores or your Home Assistant setup from a spare Raspberry Pi or Linux box. [8]
  • Have programs on the machine message you through Muse, e.g. "The garage door has been open for an hour." [9]

Parts and requirements

PartSpec / notesNeededPrice
ESP32-C5 DevKitC-1 (quickest start) or another listed ESP32 board [10]Works as-is with its status light and BOOT buttonRequired
USB cable that carries data [11]Required
Raspberry Pi 3B+, 4, 5 or Zero 2 W, or another Linux computer with Bluetooth LE [12]Raspberry Pi OS Bullseye or later, Debian 11+ or Ubuntu 22.04+, 32- or 64-bitOptional
The Muse app on your phone [13]Required
  • Computer for setup: macOS or Linux computer
  • Muse SDK token from gadgets.muse.ai: required [14]
  • Requirements source: [15]

How it works

Device SDKs and firmware connect Muse to your displays, buttons, sensors and actuators; on a Linux gadget Muse can run commands and move files, and you can add your own commands.

LayerComponentRuns on
AppMuse app (gadgets pair with it) [4]iOS or Android
ESP32 firmwareESP32 Device SDK firmware, built with ESP-IDF v6.0.1 [16]ESP32
Linux servicemusegadget service in /opt/musegadget [17]The machine

Setup effort

Skill level
Intermediate

Quickest path: install Muse Code, clone the repo and ask it to build and flash; the README also gives the manual ESP-IDF build. [18]

Limits and caveats

  • Every gadget needs an SDK token to pair; the README says to review the Gadget SDK Terms first. [14]
  • Muse gets the same access to the machine as the account you install it for, including sudo if that account has it. [19]
  • Community devices: pairing has no manufacturer verification and can’t prevent an active man-in-the-middle attack, so set it up on a trusted network. [20]
  • The README warns that tinkering may brick boards or void warranties. [21]

Why it's interesting

Muse Gadgets are open-source devices you build yourself and pair with the Muse app on iOS or Android. The repository holds two SDKs: an ESP32 device SDK for adding a screen, audio in and out or other sensors to an off-the-shelf board, and a Linux device SDK that turns a spare Raspberry Pi or Linux box into a gadget, with your own commands for sysadmin chores or Home Assistant. Setup has a few required steps: get an SDK token and review the Gadget SDK Terms, since every gadget needs a token to pair, then turn on Developer mode in the app under Settings, Devices and look for devices prefixed MuseGadget. Each directory includes an AGENTS.md for coding agents. The README calls it built "just for fun" and warns that tinkering may brick boards or void warranties. It is Apache 2.0 licensed.

Source: github.com/facebookincubator/muse-gadget-sdk#readme

Hardware

  • ESP32 boards
  • Raspberry Pi (3B+, 4, 5 or Zero 2 W) or another Linux computer with Bluetooth LE

More ESP32 buildsMore Raspberry Pi builds

Credits and links

Sources

  1. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/README.md?plain=1#L26-L29
  2. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/README.md?plain=1#L37
  3. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/linux/README.md?plain=1#L17-L86
  4. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/README.md?plain=1#L45-L47
  5. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/esp32/README.md?plain=1#L42-L63
  6. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/esp32/README.md?plain=1#L65-L76
  7. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/linux/README.md?plain=1#L93-L97
  8. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/README.md?plain=1#L38
  9. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/linux/README.md?plain=1#L101-L105
  10. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/esp32/README.md?plain=1#L31-L33
  11. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/esp32/README.md?plain=1#L34
  12. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/linux/README.md?plain=1#L32-L35
  13. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/esp32/README.md?plain=1#L40
  14. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/README.md?plain=1#L40-L43
  15. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/esp32/README.md?plain=1#L29-L40
  16. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/esp32/README.md?plain=1#L17-L88
  17. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/linux/README.md?plain=1#L53-L56
  18. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/esp32/README.md?plain=1#L36-L151
  19. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/linux/README.md?plain=1#L88-L89
  20. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/linux/README.md?plain=1#L74-L77
  21. github.com/facebookincubator/muse-gadget-sdk/blob/f60347aeb851a1fe9438cfc388721776aa24f275/README.md?plain=1#L31-L33

This page is a short summary written by HardwareLab. Build steps, code and photos belong to the creator; follow “View original” for the full project.

Featured? Add this badge

Built this? Paste the badge into your README. It links back to this page.

Featured on HardwareLab

Markdown

[![Featured on HardwareLab](https://hardwarelab.dev/badge.svg)](https://hardwarelab.dev/projects/muse-gadget-sdk/?utm_source=badge&utm_medium=referral&utm_campaign=launch_2w)

HTML

<a href="https://hardwarelab.dev/projects/muse-gadget-sdk/?utm_source=badge&utm_medium=referral&utm_campaign=launch_2w"><img src="https://hardwarelab.dev/badge.svg" alt="Featured on HardwareLab" width="188" height="20"></a>