Claude Code Turns ESP-Mosaico Into a Working Muse Gadget

Espressif released ESP-Mosaico on Aug. 19, 2026, as a modular board aimed at coding agents, the account said. The board uses an ESP32-S31 chip, a 480-pixel square display, a speaker and microphone, and expansion slots on both sides. Its first 2 MB of flash holds a fixed Vibe Mode firmware for installing applications, recovery, logs and screenshots; holding the orange AI button at boot enters Vibe Mode, so a bad application does not brick the board. Espressif's suggested first step is not a pin diagram but a prompt telling an AI coding tool to download a repository, set up the environment and implement a Snake game.

The author followed the prompt with Claude Code. Most of the initial work went into the environment rather than the game. The workspace required ESP-IDF at a specific master commit, 7b9cc1ac, and reported version 6.2.0; a domestic component mirror lacked esp-gsp 1.5.1, so the installer downloaded it from Espressif's official API and checked the hash to build a local component library; the required GSP scene compiler 0.6.1 did not exist, so it used signed version 0.5.0, the same kind used by the workspace CI. Claude Code executed these steps, the author wrote. The game was written in Raylib Lite, with rules and rendering in separate files and the same C code running in a browser simulator and on the board. It uses a 20-by-18 grid; each apple adds one segment and one point; speed rises from 6 to 12.6 cells per second; hitting a wall or the snake ends the game, and filling the grid wins. Touch swipes steer, taps start or turn, and tapping the score bar pauses. Eight replay tests store each scenario and rerun it, requiring identical final-state hashes.

In the simulator, the author found a template bug: the blank game template listened only for touch point 0, while the browser reported the mouse as point 1, so clicks did nothing. The fix followed the first finger or mouse press. On the second day the board was connected to a Mac; it was not recognized until a data-capable cable was used. The first board run used recover to write the Vibe Mode base firmware, then system-update to install Snake. The author checked that the device ID was unchanged, the boot ID was new and the device reported healthy. On the board, the game ran at 33.3 fps without dropped frames, and touch worked.

The Muse work followed Meta's Oct. 2 open-sourcing of the Muse Gadgets device SDK. Meta released only the device side, an ESP32 firmware and a Raspberry Pi Linux SDK under Apache 2.0; the Muse assistant itself is not open source. A device must obtain an SDK token from gadgets.muse.ai and then be paired in developer mode in the Muse App. Among officially supported boards with a full interface is Waveshare's 1.75-inch round AMOLED model. The author wrote that Muse responded much faster than GrokBot and appeared to have better intent recognition; GrokBot took longer, did no device-interaction work and did not always match displayed and spoken content.

Compiling Muse firmware directly for the ESP-Mosaico ran into three blockers, according to the account. The Muse firmware accepts only ESP-IDF 6.0.1, but 6.0.1 does not support the ESP32-S31, while the board's workspace pins ESP-IDF to a 6.2 commit. Meta's official flashing method writes the bootloader and partition table and would overwrite the first 2 MB, including Vibe Mode. The board also could not reach api.muse.ai or hatch.metaaivm.com, with direct connections timing out in China. The author decided to leave Vibe Mode intact and make Muse a standard ESP-Mosaico application installed through Vibe Mode. Pairing information, Wi-Fi password and device credentials sit in a 64 KB nvs partition that is not erased by firmware changes; more than ten later installations required no re-pairing, the account said.

The code was split into three layers. Meta's SDK exposed only a few entry points, and all board-specific work went into a wrapper project; the SDK's esp-mosaico branch contained nine commits, each disabled by default so other boards behaved as before. Two SDK changes were necessary. A startup hook: Muse's app_main calls a weak symbol, muse_gadget_platform_start(), before starting Wi-Fi, Bluetooth and the interface; the wrapper provides a strong definition that starts the ESP-Iris device management service and accepts the firmware, and camera, voice and greeting functions register there. The other change disables Muse's upstream rollback, which would return the firmware if it could not reach the Muse cloud within five minutes; the board could not connect before pairing, so the switch was turned off and acceptance was left to Vibe Mode.

For networking, the author turned a Mac Studio into a side router for only the board, leaving other home devices on the ordinary router. The firmware added two configuration items, both empty by default: an external next hop, using lwIP's built-in forwarding hook while DHCP was untouched, and a DNS setting that switches to a DNS server outside the LAN when the gateway is active. Because router renewals could reset DNS, the firmware checked every 30 seconds and restored the setting. On the Mac, the author found no need to open LAN connections or DNS ports for Clash: TUN mode already handled public traffic, and dns-hijack: any:53 answered DNS queries passing through it. Only IP forwarding had to be enabled with a sysctl command, which does not survive a reboot.

Bluetooth pairing appeared stuck for a night. The no-screen Muse firmware kept advertising as MuseGadget-F46F0A, waiting for a pairing. The iPhone's Bluetooth list in Settings shows only devices such as headphones and keyboards and does not display low-energy advertisements. The ESP-Mosaico official BSP feature list also did not include Bluetooth, and none of its examples used it, so the board's Bluetooth had not been verified. To isolate Muse, the author built a diagnostic firmware, ble_diag, that advertises as MosaicoBLE-DIAG and uses the board's own Bluetooth to scan for five seconds every 15 seconds. Each round heard 34 to 37 devices, with the strongest signal at -37 dBm; nRF Connect on a phone also saw it. The board's Bluetooth transmit and receive worked, and the diagnostic firmware was left in the repository.

The author described the project as deliberately 'useless' but valuable as a demonstration that Vibe Coding is growing quickly in hardware. Without current AI agents, the author wrote, learning the hardware, embedded, software and communications knowledge needed for the work would have taken far longer; the digital world had reached a point where it needed feedback from the physical world. The account disclosed that Claude wrote the firmware code and an initial draft from session records and handoff documents, while the author set requirements, installed and verified firmware on the board, made decisions, rewrote and checked the text, and took the photos and screenshots.