Agent link protocol

Status: v0. Goal: any agent, anywhere, drives Molty with no laptop in the loop. Built to be swapped module by module. Background and design rationale: ARCHITECTURE.md.

Molty (home Wi-Fi) ──outbound WSS──▶ molty-relay (Cloudflare Worker + Durable Object)
                                       ├─ /mcp   remote MCP (HTTPS)   ◀── any MCP client (Claude, Grok, …)
                                       ├─ /v1/*  REST                 ◀── molty CLI --relay, scripts
                                       ├─ voice  TTS/STT provider     (Workers AI first; swappable)
                                       └─ (later) OpenClaw node adapter

Modules and seams (each swappable)

Module	Where	Swappable seam
Relay core (rooms, auth, pairing, routing, jobs)	`relay/src/core/`	pure TS, no Cloudflare imports
Relay host	`relay/src/worker.ts` (+ DO)	another host (Node server) wraps the same core
Agent surfaces	`src/surfaces/mcp.ts`, `src/surfaces/rest.ts`	add OpenClaw node adapter beside them
Voice provider	`src/voice/*.ts` → `interface VoiceProvider { tts(text, opts) → PCM16@16k; stt(PCM16@16k) → text }`	workers-ai · openai · none
Device link client	`firmware/main/link/`	transport only; commands come from the dispatcher
Device command dispatcher	`main/link/link_cmd.c`	ONE registry used by both the WSS link and (later) local HTTP
Pairing UI	static: the 4-digit code alone, held as one centred token in the fluid text target (no ORP accent), until accept/reject/timeout (`link_ui_show_pair`); agent name is shown by the console via `GET /api/link`	a dedicated `pair` creature state is the planned upgrade

Identities and secrets

device_id = molty- + 10 random chars [a-z2-9] (e.g. molty-k7m2w9c4ya). Public, generated together with the key on first boot and persisted in NVS. It is unrelated to the MAC; unpredictable IDs prevent guessing a new device’s TOFU room before it connects.
device_key = 32 random bytes, generated with the ID on first boot, kept in NVS. The device sends it on every connect. The relay stores sha256(device_key) on first claim of a device_id (trust on first use); later connects must match. Console “forget link” sends {"t":"forget"} on the authenticated socket, waits up to 3 s for close 4410, then wipes the NVS identity and relay URL. The next identity has a fresh ID and key. If offline or the relay does not acknowledge, local erasure still proceeds; remote revocation is confirmed only by 4410. Older key-only installations migrate to a new random ID/key.
link_code = 8 chars [A-Z2-9], generated by the device, shown in the console’s “Connect an agent” instructions, rotates every 15 min and after each successful pairing. Stops strangers from spamming pair requests: a pair request must carry a valid link_code.
agent_token = 32 random bytes (base64url), issued by the relay after on-device approval. Relay stores only its hash, scoped to one device_id, with the agent’s name. Revocable.

Pairing flow

Console (USB or molty.local) → Wi-Fi set → “Connect an agent” → the user pastes ONE message into any agent:
```
Hi! Please connect yourself to my Molty.

Instructions: <relay https origin>/skill.md
Device: molty-xxxxxxxxxx
Code: K7M2W9C4

I'll tap Molty to accept you.
```
GET <relay>/skill.md (also /skill, text/markdown) is the agent-facing guide: what Molty is, pairing over plain HTTPS, the REST command table, the optional MCP route, etiquette. All URLs in it are built from the request origin (relay/src/surfaces/skill.ts).
The agent calls POST /v1/pair {device_id, link_code, agent_name} (or the molty_pair MCP tool, which is the only tool usable without a token). The request long-polls ≤ 120 s.
Relay → device: pair.request {req, agent_name, code} where code is 4 digits.
Device shows the 4-digit code statically (one held token in the fluid) until decided (touch tap or BOOT button = accept; PWR = reject; 110 s = timeout, so the device decides just before the relay’s 120 s window closes). The console polls GET /api/link (~1 Hz) and advances on pair / last_pair.
Device → relay: pair.result {req, ok}; failures include reason:"timeout" or reason:"rejected". On ok the relay mints agent_token and returns it to the waiting request: {agent_token, device_id}. Device rotates link_code and reports the new code on the same connection. Successful pair.result also carries the new link_code, allowing the relay to replace the old code within its approval transaction.

Device ↔ relay wire (WS/WSS `/v1/device?id=<device_id>`)

First frame from device: {"t":"hello","id":"molty-k7m2w9c4ya","key":"<b64 device_key>","fw":"...","caps":["fluid","say","listen","show","mood","creature"],"link_code":"..."} → relay answers {"t":"welcome"} or closes with 4401.

Configure a relay origin (wss://host or ws://LAN-IP:8787 for wrangler dev); the device appends /v1/device?id=<device_id>. WSS verifies certificates/hostnames. Close codes: 4400 malformed, 4401 bad key, 4409 replaced by a newer socket, 4410 forgotten. A 4401 retries every 5 minutes and sets auth_failed:true in local GET /api/link until welcome or local reconfiguration. Other disconnects use 1–60 s backoff. Local status also includes close_code for the current/last attempt.

Text frames are JSON:

relay→device {"t":"cmd","id":"c17","name":"show","args":{...}}
device→relay {"t":"res","id":"c17","ok":true,"result":{...}} or {"ok":false,"error":"code"}
device→relay {"t":"evt","name":"touch|state|listen.level|...","data":{...}}
pair.request / pair.result as above; {"t":"ping"}/{"t":"pong"} every 20 s.
device→relay on each rotation: {"t":"link_code","code":"...","link_code":"..."}. code is the requested spelling; link_code is a compatibility alias required by the built relay’s current room.ts handler. Rotation does not reconnect the socket.
device→relay {"t":"forget"}; relay revokes its room’s bindings/tokens/jobs, sends {"t":"forgotten"}, then closes 4410. Local POST /api/link/forget returns 202 pending while the device performs this bounded exchange before erasing its identity.

Binary frames carry audio: <JSON header>\n<payload> (header ≤ 512 B, payload ≤ 16 KB). Header: {"s":"<stream id>","seq":N,"kind":"pcm16","rate":16000,"end":bool}.

Stream IDs allow 1–128 characters; omitted end means false. Header-only binary frames (without a newline/payload) are accepted. PCM payloads contain whole int16 samples. Text-frame limit is 64 KiB, independently of the audio-frame limit.
say: relay sends cmd say {job,text,ms,rate:16000,voice?}; device starts speaking and text immediately, then acknowledges. Relay sends 320 ms PCM frames (s = job id), with three lead frames and paced delivery thereafter. After the last frame leaves the ring and I2S DMA, device returns to idle and sends {"t":"evt","name":"say.done","data":{"job":"..."}}. Failed/cancelled playback does not emit successful completion. Reader timing remains approximate (no word timestamps).
listen: device streams mic PCM up; relay runs STT on end.

Moods (mood cmd, and the implicit listening/speaking around audio jobs) are moods of the fluid creature’s loaded .molty creature: the device applies {"state":"creature","mood":"<mood>"} (blob when no creature is loaded). During say the text state wins over the creature view.

Commands (v0)

name	args	result	notes
`status`	—	fw, uptime, creature, state, mood, fps, rssi
`state`	`{state, ...}`	creature state JSON	passthrough to `/api/creature/state` semantics
`mood`	`{mood}`	—	idle listening thinking speaking happy sleepy surprised
`show`	`{text, wpm?}`	—	speed reader in the fluid
`say`	`{text, voice?}`	`{job}` immediately	relay does TTS + streams; device sets `speaking`, shows text in sync
`listen`	`{ms≤10000}`	`{job}` immediately	device sets `listening`, streams mic; relay STT → job result `{text}`
`volume`	`{volume 0..21}`	—
`imu`	—	snapshot
`diag`	`{timeline? 0..180}`	`/api/diag` JSON	frame histograms, task CPU %, heap, stalls, net; `timeline` = seconds of 1 Hz rows
`log`	`{since?, max? 1024..32768}`	`{next, text}`	device log ring tail (ESP_LOG + `EVT` lines); pass `next` back as `since`
`creature.file`	(later)	—	upload `.molty` via relay

Agent-facing calls are async-friendly (agent runtimes commonly cap a tool call at ~10 s / 64 KiB): say/listen return a job id; molty_wait {job, timeout_s≤8} returns the result or pending. Short calls answer inline.

Agent surfaces

Remote MCP https://<relay>/mcp (streamable HTTP), Authorization: Bearer <agent_token>. Tools: molty_pair, molty_status, molty_mood, molty_show, molty_say, molty_listen, molty_wait, molty_state, molty_volume, molty_imu. Same names/semantics as the local stdio MCP in cli/ so agents don’t care which transport they’re on.
REST POST /v1/devices/<id>/cmd {name,args}, GET /v1/jobs/<job>, same bearer.
Skill page GET /skill.md (and /skill), no auth: the agent’s self-setup guide (see Pairing flow step 1). Kept in sync with commands.ts/MCP tools by test/skill.test.ts.
CLI molty --relay <url> --token <t> ..., or molty pair once and the saved link (see CLI.md).

REST envelopes

Agreed between the relay (relay/) and the Go CLI relay transport. The bearer is agent_token = <device_id>.<base64url(32 bytes)>; the prefix only routes.

Call	Success	Failure
`POST /v1/pair {device_id, link_code, agent_name}` (no token; long-polls ≤ 120 s)	`200 {ok:true, agent_token, device_id, agent_name, code}`	`{ok:false, error}`: `bad_link_code` 403, `rejected` 403, `timeout` 408, `device_offline` / `pair_busy` 409
same with `wait_s` < 120, or `{device_id, pair_id}` to resume	`202 {ok:true, status:"pending", pair_id, code, expires_in_s}` while undecided	as above
`POST /v1/devices/<id>/cmd {name, args}`	`200 {ok:true, result:{...}}`; `say`/`listen` → `{ok:true, result:{job}}`	`{ok:false, error:"code"}`: 401 bad/missing bearer, 400 `invalid_args`/`unknown_command`, 409 `device_offline`/`audio_busy`, 504 `device_timeout`, 502 device-reported codes
`GET /v1/jobs/<job>[?wait_s=0..8]`	`200 {job, kind, status:"queued"\|"running"\|"done"\|"failed", result?, error?}`; say done `{ms, text}`, listen done `{text, duration_ms, peak, rms}`	401, 404 `not_found`
`DELETE /v1/token`	`200 {ok:true, result:{revoked:true}}`	401

Device-local console API (same Wi-Fi, no auth; served by the firmware, used by console/). One spelling per field — relay_url is the ws/wss origin the device dials; agents/CLI use its http(s) twin (ws://h → http://h, wss://h → https://h).

Call	Success	Failure
`GET /api/link`	`200 {device_id, link_code, relay_url, pair, last_pair, connected, auth_failed, close_code}`	`500` link unavailable
`POST /api/link {relay_url}` (any Content-Type)	same body as GET; saved to NVS, reconnects	`400` not a `ws://`/`wss://` origin
`POST /api/link/forget`	`202 {ok:true, pending:true}`	`500`

pair: null, or {agent_name, code, expires_in_s} for a request currently awaiting the tap. last_pair: null until the first decision since boot (RAM only), then {seq, agent_name, ok, reason} for the most recent pair.result the device sent; seq increments per decision; reason is null when ok, else "rejected" or "timeout" (immediate rejections — e.g. while audio is busy — count too). A pending request dropped by a relay disconnect clears pair without a decision. Example while pending, then after:

{"device_id":"molty-k7m2w9c4ya","link_code":"K7M2W9C4","relay_url":"wss://relay.example","pair":{"agent_name":"Claude Code","code":"4821","expires_in_s":97},"last_pair":null,"connected":true,"auth_failed":false,"close_code":0}
{"device_id":"molty-k7m2w9c4ya","link_code":"Q3XH8ND2","relay_url":"wss://relay.example","pair":null,"last_pair":{"seq":1,"agent_name":"Claude Code","ok":true,"reason":null},"connected":true,"auth_failed":false,"close_code":0}

state cmd: empty args = read; {state:{...}} or flat fields = patch (sent to the device flat). MCP molty_wait maps job status to pending (queued/running), the job result plus status:"done", or a tool error for failed.

Default relay

Each component has exactly one default relay URL, all set to the same value (molty-relay.alicenet.workers.dev, the project’s hosted relay). Change it in these three places to point a build at your own relay (see relay/README.md to deploy one):

Component	Where	Used when
firmware	`CONFIG_MOLTY_DEFAULT_RELAY` in `firmware/main/Kconfig.projbuild` (`wss://…`)	no relay URL saved in NVS (`POST /api/link {relay_url}` overrides)
console	`DEFAULT_RELAY` in `console/src/ui/agent-link.ts` (`https://…`)	the device has not reported a `relay_url` yet
CLI	`DefaultRelay` in `cli/internal/molty/link.go` (`https://…`)	`molty pair` without `--relay` / `MOLTY_RELAY`

Explicitly out of v0 (later)

Auth on the device’s local HTTP API · OpenClaw node adapter · streaming STT/TTS · end-to-end encryption past the relay · multi-device accounts · rate limits beyond basics.