Desktop companion robot  /  Raspberry Pi 4

He dances
with you.

Theo's face lit by OpenRGB: left eye magenta, right eye cyan, mouth amber

Actual frame from the renderer. Each eye and the mouth are separately addressable.

Theo already had a face, a voice and a camera. Party mode gives him a fourth sense: he joins your lighting rig as a device, watches you nod, works out the tempo, and keeps the groove going when you stop.

00  —  See it running

Forty seconds of party mode, on the actual hardware.

Recorded off the Pi with OpenRGB driving the face. Sound is part of it — the music is what the lighting visualiser is reacting to, and the beat you hear is the beat Theo is recovering from colour.

↓ Download MP4 Open full size ↗ 1920×1080  /  0:40  /  15 MB  /  H.264 + AAC

What to look for

Everything below is how each of those was built, and what measurement said about it.

01  —  He is an OpenRGB device

Your lighting software drives his face.

E1.31 (sACN) is how lighting software talks to network fixtures — plain UDP with a 126-byte header, so no library, no dependency. Theo appears in OpenRGB as a real device: a 3 × 10 matrix whose three columns are his left eye, right eye and mouth.

OpenRGBAny effect, incl. the audio visualiser
E1.31 / UDPUniverse 1, 30 LEDs, 90 channels
robot‑lightsDecodes, recovers beat, republishes
RendererColour + bounce, 62 fps

A separate process on its own core. The render loop is never allowed to touch the network.

Face with magenta and cyan eyes
Sweep — each part its own colour
Face with green and violet eyes
Effects travel across the face
Face lifted upward mid-beat
Beat — whole face lifts 16 px

OpenRGB sends no audio. But when its visualiser is running the beat is already in the colour stream, so the bounce is recovered from brightness rising above its own recent average — not invented. A static colour produces no bounce, which is the honest answer rather than a bug.

02  —  He watches you

A nod barely moves your face. It moves your pitch.

The first version tracked the bounding box around your head and hardly reacted. Measured live, a nod moves the box centre through 2.6% of the frame while head pitch swings whole degrees. Reading pose instead — as deviation from a slowly adapting baseline, because a camera on a desk sees you at a permanent +30° — produced 8.5× the response from the same nod.

8.5×
More movement per nod
0.026
Frame the bbox moved
85 px
Face travel, eyes + lean
82%
Iris still on screen

He holds eye contact while dancing — the social gaze model that makes him look away during conversation is suppressed. And because he has no neck, a sustained offset migrates out of his eyes into a whole-face lean over two seconds, the way a person's head takes over from their eyes.

Face leaning to one side
Lean — eyes hand off to the face
Wide open delighted face
delighted — 235 px of visible eye
Round wide gleeful face
gleeful — alternates every 6 s

03  —  He keeps the beat

Learn the tempo. Carry it when you stop.

Beats come from upward zero crossings of the nod signal — a peak is one noisy sample, a sign change is not. Once three intervals agree he locks a tempo and phase-locks an oscillator to you. When you ease off, that oscillator fades in underneath your own movement, already in phase, so there is no seam to see.

YOU STOP HERE
Your nodding Theo, phase-locked
1.3%
Worst tempo error
55–170
BPM range tracked
4.1
Beats to lock on
11°
Phase error, steady

He hands control back over six beats rather than snapping to you — he already has your tempo, so there is no hurry. Every 20–40 seconds he takes a turn of his own even while you are still dancing. And he keeps re-learning your tempo, depth and phase the entire time he is the one leading.

04  —  What measurement found

Every fix here was a wrong assumption.

Only the background colour reached him OpenRGB colours a bar by its height, so a device one pixel tall can only ever receive the bottom of every bar. Not a decoding bug — the foreground colours were never sent. Fixed by becoming a matrix. Linear 3 LED → Matrix 3 × 10
The face flew apart while dancing The spring integrator was stable only while dt × ω < 1.5. Party mode's stiffer springs pushed it to 2.30, so a single slow frame overshot to 3× the target and rang. Latent in every mode; party mode made it loud enough to see. One hitched frame: 3.14× overshoot → 0.99
A stutter like a mistuned TV The panel runs at 62.29 Hz, not 60. And rotating the landscape face onto the portrait panel cost 65% of the frame while allocating 768 KB sixty-two times a second. 58.1–61.9 fps (spread 3.8) → 62.0–62.4 (spread 0.4)
It never actually locked onto a tempo Perception publishes ~7.4 times a second, so a 150 BPM nod gets three samples per cycle and crossings are routinely missed. Missed beats are now folded, the candidate period is searched for rather than assumed, ties go to the fundamental, and crossing times are interpolated. Never locked → 1.3% error, 55–170 BPM

05  —  The rest of the time

When the lights are off, his face is an instrument.

Outside party mode the same pixels report how he is doing: hue carries temperature, colour drains toward red if frames slip, and an undervoltage event browns the whole face out. While OpenRGB is driving, all of that stands down — it owns the colour outright.

Face in normal teal
52 °C — resting
Face in warm sand colour
63 °C — working hard
Face dimmed in a brownout
Brownout — undervoltage