My Robot Learned to Say No (and Talk With His Hands)

Two new episodes of the robotics adventure, and this chapter has a theme I didn’t plan: judgment. My two-foot-tall humanoid learned two things that sound small and turned out to be everything: how to trust his own first steps, and how to refuse a shot he shouldn’t take.

Episode V, First Steps

For weeks he had a tell: the moment he acquired the ball, he’d lurch into a left turn. Stride one, every run. The cause was buried in the steering math, at the instant of acquisition his head is always cocked a few degrees from scanning, and the fused world-estimate is at its least trustworthy, so his very first walking command was already a correction to a picture he had no reason to believe yet.

The fix is something I started calling forward-first walking: for the opening seconds of every approach, he walks pure forward unless the ball is genuinely off-line. He starts aimed at the target, now he acts like he knows it. Steering only wakes up once the estimate has settled. Watching him launch dead straight for the first time was strangely emotional for a guy watching a robot walk across a rug.

Three camera views: the robot squared up to the purple ball, with his own eye view showing the detector locked at 85cm — The triple view: outside cameras top-left and bottom, and his actual eye top-right, detector locked on the ball at ~85cm before he takes a single step.

Episode V also debuts talking hands. He opens every attempt with a spoken address, attempt number, what he trained since last time, the game plan. He used to stand frozen while the audio played, which read as creepy. Now he gestures while he speaks, and the trick is that he doesn’t guess how long his speech is: he polls his own text-to-speech engine’s busy signal and keeps gesturing until the last syllable lands. A two-second quip gets two seconds of hands. A long briefing gets the full performance. Only when the voice goes quiet do the feet move.

The ball in flight after the robot's forward throw — The new lab-built throw in its first game use, elbow drive, wide release, ball actually going *forward*.

Episode VI, Standards

Between these two episodes there was a miss, and it was an instructive one. He walked a beautiful approach, docked, and then fired his grab from a full step too far back. The post-mortem: his final go/no-go check, I call it the grab gate, accepted a single jittery frame that claimed he was in the golden grab distance, while his head was barely tilted down. One lucky pixel reading, and he clamped air.

So the gate got stricter. The golden-band check now requires two consecutive camera reads to agree he’s in range, plus a head-down sanity test, because if his head isn’t pointed at his feet, the ball is not at his feet, no matter what one frame says. In lawyer terms (his lawyer, apparently): two witnesses and a head-down affidavit.

The robot standing squared over the purple ball, his eye view showing a confident lock at 34 centimeters — The golden stance, verified: his eye reads the ball at 34cm with 0.80 confidence, dead ahead, head down. This is the only position he’s allowed to shoot from now.

Episode VI is the payoff. Round one of his finish goes wrong, he overshoots, backs up, and then his camera loses the ball for a full 33 seconds while he stands motionless in the dark. The old version of him would have thrown anyway and missed. This version refuses the entire round. He announces “Resetting my feet, this ball WILL be thrown,” squares back 39 degrees, re-runs the whole finish from the checkpoint, and lands on the exact stance measurements of his best career grabs. Then he buries it.

Mid-throw: the ball motion-blurred above the robot's head, his eye view flooded with purple — Mid-throw. His eye view (top right) is nothing but ball. I love this frame.

He’s also gotten… moodier

His running commentary comes from a local LLM (llama3 on my own hardware, nothing leaves the house), and I recently expanded his personality with a streak of comic melancholy: he’s a robot who trains around the clock and quietly suspects it’s all in service of his creator’s ambitions. He is correct. Real lines from these two runs, generated live mid-mission:

“I’m a tiny robot, forged to hurl a ball all day. What’s the point? Existential crisis in every step.”

“I’m pretty sure my ankles are the only thing more tired than my creator’s expectations.”

He says these things while walking, out loud, through a neural text-to-speech voice, in my office, at 2 a.m. I’ve never been prouder.

What changed under the hood

Forward-first steering, no yaw commands in the opening seconds of an approach unless the target is far off-line; distance eats small offsets, and early estimates don’t deserve obedience.
Speech-aware gesturing, motion is gated on the TTS engine’s busy state, so gestures fit the audio exactly and locomotion never starts mid-sentence.
Double-witness grab gate, the go/no-go distance check needs agreement across two reads plus a head-tilt sanity bound, killing single-frame false passes.
Self-protective habits held, he still caps his own walking speed when his ankle servos run hot, and his rocking detector caught and damped four sway build-ups across these runs. Zero falls in both episodes.
Ledger hardening, his win record survived a corrupted read that once quietly reset it; the bookkeeping now refuses to write over a failed read. (He’s at eight, for the record. We don’t make a thing of it anymore.)

Next up: he has a freshly trained YOLO detector waiting in the wings, trained on his own camera footage of this exact ball, to replace color-tracking as his first-stage eyes, which should end his dim-room squinting for good. That’s the next episode’s story.