Teaching a Humanoid to Grab a Ball: A Late-Night Robotics Log

Some nights you sit down to make a robot do one small thing, and by the end you’ve accidentally built a little vision-and-learning stack. This was one of those nights. The goal was simple to say and stubborn to do: get my AiNex humanoid to find a ball on the floor, walk over, and pick it up. On its own.

Here’s the field log from the session, the dead ends, the breakthrough, and the moment it finally worked.

The deceptively hard part: two hands, one ball

The ball is a soft foam sphere, too big for the robot’s single gripper to pinch. So it has to be a two-handed grab, cradling the ball between both forearms. Easy for a toddler. Genuinely hard for a robot with short, side-mounted arms.

The first dozen attempts all failed the same way: I’d tell it to reach for the ball and its arms would swing out wide to the sides, like it wanted a hug, instead of forward and together. When it did reach down, its elbows jammed into its own knees before its hands got anywhere near the floor.

The AiNex humanoid crouched and leaning forward, reaching both arms down toward a purple ball on the floor — Mid-grab: crouched, torso pitched forward, both arms reaching for the ball.

The fix came from two realizations, tested one servo at a time on camera:

Lean the torso forward so the shoulders clear the knees, then the arms can actually swing out in front.
Squat deep so the shoulders drop to ball height, then the forearms stay horizontal and wrap the ball instead of reaching past it.

Deep squat + forward lean + an elbow “clap” to close the forearms together. That was the recipe. And then:

The AiNex humanoid standing upright holding a purple ball clamped between both forearms against its chest — He did it. Ball clamped between both forearms, lifted to the chest. (That’s my SpotMicro quadruped watching from the right.)

The best part of hardware work: there’s no unit test that feels like watching the thing actually lift the ball off the ground.

Doing it autonomously

A hand-posed grab is a puppet show. The real goal was for the robot to do the whole thing itself, so I wired up a full loop:

Find the ball by sweeping its head until it spots it.
Lock on, the head tracks the ball, keeping it centered in view (its own camera, its own decision).
Face & approach, turn the body to line up, then walk in, correcting course each step so it never veers off.
Grab, run the deep-squat cradle once it’s in range.

Plenty went sideways along the way, it walked into the ball, it stopped too far back, it turned the wrong direction and had to be re-taught its own left from right. But stitched together, it worked: the robot genuinely hunts the ball down and picks it up with no human in the loop.

Teaching it to learn what it sees

Early on, detection was pure color thresholding, “find the purple blob.” It worked in good light and fell apart the moment the room dimmed or the carpet (also purple-ish!) confused it. Chasing color thresholds is a losing game.

So late in the night I tried something better: let the robot learn the ball by example. The pipeline that emerged:

An on-device model proposes candidate objects in the frame.
Each candidate gets turned into an embedding, a compact numerical “fingerprint” of its appearance.
Those fingerprints go into a vector database. Show the robot the ball a handful of times and it stores what “ball” looks like.
To recognize anything later, it embeds what it sees and asks the database: what does this look most like?

No training runs, no labeled datasets, just a few examples and a similarity search. And it worked in the exact dim lighting that broke the color approach: it correctly picked the ball out at high confidence while rejecting the chair, the clutter, and the carpet.

The robot's own camera view with a green box labeled purple_ball on the ball and other objects rejected — What the robot sees: it recognized the ball from a handful of learned examples (green box) and correctly ignored everything else, no color rules involved.

The lovely thing about this approach is that it generalizes. Teach it a new object a few times and it can find that too. A robot with a growing visual memory, rather than a hard-coded list.

Bonus: wandering the room

Earlier in the session I also gave it a simple “explore and don’t bump into things” mode, using only the camera to tell floor from obstacle, steering toward open space. No fancy sensors, just appearance: sample the carpet in front of the feet, treat anything that doesn’t match as something to avoid.

The AiNex humanoid on the left, the purple ball in the foreground, and the SpotMicro quadruped robot on the right in a home workshop — The cast: the AiNex humanoid, the ball, and my SpotMicro quadruped sharing the workshop floor.

Where this is heading

By the end of the night the picture was clear: appearance-based, learned perception beats hand-tuned rules, and you can get surprisingly far on a small robot with a single camera and some clever software. Next up: making the pickup rock-solid, ranging the ball more precisely, and letting the robot reason about what it sees in plain language.

Not bad for “one small thing.” More soon.