For weeks the robot’s whole life was walking up to a ball and throwing it. That was the mission. This week I gave him a completely different job, and it is the one I am most excited about: he looks around a room and learns what is in it, just by being shown.
No training step
The usual way to make a robot recognize objects is to collect a few hundred labeled photos of each thing and train a model overnight. I did not want that. I wanted to hold something up, have him look at it once, and from then on know it. So that is what he does now. There is no dataset and no training run. You put an object in front of him and he remembers it from that single look.
How he actually does it
When his detector locks onto something, he crops out just that object and runs it through a small network that turns the shape into a list of numbers, a fingerprint of what it looks like. Then he compares that fingerprint against everything he has seen before. He keeps two memories. A long-term one lives on my desktop in a vector database and holds everything he has ever learned. A short-term one lives on the robot itself and holds the last several dozen things he saw this session, so he recognizes something he glanced at seconds ago even before it has been written to long-term memory.
If the fingerprint matches something he knows, he says so out loud. If it is new, he stores it on the spot and calls it out as a new thing. To put a human-readable name on each object the first time, he asks a vision model running on my workstation what it is looking at, and files the answer with the fingerprint.
Watching him do it
I laid a pile of stuff on the table and let him pan across it. He worked through the room naming things: a keyboard, a basket, a spider-man figure, a screwdriver, a rubber duck. The ones he had never seen went into memory as he found them, sixteen new objects in one scan. The ones he already knew he simply recognized and skipped, and some of those recognitions were strong. A stuffed animal came back at ninety three percent confidence. A toy car at ninety two. That is him being genuinely sure, not guessing.
The part that still gets me is that this is one-shot. Show him a soda can once and he owns it. Lay out ten things and he calls them out one by one. He does not have to do anything but look and see, and he keeps what he sees.
Why it matters
Throwing a ball was a control problem. This is a memory problem, and memory is what turns a machine that reacts into one that has a history. Everything he learns now compounds. The more I show him, the more of his world he can name, and none of it needed me to sit and label a single image.