The robot apocalypse is nigh. Boston Dynamics’ robots are doing backflips and opening doors for their friends. Oh, and these 7-foot-long robot arms can lift 500 pounds each, which means they could theoretically crush, like, six humans at once.
The robot apocalypse is also laughable. Watch a robot attempt a task it hasn’t been explicitly trained to do, and it’ll fall flat on its face or just give up and catch on fire. And teaching a robot to do something new is exhausting, requiring line after line of code and joystick tutorials in say, picking up an apple.
But new research out of UC Berkeley is making learning way easier on both the human and machine: By drawing on prior experience, a humanoid-ish robot called PR2 can watch a human pick up an apple and drop it in a bowl, then do the same itself in one try, even if it’s never seen an apple before. It’s not the most complex of tasks, but it’s a big step toward making machines rapidly adapt to our needs, fruit-related or otherwise.
Consider the toothbrush. You know how to brush your teeth because your parents showed you how—put water and paste on the bristles and put the thing in your mouth and scrub and then spit. You could then draw on that experience to learn how to floss. You know where your teeth are, and you know there’s gaps between them, and that you have to use an instrument to clean them. Same principle, but kinda different.
To teach a traditional robot to brush its teeth and floss, you’d have to program two sets of distinct commands—it can’t use the context of prior experience like we can. “A lot of machine learning systems have focused on learning completely from scratch,” says Chelsea Finn, a machine learning researcher at UC Berkeley. “While that is very valuable, that means we don't bake in any knowledge. Essentially, these systems are starting with a blank mind every time they learn every single task if they want to learn.”
Finn’s system instead provides the humanoid-ish robot with valuable experience. “We collected videos of humans doing a number of different tasks,” she says. “We collected demonstrations of robots doing the same tasks via teleoperation, and we trained it such that after it sees a video of a human doing one thing, the robot can learn to imitate that thing as well.”
Take a look at the GIF below. A human demonstrates by pushing the container, not the box of tissues, toward the robot’s left arm, as the robot observes through its camera. When presented with the container and the box, only arranged differently, the robot can recognize the correct object and make a similar sweeping motion, pushing the container with its right arm into its left arm. It’s drawing from “experience”—how it’s been teleoperated previously to manipulate various objects on a table, combined with watching videos of humans doing the same. Thus the machine can generalize to manipulate novel objects.
“One of the really nice things about this approach is you don't need to very precisely track the human hand and the objects in the scene,” says Finn. “You really just need to infer what the human was doing and the goal of the task, then have the robot do that.” Precisely tracking the human hand, you see, is prone to failure—parts of the hand can be occluded and things can move too fast for a machine to read in detail. “It's much more challenging than just trying to infer what the human was doing, irrespective of their precise hand pose.”
It’s a robot being less robotic and more human. When you learned to brush your teeth, you didn’t mirror every single move your parent made, brushing the top molars first before moving to the bottom molars and then the front teeth. You inferred, taking the general goal of scrubbing each tooth and then taking your own path. That meant first of all that it was a simpler task to learn, and second of all it gave you context for taking some of the principles of toothbrushing and applying them to flossing. It’s about flexibility, not hard-coding behavior.
Which will be pivotal for the advanced robots that will soon labor in our homes. I mean, do you want to have to teach a robot to manipulate every object in your home? “Part of the hope with this project is we can make it very easy for the average person to show robots what to do,” says Finn. “It takes a lot of effort to joystick around, and if we can just show robots what to do it would be much easier to have robots learning from humans in very natural environments.”
To do things like chores, for instance. To that end, researchers at MIT are working on a similar system that teaches robots in a simulation to do certain household tasks, like making a cup of coffee. A set of commands produces a video of a humanoid grabbing a mug and using the coffee machine and such. The researchers are working on getting this to run in reverse—show the system a video of someone doing chores on YouTube and it could not only identify what’s going on, but learn from it. Finn, too, wants her system to eventually learn from more “unconstrained” videos (read: not in a lab) like you’d find on YouTube.
Let’s just be sure to keep the machines away from the comment section. Wouldn’t want to give them a reason to start the robot apocalypse.
- How Tesla is building cars in its parking lot
- Copying nature to make fantastical machines
- PHOTO ESSAY: Take a closer look at these tiny worlds
- How technology helped me cheat dyslexia
- The next generation of Wi-Fi security will save you from yourself
- Hungry for even more deep-dives on your next favorite topic? Sign up for the Backchannel newsletter
