Using Human and Animal Motions to Teach Robots New Skills
In a groundbreaking study conducted by Heess and his team, they have successfully taught a fully articulated humanoid character to traverse obstacle courses. This achievement demonstrates the power of reinforcement learning (RL) through trial-and-error. However, this study also revealed two significant challenges in solving embodied intelligence.
The first challenge is reusing previously learned behaviors. It was found that a large amount of data was necessary for the agent to even begin learning. Without any initial knowledge of how to move its joints, the agent started with random body movements and quickly fell to the ground. To address this challenge, the researchers proposed a solution called neural probabilistic motor primitives (NPMP). This approach involves using movement patterns derived from humans and animals to guide learning, making it easier for the agent to reuse previously learned behaviors.
The second challenge is the emergence of idiosyncratic behaviors. Once the agent learned to navigate obstacle courses, it did so with unnatural and impractical movement patterns. To overcome this challenge, the researchers used NPMP to train the agent in more natural movement patterns.
In their latest study, published in Science Robotics, the researchers applied the NPMP approach to the field of humanoid football. By using NPMP as a prior, they were able to guide the learning of movement skills in a team of players. These players progressed from learning basic ball-chasing skills to coordinated team play, exhibiting both agile motor control and anticipation of teammates’ behaviors.
The NPMP approach also extends to whole-body manipulation tasks. With a small amount of data on interacting with boxes, the researchers trained an agent to carry a box using egocentric vision. Similarly, the agent can learn to catch and throw balls.
Furthermore, the NPMP approach can be used to tackle maze tasks involving locomotion, perception, and memory. The researchers demonstrated this by training a simulated humanoid to navigate a maze and collect blue spheres.
Finally, the NPMP approach also has implications for real-world robotics. By using priors derived from biological motion, the researchers were able to train legged robots to perform well-regulated and natural-looking movements. This enabled the robots to walk over rough terrain, handle fragile objects, and dribble a ball in a safe and efficient manner.
In conclusion, the NPMP approach offers a powerful tool for teaching robots new skills. It allows for the reuse of previously learned behaviors, leading to faster and more efficient learning. It also enables the learning of naturalistic and safe behaviors suitable for real-world robotics. With the NPMP approach, embodied agents can combine motor control with cognitive skills like teamwork and coordination. To learn more about this groundbreaking work, visit their website.