Humans have a unique ability to understand others’ goals, desires, and beliefs, allowing us to anticipate their actions. For example, if someone is taking bread out of the toaster, we know they’ll need a plate. If someone is sweeping up leaves, we’ll grab the green trash can. This skill, known as “theory of mind,” is natural for humans but challenging for robots. However, if robots are to become truly collaborative helpers in manufacturing and everyday life, they need to learn these same abilities.
Teaching Robots to Predict Human Preferences
In a recent study, USC Viterbi computer science researchers aimed to teach robots how to predict human preferences in assembly tasks. The researchers wanted to enable robots to assist with everything from building satellites to setting tables. The study was a finalist for the best paper award at the ACM/IEEE International Conference on Human-Robot Interaction (HRI).
Lead author Heramb Nemlekar, a USC computer science PhD student, worked under the supervision of Stefanos Nikolaidis, an assistant professor of computer science. According to Nemlekar, when working with people, robots need to constantly guess what the person will do next. For example, if a robot predicts that a person will need a screwdriver for the next assembly task, it can fetch the screwdriver ahead of time, saving the person’s time and helping them finish the assembly faster.
However, predicting human behavior is challenging because different people have different preferences. Some individuals prefer to start with the most difficult parts, while others prefer to start with the easiest parts. Most existing techniques require people to demonstrate their assembly preferences to the robot, which can be time-consuming and defeat the purpose. The researchers aimed to find a more efficient solution.
Making Predictions Based on Similarities
The researchers discovered that there are similarities in how individuals assemble different products. For example, if someone starts with the hardest part while building an Ikea sofa, they are likely to use the same approach when putting together a baby’s crib. Rather than showing the robot their preferences in a complex task, the researchers came up with a small assembly task called a “canonical” task.
Participants were asked to assemble parts of a simple model airplane, such as the wings, tail, and propeller. The robot observed the humans completing the task using a camera placed directly above the assembly area. By using AprilTags attached to the parts, the system was able to detect the parts operated by the humans.
The researchers used machine learning to analyze the sequence of actions performed by the humans in the canonical task. Based on this analysis, the robot could predict a person’s preferences in a larger assembly task. For example, if the robot observed that a person likes to start with the easiest part in the small assembly, it would predict that they would do the same in the larger assembly.
Building Trust and Future Applications
In a user study, the researchers found that their system was able to predict human actions with approximately 82% accuracy. Nemlekar hopes that their research will make it easier for people to show robots their preferences. By helping individuals in their preferred way, robots can reduce their work, save time, and even build trust with humans.
This technology has various potential applications. For example, imagine assembling furniture at home with the help of a robot that knows your assembly preferences. It could provide you with the necessary tools and parts ahead of time, making the process easier. This technology could also improve efficiency and safety in industrial settings where workers assemble products on a large scale.
The researchers emphasize that their goal is not to replace humans on the factory floor but to enhance safety and productivity. They plan to develop a method for automatically designing canonical tasks for different types of assembly tasks. They also aim to evaluate the benefits of learning human preferences from short tasks and predicting their actions in more complex tasks, such as personal assistance in homes.
Nikolaidis believes that a robot that can quickly learn our preferences could have a significant impact on our daily lives. It could assist with preparing meals, rearranging furniture, or even doing house repairs.