Programmable Robots Helping People With Dementia Find Misplaced Objects
Engineers from the University of Waterloo have developed an innovative way to program robots to assist people with dementia in finding lost items like medicine, glasses, and phones. While this technology currently targets dementia patients, it has the potential to benefit anyone who has ever searched for something they can’t find.
According to Dr. Ali Ayub, a post-doctoral fellow in electrical and computer engineering, the long-term impact of this technology is very promising. He believes that users can not only have a companion robot but also a personalized one that promotes independence.
The team of engineers decided to focus on dementia patients due to the increasing number of individuals affected by the condition. Dementia limits brain function, leading to memory loss, confusion, and disability. Many patients repeatedly forget the whereabouts of everyday items, negatively impacting their quality of life and placing additional burdens on caregivers.
The engineers aimed to create a companion robot with its own episodic memory, which they believed could be a game-changer in such situations. Using artificial intelligence, they successfully developed a unique form of artificial memory.
How the Technology Works
The engineers started by using a Fetch mobile manipulator robot equipped with a camera for perceiving its surroundings. They then programmed the robot with an object-detection algorithm, enabling it to detect, track, and log specific objects in its camera view through stored video. By distinguishing one object from another, the robot can record the date and time an object enters or leaves its field of vision.
To make the system user-friendly, the researchers created a graphical interface that allows users to select objects they want the robot to track. By simply typing the objects’ names, users can search for them through a smartphone app or computer. The robot will then indicate the last time and place it observed the specified object.
Tests have shown that this system is extremely accurate, and even though some dementia patients may find it challenging, caregivers can easily make use of it. Moving forward, the engineers plan to conduct user studies with both individuals without disabilities and those affected by dementia.
A recent paper on this project, titled “Where is my phone? Towards developing an episodic memory model for companion robots to track users’ salient objects,” was presented at the 2023 ACM/IEEE International Conference on Human-Robot Interaction.