A confused commuter is about to step out the door when they discover they’ve misplaced their keys and must hunt through piles of belongings to retrieve them. They hope they could figure out which pile was hiding the keys as they search through the mess.
MIT researchers have developed a robotic device that can accomplish just that. RFusion is a robotic arm equipped with a camera and radio frequency (RF) antenna attached to its gripper. It combines antenna signals with visual information from the camera to locate and retrieve an item, even if it is buried behind a mound and completely hidden from view.
The researchers’ RFusion prototype is based on RFID tags, which are inexpensive, battery-free tags that may be attached to an item and reflect signals sent by an antenna. Because RF signals can pass through most surfaces (even the mound of dirty laundry that may be covering the keys), RFusion can find a tagged item within a pile.
Using machine learning, the robotic arm automatically zeroes in on the exact location of the object, moves the things on top of it, grasps the object, and checks that it has picked up the correct thing. Because the camera, antenna, robotic arm, and AI are all fully integrated, RFusion can operate in any situation without any additional setup.
While finding lost keys is useful, RFusion has the potential to have many broader applications in the future, such as sorting through piles to fulfil orders in a warehouse, identifying and installing components in an auto manufacturing plant, or assisting an elderly individual with daily tasks in the home, though the current prototype isn’t quite fast enough for these uses yet.
RFusion starts looking for an object by using its antenna, which bounces signals off the RFID tag (similar to sunlight reflected off a mirror) to determine a spherical area in which the tag is placed. It uses that sphere with the camera input to narrow down the location of the object. For example, the object cannot be found in an empty section of a table.
However, after the robot has a general idea of where the item is, it would need to swing its arm about the room taking further measurements to determine the exact location, which would be slow and inefficient.
The researcher applied reinforcement learning to construct a neural network that can optimize the robot’s path to the target. Reinforcement learning involves training an algorithm through trial and error with a reward scheme.
“This is also how our brain learns. We get rewarded from our teachers, from our parents, from a computer game, etc. The same thing happens in reinforcement learning. We let the agent make mistakes or do something right and then we punish or reward the network. This is how the network learns something that is really hard for it to model,” said the author of the study.
As of now, it has slow-working but the researchers assure to increase the speed of the system so it can move smoothly, rather than stopping periodically to take measurements. This would enable RFusion to be deployed in a fast-paced manufacturing or warehouse setting.
Beyond its potential industrial uses, a system like this could even be incorporated into future smart homes to assist people with any number of household tasks, according to the author of the study.
Image Credit: Getty