MIT researchers show how a robotic hand reorients 2,000 different objects in order to grasp them. At just one year old, babies are more dexterous than industrial robots. They grasp objects of different shapes, colors and sizes, but also learn to put down and sort objects. Engineers have been trying for a long time to transfer such properties to a robotic hand, so far without any resounding success.
Researchers at the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT) have now made a breakthrough. They have developed a system that can reorient more than 2,000 different objects with the robotic hand pointing both up and down. This ability to manipulate objects, from a mug to a can of tuna to a box of cookies, could help the hand quickly place objects in specific ways and places. The researchers hope that what they have learned can be transferred to completely unknown objects.
Known issues with robotic hands
For background: "In industry, a gripper with parallel jaws is the most commonly used, partly because of its ease of control, but also because robots are not able to use many of the tools that we use in daily life," says Tao Chen , PhD student at MIT CSAIL. "Even using pliers is difficult because they don't move a handle back and forth adeptly."
Reorienting objects in the robot's hand is considered a difficult problem because there are many motors to control and contacts between the fingers and the objects change frequently. The problem gets even trickier when the hand is pointing downwards. The robot not only has to move the object, but also has to overcome gravity to keep an item from falling.
Engineers at MIT found that a comparatively simple approach can solve complex robotic hand problems. They used a reinforcement learning algorithm, also called reinforcement learning. With the machine learning method, a system tries to obtain the largest possible virtual reward based on actions. However, it does not know which action in a situation with a specific object leads to the best possible goal.
The particular approach was a 'teacher-student' training method. The "teacher" network was trained using information about objects and the robot itself that is readily available in simulations but not in the real world. These included, for example, the position of the fingertips in a coordinate system or the speed of the object.
In order to ensure that a robot can also work outside of the simulation, the knowledge of the "teacher" is brought into line with experimentally collected data from the "student". The MIT research group worked with depth images from cameras. Added to this were the object position and the joint positions of the robot. A "gravity curriculum" was also used, in which the robot first learns skills in a hypothetical zero-gravity environment, and then slowly adapts the controls to normal gravity.
Robots grab objects with up to 100% success
A single controller, the robot's "brain," could reorient a large number of objects that it had never seen before, without knowledge of the shape. Many small, circular objects such as apples, tennis balls, or marbles had near 100% success rates when repositioned up and down by hand. Unsurprisingly, the lowest success rates occurred with more complex objects such as a spoon, screwdriver, or scissors. They were in the order of about 30%. Since the success rates varied depending on the shape of the object, the researchers hope that in the future the system could be trained based on common object shapes to better recognize other objects.
An intelligent robotic hand in medicine soon?
Other applications of the model beyond industry are also conceivable. Back in August 2021, MIT engineers presented a soft, lightweight, and potentially inexpensive neuroprosthetic hand for humans. Amputees who tested the artificial limb performed daily activities like closing a suitcase, pouring a juice carton and petting a cat just as well and in some cases better than those with stiffer neuroprostheses. The new design is also surprisingly durable. But it would also benefit from innovative robotic hand algorithms.