Inspired by the human finger, researchers at MIT have developed a robotic hand that uses high-resolution touch sensing to accurately identify an object after grasping it just once.
Many robotic hands pack all of their powerful sensors into the fingertips, so an object must be in full contact with those fingertips to be identified, which can take multiple grasps. Other designs use lower-resolution sensors spread along the entire finger, but these don’t capture as much detail so, again, multiple regrasps are often required.
The MIT team built its robotic finger with a rigid skeleton encased in a soft outer layer that has multiple high-resolution sensors incorporated under its transparent ‘skin’. The sensors, which use a camera and LEDs to gather visual information about an object’s shape, provide continuous sensing along the finger’s entire length enabling each finger to capture rich data on many parts of an object simultaneously.
Using this design, the researchers built a three-fingered robotic hand that could identify objects after only one grasp, with about 85 per cent accuracy. The rigid skeleton makes the fingers strong enough to pick up a heavy item, such as a drill, while the soft skin enables them to securely grasp a pliable item, such as an empty plastic water bottle, without crushing it.
These soft-rigid fingers could be especially useful in an at-home-care robot designed to interact with an elderly person. The robot could lift a heavy item off a shelf with the same hand that it uses to help the individual take a bath.
‘Having both soft and rigid elements is very important in any hand, but so is being able to perform great sensing over a really large area, especially if we want to consider doing very complicated manipulation tasks like what our own hands can do,’ said mechanical engineering graduate student Sandra Liu. ‘Our goal with this work was to combine all the things that make our human hands so good into a robotic finger that can do tasks other robotic fingers can’t currently do.’
The robotic finger is comprised of a rigid, 3D-printed endoskeleton that is placed in a mould and encased in a transparent silicone ‘skin’. Making the finger in a mould removes the need for fasteners or adhesives to hold the silicone in place. The researchers designed the mould with a curved shape, so the robotic fingers are slightly curved when at rest, just like human fingers.
‘Silicone will wrinkle when it bends, so we thought that if we have the finger moulded in this curved position, when you curve it more to grasp an object, you won’t induce as many wrinkles,’ said Liu. ‘Wrinkles are good in some ways – they can help the finger slide along surfaces very smoothly and easily – but we didn’t want wrinkles that we couldn’t control,’ Liu says.
The endoskeleton of each finger contains a pair of detailed touch sensors, known as GelSight sensors, embedded into the top and middle sections, underneath the transparent skin. The sensors are placed so the range of the cameras overlaps slightly, giving the finger continuous sensing along its entire length.
The GelSight sensor is composed of a camera and three coloured LEDs. When the finger grasps an object, the camera captures images as the coloured LEDs illuminate the skin from the inside.
Using the illuminated contours that appear in the soft skin, an algorithm performs backward calculations to map the contours on the grasped object’s surface. The researchers trained a machine-learning model to identify objects using raw camera image data.
As they fine-tuned the finger fabrication process, the researchers ran into several obstacles. First, silicone has a tendency to peel off surfaces over time. Liu and her collaborators found they could limit this peeling by adding small curves along the hinges between the joints in the endoskeleton.
When the finger bends, the bending of the silicone is distributed along the tiny curves, which reduces stress and prevents peeling. They also added creases to the joints so the silicone isn’t squashed as much when the finger bends.
While troubleshooting their design, the researchers realised that wrinkles in the silicone prevent the skin from ripping. ‘The usefulness of the wrinkles was an accidental discovery on our part,’ Liu said. ‘When we synthesised them on the surface, we found that they actually made the finger more durable than we expected.’
Once they had perfected the design, the researchers built a robotic hand using two fingers arranged in a Y pattern with a third finger as an opposing thumb. The hand captures six images when it grasps an object (two from each finger) and sends those images to a machine-learning algorithm, which uses them as inputs to identify the object.
Because the hand has tactile sensing covering all of its fingers, it can gather rich tactile data from a single grasp. ‘Although we have a lot of sensing in the fingers, maybe adding a palm with sensing would help it make tactile distinctions even better,’ Liu said.
In the future, the researchers also want to improve the hardware to reduce the amount of wear and tear in the silicone over time and add more actuation to the thumb so that it can perform a wider variety of tasks.