Meta: Touch-sensitive robotic skin could form part of the metaverse

By Chris Stokel-Walker

A thin, replaceable skin that allows robots to “feel” could help in the construction of the metaverse, the proposed virtual future of the internet being developed by Meta (formerly Facebook) and others.

The skin, jointly developed by Meta and Carnegie Mellon University in Pennsylvania, combines a rubbery plastic less than 3 millimetres thick and studded with magnetic particles with an artificial intelligence to calibrate its sense of touch.

“If you look at how AI has advanced, we’ve made huge advances in computer vision and sound,” says Abhinav Gupta at Meta AI Research. “But conspicuously, touch has been missing from this advancement.”

When the skin touches a surface, the plastic deforms and alters the magnetic field from the embedded particles. A nearby circuit board monitors these changes and feeds them to an AI, translating them into a force and thus a sense of touch.

The technology, known as ReSkin, can measure a touch as light as 0.1 newton of force with an accuracy of 1 millimetre, and monitor changes up to 400 times a second. The associated AI requires 100 touches from a human to get enough data to understand how to translate the changes in magnetic field into a sense of touch. The team tested the system’s delicate touch on soft fruit, including grapes and blueberries.

Prior robotic skins that can “feel” have required built-in electronics to monitor the electrical changes that occur when the skin comes into contact with a surface. The ReSkin system only requires the monitoring equipment to be in close proximity. That means the material can be thinner and is less expensive to make. At present, the materials required to make each ReSkin cost less than $6 each, if manufacturing more than 100 units, says Gupta.

The team says the thinness of the skin, combined with its ability to be reused over 50,000 times before the magnetic particles need replacing, means it could have a number of applications – from robotic hands to dog shoes able to track how they walk, run and rest.

Gupta also says the improved sense of touch could be used to give a physical, haptic sensation to the virtual reality metaverse, which Meta believes is the future. “When you’re wearing these headsets, you want to generate richer and richer experiences – and key to that is haptics.”

“Overall, the device is useful, and the use case of handling soft fruit is one of the very natural immediate applications. Soft fruit and automated harvesting is a difficult job, as it’s very easy to damage the produce. It requires a very soft touch and traditional robot sensors can fall short,” says Jonathan Aitken at the University of Sheffield, UK. But Aitken points out that the device can only sense touch in one place at a time.