The magic touch: bringing sensory feedback to brain-controlled prosthetics

Bensmaia and his lab, as well as other UChicago colleagues, is collaborating with the University of Pittsburgh and the University of Pittsburgh Medical Center to advance the understanding and implementation of prosthetics controlled via brain-computer interfaces. The UChicago research team’s portion of the project, which was recently awarded a $3.4m grant from the National Institutes of Health, covers the somatosensory aspects – the system that transmits sensory information from the body’s nerve fibres to the brain’s somatosensory cortex, which sits next to the movement-oriented motor cortex.

“What I’ve spent most of my time in my lab doing is trying to understand how is it that the nervous system encodes this information about objects we touch,” Bensmaia says. “What we’re trying to do in these neural prosthetics applications is leverage what we’ve learned about how the intact nervous system does that and try to reproduce that by electrically stimulating different structures.”

For patients who have lost an arm and are living with a prosthetic, the team’s method would be to electrically stimulate the nerve with electrodes, while for tetraplegic patients, the electrode array would be implanted in the somatosensory cortex. In either case, sensors on a bionic hand – disembodied, in the case of tetraplegic users – generate information that complex algorithms can convert into patterns of electrical stimulation to the nerve or brain in real-time to convey sensory feedback or elicit sensations.

“We developed a model last year, which was published in PNAS [Proceedings of the National Academy of Sciences of the United States of America], that allows us to reproduce with striking precision how the nerve would respond to any kind of object interacted with,” Bensmaia says.

Indeed, in 2016 Bensmaia and colleagues from Chicago and Pittsburgh demonstrated a version of the system in a 28-year-old tetraplegic subject, Nathan Copeland, who was able to distinguish touches on different fingers of a robotic arm based on the stimulated sensations he felt in his own non-functional hand. “Sometimes it feels electrical and sometimes it’s pressure, but for the most part I can tell most of the fingers with definite precision,” said Copeland in 2016. “It feels like my fingers are getting touched or pushed.”

“Without thinking, he can tell which part of the prosthetic hand is touching the object. That’s super-useful,” Bensmaia says. “The more pressure he exerts on an object, the stronger the evoked sensation is. So he has a pretty good idea of where he’s touching the object and how much force he’s exerting on it.”

The progress on sensory feedback is encouraging but limited, and there are a host of questions for Bensmaia and other researchers to answer before the true technological revolution can begin. In terms of the nerve, Bensmaia says the challenge is a purely technological bottleneck.

“If we had perfect technology that allowed us to individually stimulate each nerve fibre selectively, we could restore the sense of touch perfectly. But of course, the technology doesn’t allow us to do that. You have this array with up to a couple of hundred electrodes, and with this relatively small channel count, we’re trying to stimulate 10,000-15,000 fibres, which each have their own idiosyncratic response in an intact organism.

“We’re not going to be able to do each of these 12,000 or 15,000 fibres individually. What we’re able to stimulate is tens or hundreds of them at a time. How do tens or hundreds of fibres respond? That we can use our model to figure out, then we use the electrical stimulus to try and reproduce that, or evoke that to the extent that we can.”

The brain, of course, is considerably more complex. Understanding of how the brain operates is improving every year, but gaps in knowledge are still present and need to be unpicked to improve the decoding algorithms that translate the brain’s language of touch. Given that the hand and fingers primarily used to exert force rather than move in a space, leveraging sensory feedback to improve dexterity will be a knotty problem to unravel. The naturalism of evoked sensation will need to be refined, too.

“I don’t think we’re going to be able to tell the difference between good silk and cheap silk with a prosthetic hand anytime soon,” says Bensmaia. “But being able to, for instance, convey more precisely exactly when contact is established with an object, about how much force is exerted on it, in a much more realistic and naturalistic way are our more short-term goals.”

As well as advances in wireless and making the most sophisticated bionic hands rugged enough to serve outside of a lab environment, improvements to the underlying brain-computer interface that powers brain-controlled movement for tetraplegics will also be necessary. The most important leap, Bensmaia contends, will be on the biocompatibility and physical flexibility of the implanted electrode arrays.

“These neural implants, you’re basically sticking a bed of nails in the brain – and the brain isn’t happy,” he says. “It fights the device, and long story short, these devices don’t last long enough. They last a few years, and you don’t want to be doing a craniotomy every few years to put a fresh one in. So I think the main challenge is to create implants that are going to last forever.”

Bensmaia and colleagues at UChicago are now gearing up for more human trials of the sensory feedback system. UChicago Medicine neurosurgeon Peter Warnke has been brought onboard to implant the devices, and Bensmaia says the team is “actively searching for subjects”, although a small pool of eligible patients and the early-stage nature of the research could make this process slow going.

Looking to the future, Bensmaia is hopeful that more sophisticated prosthetic devices are on the way, especially with greater private sector involvement, as seen with Elon Musk’s brain-computer interface firm Neuralink and Deka’s impressive LUKE Arm.

“Scientists are just interested in science,” Bensmaia says. “Once they’ve made a device sort of work, they get bored with it and work on something else. Whereas in the private sector, people invest to make money and therefore they really want to bring it to market. So that’s promising.”

It’s the beginning of a long and winding road towards brain-controlled prosthetic hands that can leverage sensory feedback to approximate the hand’s natural dexterity. But the unmet needs of amputees, and tetraplegics especially, are many, and restoring a sense of touch and movement to those who have had to live without them is a powerful motivating force.

“If you talk to amputees who now have bionic hands that they can feel through, that almost makes them feel whole again,” says Bensmaia. “Because they now feel the bionic hand, they sometimes describe the bionic hand as ‘their’ hand; it’s endowed with a sense of touch, and when they see the hand touching something they can feel it.”