Monkeys See Things That Are Not Existing With Machine Vision Brain Implant

Using brain implants, scientists triggered the perception of shapes in the visual field of monkeys. It sounds scary, but the technology could allow artificial vision in people with severe visual impairment.

The researchers used a high-definition neuroprosthesis to trigger the perception of discernible shapes, and even the movement of these shapes, in monkeys. As described in science, the device is implanted in the visual cortex of monkeys and makes them see points of light called phosphenes. These artificial dots can be displayed in meaningful patterns and possibly be made to represent objects in the real world, in what would be a remarkable advance in the treatment of blindness.

“Electrical stimulation of the visual cortex has long been proposed as an approach to restore vision in blind people, bypassing severe retinal degeneration or damage to the eye or optic nerve,” the authors write in the new article.

Indeed, this idea dates back to the 1970s, and scientists have made many attempts to stimulate the brain to produce artificial images. The problem, however, is that these earlier solutions were only able to generate a small amount of data, i.e. pixels, at a time, severely limiting their practical use. The new approach, in which scientists created a brain implant with 1,024 electrodes, was made possible by cutting-edge implant technology, namely new materials and better microelectronics. In addition, the new implant is more stable and durable than the previous versions.

The electrodes work by zapping the visual cortex with tiny bursts of electrical stimulation. This causes the perception of phosphenes, which can be caused to appear in specified regions of a person’s visual field. Or, in the case of this experiment, in the visual space of two male rhesus macaque monkeys.

With their 1024-channel neuroprosthesis, the scientists, led by Pieter Roelfsema of the Dutch Institute for Neuroscience, were able to conjure up “interpretable artificial percepts”, which were composed of several phosphenes appearing simultaneously, according to the article. The number of artificial pixels made possible by the implant is unprecedented, as previous implants did not exceed 200 electrodes.

I asked Roelfsema to describe what the monkeys saw.

“The best analogy is a matrix board,” he explained in an email. “When you turn on a light bulb, the viewer sees a point of light. It’s like a single phosphene. But you can convey meaningful information by lighting multiple bulbs as a pattern. It would be a pattern of phosphenes which conveys shape information. “

During the first phase of the experiment, Roelfsema and his colleagues trained the two monkeys to follow and identify the dot patterns passed to them in real life, as these monkeys had normal vision. These training exercises were then reproduced with the phosphenes. Roelfsema said the training process was a breeze as the team takes “baby steps” to “make sure they can always know what they’re supposed to be doing”.

The monkeys, fitted with their brain implants, were first asked to perform basic tasks, such as moving their eyes to indicate the location of the phosphenes. The monkeys were then tested on more complex tasks, such as indicating the movement of phosphenes, which was done by triggering a linear sequence of flashing phosphenes. Incredibly, the monkeys were also able to identify letters, which were produced by firing eight to 15 electrodes simultaneously.

“We trained them a lot before implanting the electrodes, for a task in which they could use their eyes,” Roelfsema said. “But when we put the prosthesis on after brain surgery, we were delighted that they immediately recognized the patterns imposed on their brains – the same patterns they had learned to recognize visually.”

In all, the monkeys were able to recognize shapes, including lines, moving dots and letters, in a promising demonstration of machine vision. Ultimately, similar technology could be used to treat people with severe eye damage or degenerative eye and optic nerve disorders, as the implants bypass visual processing in the eye and act directly on the visual cortex in the eye. the brain.

In terms of limitations, the electrodes used in the study degrade and stop working after a few years. Roelfsema said his team are currently researching another electrode material to increase the longevity of the neuroprosthesis.

The experiment followed the NIH Guide to the Care and Use of Laboratory Animals. Animal welfare “is crucial” in this kind of work, Roelfsema said, and his team made sure the monkeys are kept in “excellent conditions.” If the monkeys were uncomfortable, “they wouldn’t cooperate and participate in the tasks,” he said.

Brain implants that trigger the perception of phosphenes have already been used in humans, including one fascinating experience made in 2014 that enabled a rudimentary form of brain-to-brain communication. In addition, a co-author of the new study, Eduardo Fernandez of the Miguel Hernández University of Elche in Spain, has already tested the same type of electrode in a blind person, but with much less electrodes (so no pattern could be discerned).

The potential of machine vision is quite exciting, and it led me to imagine versions in which visually impaired people can recognize objects in their surroundings or even text from a book. The authors imagine the same thing, as illustrated in the graphic above, which shows a possible phosphenic representation of a street scene.

Basically, this solution can convey anything that can be represented by blinking pixels, which is actually a lot. Creating this visual “language” sounds like something that future scientists, linguists and semioticians must understand. It could be like Braille on steroids. And as this technology improves, meaning more pixels, these patterns might actually look more like the objects they’re meant to represent. It will be fascinating to see where this technology goes.