[ad_1]
Today's artificial intelligence systems, including artificial neural networks largely inspired by neurons and nervous system connections, perform wonderfully well-known tasks. They also tend to require a lot of computing power and large amounts of training data. All this makes them perfect for playing chess or Go, to detect if there is a car in an image, to differentiate representations of cats and dogs. "But they are rather pathetic at composing music or writing news," said Konrad Kording, a neuroscientist in computer science at the University of Pennsylvania. "They have a lot of trouble reasoning significantly in the world."
Quanta Magazine
Sure
To overcome these limitations, some research groups are turning to the brain to find new ideas. But a handful of them choose what may seem at first an unlikely starting point: the sense of smell or the olfaction. Scientists who are trying to better understand how organisms deal with chemical information have discovered coding strategies that seem particularly relevant to AI problems. In addition, olfactory circuits have striking similarities with more complex brain regions that have been interesting in the quest for better machines.
Computer scientists are now beginning to explore these results in machine learning contexts.
Flukes and revolutions
The modern machine learning techniques used today were designed at least in part to mimic the structure of the visual system, which relies on hierarchical extraction of information. When the visual cortex receives sensory data, it first selects small, well-defined features: edges, textures, colors, which implies spatial mapping. Neuroscientists David Hubel and Torsten Wiesel discovered in the 1950s and 1960s that specific neurons in the visual system corresponded to the equivalent of specific pixel positions in the retina, a discovery for which they won a Nobel Prize.
When visual information is transmitted through layers of cortical neurons, the details on edges, textures, and colors come together to form increasingly abstract representations: the object is a human face, and the object is a human face. Face identity is Jane. . Each layer of the network helps the organization achieve this goal.
Deep neural networks have been designed to function in the same hierarchical way, resulting in a revolution in machine learning and AI research. To teach these networks to recognize objects such as faces, they receive thousands of samples. The system reinforces or weakens the connections between its artificial neurons to more accurately determine that a given collection of pixels forms the most abstract motif of a face. With enough samples, he can recognize faces in new images and contexts he has never seen before.
Researchers have been very successful with these networks, not only in image classification, but also in speech recognition, language translation and other machine learning applications. Still, "I like to think of deep nets as freight trains," said Charles Delahunt, a researcher at the Computational Neuroscience Center at the University of Washington. "They are very powerful, as long as you have relatively flat terrain, where you can set up tracks and have a huge infrastructure. But we know that biological systems do not need all of this – that they can handle difficult problems that the deepest networks can not solve right now.
Take a topical issue in AI: autonomous cars. As a car navigates in a new real-time environment – a constantly changing environment, this is full of noises and ambiguities – visual-based deep learning techniques may not be enough. Vision-based methods may not be the right one. This vision was such a dominant insight that it was in part incidental, "a stroke of historic luck," said Adam Marblestone, a biophysicist at the Massachusetts Institute of Technology. This is the system that scientists have best understood, with clear applications for image-based machine learning tasks.
Saket Navlakha, a computer scientist at the Salk Institute, has developed algorithms based on the olfactory circuit of flies, hoping to improve machine learning techniques for similarity searches and simulations. novelty detection tasks.
Salk Institute
But "every kind of stimulus is not treated the same way," said Saket Navlakha, a computer scientist at the Salk Institute for Biological Studies in California. "Vision and smell are very different types of signals, for example. … There may be different strategies for handling different types of data. I think that there could be many more lessons beyond studying the operation of the visual system.
He and others are beginning to show that the olfactory circuits of insects can retain some of these lessons. Olfaction research began only in the 1990s, when biologists Linda Buck and Richard Axel, both from Columbia University at the time, discovered odor receptor genes. Since then, however, the olfactory system has become particularly well characterized and can be easily studied in flies and other insects. According to some scientists, visual systems do not make it possible to study general computer challenges.
"We are working on olfaction because it is a finite system that you can characterize relatively completely," Delahunt said. "You have a chance to fight."
"People can already do such things with vision," said Michael Schmuker, a computer neuroscientist at the University of Hertfordshire in England. "Maybe we can also do fantastic things with the olfaction."
Random and sparse networks
Olfaction differs from vision on many fronts. Odors are not structured. They have no edge; they are not objects that can be grouped in space. It is a mixture of compositions and varying concentrations, and it is difficult to classify them in a similar or different way. It is therefore not always easy to know which features should attract attention.
These odors are analyzed by a shallow, three-layer network, much less complex than the visual cortex. Neurons in olfactory areas randomly sample all the space of the receiver, not specific regions in a hierarchy. They use what Charles Stevens, a neurobiologist at the Salk Institute, calls an "antimap". In a mapped system like the visual cortex, the position of a neuron reveals something about the type of information that it carries. But in the antimap of the olfactory cortex, this is not the case. Instead, the information is distributed throughout the system, and reading this data involves sampling a minimum number of neurons. An antimap is obtained through what is called a sparse representation of information in a larger space.
Take the olfactory circuit of the fruit fly: 50 projection neurons receive an input of receptors sensitive to different molecules. A single odor will excite many different neurons, and each neuron represents a variety of odors. It is a waste of information, superimposed representations, which is at this stage represented in a space of 50 dimensions. The information is then randomly projected on 2,000 so-called Kenyon cells, which code particular odors. (In mammals, the cells of the so-called piriform cortex handle this.) This constitutes a 40-fold expansion in the dimension, which facilitates the distinction between odors and neuronal reactions.
Lucy Reading-Ikkanda / Quanta Magazine
"Let's say you have 1,000 people and put them in a room and try to organize them for leisure," Navlakha said. "Of course, in this cluttered space, you might find a way to structure these people into groups. But now, let's say you spread them out on a football field. You have all this extra space to play and structure your data. "
Once the olfactory circuit of the fly has done this, it must find a way to identify distinct odors with neurons that do not overlap. This is done by "sparsifying" the data. Only about 100 of the 2,000 Kenyon cells – 5% – are very active in response to given odors (the less active cells are silenced), giving each a unique label.
In summary, while traditional deep networks (resuming their visual cues) constantly change the strength of their connections as they "learn," the olfactory system does not generally seem to train by adjusting the connections between its neurons. and Kenyon. cells.
Thomas Nowotny, professor of computer science at the University of Sussex, has highlighted parallels between the olfactory system and a class of models called support vector machines. Since then, he has continued to develop a better understanding of the functioning of the olfaction, always keeping in mind the potential applications of AI.
Courtesy of Thomas Nowotny
In the early 2000s, while researchers were studying olfaction, they developed algorithms to determine how random integration and low density in higher dimensions contributed to computer efficiency. A pair of scientists, Thomas Nowotny of the University of Sussex in England and Ramón Huerta of the University of California at San Diego, have even established links with another type of machine learning model, called a carrier vector machine . They argued that the ways in which natural and artificial systems process information, using random organization and the expansion of dimensionality to effectively represent complex data, were formally equivalent. AI and evolution had converged, independently, on the same solution.
Intrigued by this link, Nowotny and his colleagues continue to explore the interface between olfaction and machine learning, looking for a deeper connection between the two. In 2009, they showed that an olfactory model based on insects, originally created to recognize odors, could also recognize handwritten figures. In addition, removing the majority of its neurons – to mimic how brain cells die and are not replaced – has not affected its performance too much. "Parts of the system could fail, but the system as a whole would continue to function," Nowotny said. He plans to implement this type of hardware in something like a Mars rover, which has to work in harsh conditions.
But for a while, little work has been done to follow up on these discoveries – until very recently, when some scientists began to revisit the biological structure of the olfaction to better understand how to improve problems specific machine learning.
Wired knowledge and rapid learning
Delahunt and his colleagues repeated the same kind of experiment conducted by Nowotny, using the olfactory system of ringworm as a base and comparing it to traditional models of machine learning. Since there are less than 20 samples, the moth-based model has better recognized the handwritten figures, but when there was more training data, the other models proved to be much more more solid and precise. "Machine learning methods can give very accurate classifiers, with tons of data, while the insect model is very efficient at performing raw classification very quickly," said Delahunt.
The olaction seems to work best in terms of speed of learning because, in this case, "learning" is no longer about looking for features and optimal representations for the task at hand. Instead, it is reduced to recognizing which random functions are useful and which are not. "If you can train with one click, it would be a lot more beautiful, right?" Said Fei Peng, a biologist at Southern Medical University in China.
Indeed, the olfactory strategy is almost like making basic and primitive concepts in the model, much like a general understanding of the world is apparently embedded in our brain. The structure itself is then capable of simple and innate tasks without instruction.
Charles Delahunt (left) and J. Nathan Kutz (right) at the University of Washington integrated the structure of the olfactory butterfly network into a machine learning context, creating what they call "insect cyborgs" ".
Gabriella Wolff
One of the most striking examples is that of the Navlakha laboratory last year. With Stevens and Sanjoy Dasgupta, a computer scientist at the University of California at San Diego, he wanted to find an olfaction-inspired way to research on the basis of similarity. Just as YouTube can generate a list of online videos for users based on what they watch, organizations must be able to make quick and accurate comparisons when identifying odors. A fly can learn early that it should approach the smell of a ripe banana and avoid the smell of vinegar, but its environment is complex and full of noise – it will never suffer the same smell. When it detects a new odor, the fly must determine what odors have been felt before, so that it can remember the proper behavioral response to apply.
Navlakha created an olfactory similarity search algorithm and applied it to sets of image data. He and his team found that their algorithm worked better, and sometimes two to three times, than traditional non-biological methods involving only size reduction. (In these more standard techniques, objects were compared by focusing on a few basic features or dimensions.) The fly-based approach also "used a lower order of magnitude to achieve similar levels of precision," he said. said Navlakha. "So he gained in cost or performance."
Nowotny, Navlakha and Delahunt showed that an essentially untrained network could already be useful for classification calculations and similar tasks. Setting up such an encoding scheme leaves the system in place to facilitate later learning. It could be used in tasks involving navigation or memory, for example – situations in which changing conditions (eg, clogged paths) may not leave the system with a lot of time to learn or many examples.
Peng and his colleagues have begun researching this, creating an olfactory model to make decisions on how to navigate a familiar route from a series of superimposed images.
In the work being reviewed, Navlakha has applied a similar method based on olfaction for the detection of novelty, the recognition of something as new even after being exposed to thousands of years. similar objects in the past.
And Nowotny examines how the olfactory system processes mixtures. He already sees potential applications for other machine learning challenges. For example, organisms perceive certain smells as a single scent and others as a combination: a person can take dozens of chemicals and know that she has smelled a rose, or smell the same number of chemicals in a bakery neighbor. some croissants. Nowotny and his team have found that separable odors are not perceived at the same time; the coffee and croissant aromas are rather alternately processed.
This idea could also be useful for artificial intelligence. The cocktail problem, for example, refers to the difficulty of separating many conversations in a noisy environment. Given several speakers in a room, an AI could solve this problem by cutting off audio signals in very small time windows. If the system recognized the sound coming from a speaker, it could try to delete the entries of others. By alternating like this, the network could unravel the conversations.
Enter the cyborgs of insects
In an article published last month on the arxiv.org scientific preprint site, Delahunt and his colleague at the University of Washington, J. Nathan Kutz, pushed this type of research further by creating what they call " insect cyborgs ". Ringworm-based model like the inputs of an automatic learning algorithm, and saw improvements in the system's ability to classify images. "This gives the machine learning algorithm a much more powerful hardware with which to work," said Delahunt. "The brain of the butterfly of the night draws whatever type of structure it is and having this type of different structure helps the machine learning algorithm."
Some researchers now also hope to use olfaction studies to understand how several forms of learning can be coordinated in deeper networks. "But for now, we have covered only a few things," Peng said. "I'm not sure how to improve deep learning systems yet."
Tatyana Sharpee, neurobiologist at the Salk Institute, recently discovered that odors can be mapped in a hyperbolic space. She wonders if this idea could shed light on the best way to structure the input data used in deep learning systems.
Salk Institute
A starting point could be not only the implementation of an architecture based on the olfactions, but also the definition of the inputs of the system. In an article just published by Science Advances, a team led by Tatyana Sharpee of the Salk Institute was looking for a way to describe smells. The images are more or less similar depending on the distances between their pixels in a kind of visual space. But this kind of distance does not apply to olfaction. Nor can structural correlations provide reliable indications: odors with similar chemical structures can be perceived as very different, and odors with very different chemical structures can be perceived as similar.
Sharpee and his colleagues have instead defined odor molecules based on how often they are found in nature (for the purpose of their study, they examined how often the molecules co-present in fruit samples and other substances) . They then created a map by comparing the odorous molecules if they tended to co-activate, and farther away if they did so infrequently. They discovered that, just as cities are mapped on a sphere (the Earth), odor molecules fit into a hyperbolic space, a sphere with a negative curvature that looks like a saddle.
Sharpee hypothesized that introducing inputs with a hyperbolic structure into machine learning algorithms could help classify less structured objects. "There is a starting point for deep learning that data should be used in a Euclidean metric," she said. "I would say that one could try to turn this metric into a hyperbolic metric." Such a structure could perhaps further optimize deep learning systems.
A common denominator
At the present time, much of this work remains theoretical. The work of Navlakha and Delahunt needs to be expanded to much more difficult machine learning problems to determine if models inspired by olfactions can make a difference. "I think all of this is still emerging," Nowotny said. "We'll see where it goes."
What gives hope to researchers is the striking resemblance between the structure of the olfactory system and the other brain regions of many species, particularly the hippocampus, involved in memory and navigation, and the cerebellum, responsible for control engine. Olfaction is an ancient system that goes back to chemosensitivity in bacteria and is used in any form by all organisms to explore their environment.
"It seems to be closer to the point of origin of the evolution of all things we would call the cortex in general," Marblestone said. The olfaction could provide a common denominator for learning. "The system offers us a truly preserved architecture, used for a wide variety of organisms," said Ashok Litwin-Kumar, neuroscientist at Columbia. "There must be something fundamental there that is good for learning."
The olfactory circuit could serve as a bridge to understand the more complex algorithms and learning calculations used by the hippocampus and the cerebellum – and to determine how to apply this knowledge to AI. Researchers have already begun to turn to cognitive processes such as attention and different forms of memory, in the hope that they could offer ways to improve architectures and skills. current mechanisms of machine learning. But the olfaction could offer a simpler way to start forging these connections. "It's an interesting connection point," said Marblestone. "An entry point into thinking about next-generation neural networks."
Biggest cable stories
Source link
