Caltech researchers develop two new illusions with audiovisual postdictive effects

We experience the world through our senses, a constant stream of sights, sounds, smells and more. Our brain takes these signals and processes them, giving rise to our individual perceptions of the world. But sometimes our senses play tricks, especially in the case of perceptual illusions.

Now, Caltech researchers have developed two new illusions that reveal how the senses can influence each other, especially how sound can generate visual illusions. These illusions occur so quickly that they illustrate a phenomenon called a postdiction (as opposed to prediction) in which a stimulus that occurs later can retroactively affect our perception of an earlier event.

Caltech's work is among the first to show this kind of illusion of time travel through multiple senses.

The work was done in the laboratory of Shinsuke Shimojo, Professor of Experimental Psychology at Gertrude Baltimore Chair and affiliate faculty member of the Tianqiao Institute of Neuroscience and Chrissy Chen at Caltech. An article describing the research appears in the October 3 issue of the journal PLOS ONE.

"Illusions are a really interesting window on the brain," said first author, Noelle Stiles (PhD & # 39; 15), visiting biology & bioengineering and postdoctoral research associate at the USC. "By investigating illusions, we can study the decision-making process of the brain – for example, how does the brain determine reality with information from multiple, sometimes noisy and conflicting senses?" The brain uses assumptions about the brain. When these assumptions are false, illusions can occur when the brain tries to give the best possible meaning to a confusing situation.We can use these illusions to reveal the underlying inferences that the brain is making. "

The two illusions of this study have been developed to illustrate how later stimuli can affect the perception of stimuli already produced. Postdoctoral treatment has been demonstrated in the individual senses, but this work focuses on how the phenomenon can fill several senses. The key to the two new illusions is that audio and visual stimuli occur quickly, in less than 200 milliseconds (one-fifth of a second). The brain, trying to understand this dam of information, synthesizes the stimuli of both senses to determine the experience, using the postdiction to do it.

The first illusion is called the illusory rabbit. To produce the illusion, a short beep and a fast flash are first broadcast almost simultaneously on a computer, the flash appearing on the left of the screen. Then, 58 milliseconds after the first beep, a single beep is played. Finally, 58 milliseconds after the second beep, a second pair of almost simultaneous beep-beep occurs, but the flash appears on the right side of the screen. The location of the beep is always central and does not move. Although only two flashes are read, most people viewing the illusion perceive three flashes, an illusory flash coinciding with the second beep and appearing to be in the center of the screen.

The fact that the illusory flash is perceived between the left and right flashes is the main proof that the brain uses a postdiction treatment.

"When the final beep-flash torque is presented, the brain assumes that it had to miss the flash associated with the unpaired beep and literally explains the fact that there must be a second flash that it missed, "says Stiles. "This already implies a post-operative mechanism at work, but more importantly, the only way to perceive the illusory flash misplaced would be that the information that arrives later in time – the final beep-flash combination – be used to reconstruct it as well. 39, the most likely location of the illusory flash. "

The second illusion is called the invisible rabbit. In this related illusion, three flashes are displayed on the screen, the first on the left, the second in the middle and finally the third on the right, only the first and third flashing coinciding with beeps. In this case, most people do not see the second flash at all – the one without the corresponding flash. The absence of the second beep causes the brain to decide after the fact that there was no flash, even though it was present.

By showing that a sound can excite a visual illusion, researchers have discovered new clues as to how the brain combines the senses in space and time to generate an integrated sense of perception.

"The scope of this study is twofold," says Shimojo. "In the first place, he generalizes postdiction as a key process of perceptual processing for a single sense and multiple meanings.The post-sentence may seem mysterious, but it is not the case.It must take into account the time that it is necessary for the brain to process the times, subsequent stimuli in a different sense may affect or modulate the first principle.The second important point is that these illusions are among the very rare cases where the sound affects the vision, not the opposite, indicating the dynamic aspects of the neural processing that is occurring New illusions will allow researchers to identify the optimal parameters for multisensory integration, necessary for the design of neurons. ideal sensory aids and optimal training for visually impaired people. "

This sound and visual illusion illustrates the postdiction phenomenon: when a stimulus presented later affects the perception of a previously presented stimulus. In this case, a quick series of beeps and flashes causes the brain to believe that it sees a flash when there was none. Credit: Caltech / Shimojo laboratory