Decoding multiple images from a single scattered exhibition

Duke University engineers have developed a way to extract a sequence of images of light scattered through an essentially opaque material, or even a wall, of light. a long photographic exhibition. The technique has applications in a wide range of areas from safety to health care through astronomy.

The study was published online September 10 in the journal Scientific reports.

"When I explain to people what this algorithm can do, it sounds magical," said Michael Gehm, an associate professor of electrical and computer engineering at Duke. "But these are just statistics and a ton of data."

When light is diffused through a translucent material, the emerging "speckled" pattern appears as random as static on a television screen without a signal. But it is not a coincidence. Because the light from one point of an object travels a path very similar to that of light coming from an adjacent point, the shimmer pattern of each seems very similar, it is just slightly offset.

With enough images, astronomers used this phenomenon of "memory effect" to create clearer images of the sky through a turbulent atmosphere, provided that the imaged object is sufficiently compact.

The technique broke down with the development of adaptive optics, which does the same job using adjustable mirrors to compensate for the diffusion.

A few years ago, however, the technique of memory effect became popular again among scientists. Modern cameras can record hundreds of millions of pixels at a time, so a single exposure is needed for statistics to work.

Although this approach can reconstruct a scattered image, it has its limits. The object must remain motionless and the scattering medium must be constant.

Gehm's new approach to imaging memory effects makes it possible to overcome these limitations by extracting a sequence of images from a single, long exposure.

The trick is to use a coded opening. Consider this as a set of filters that allow light to pass through certain areas, but not others in a specific pattern. As long as this model is known, scientists can extract by calculation what the original image looked like. (See the video)

Gehm's new technique uses a sequence of coded apertures to mark what light comes from when in time. But as each image is collected on a single, long photographic exposure, the resulting shimmer is even more messy than usual.

"People thought that the resulting speckle pattern would be too random to separate individual images," Gehm said. "But it turns out that today's cameras have such a stunning resolution that if you look closely, there is still a sufficient model to get an informal computing base and differentiate them."

In their experiment, a simple sequence of four backlit letters appeared one after the other behind a coded aperture and a scattering material. The shutter of a 5.5 megapixel CCD camera was left open for more than a minute during the sequence to collect the images.

Although the best results were obtained with a 100 second exposure time, good results could still be obtained with much shorter exposure times. After only a few seconds of processing, the computer successfully returned the individual images of a D, U, K, and E sequence. The researchers then showed that the approach also works when the scattering medium is changed, and even when the images and scattering media change.

The best results were obtained when the letters appeared for 25 seconds each, since the intensity of the backlighting was not very high at the beginning and was further diminished by the coded aperture and the diffusing material. But with a more sensitive camera or a brighter source, there's no reason the approach can not be used to capture live footage, Gehm said.

The technique has many potential applications. Not only does it work for the diffusion of light through a material, but it also works for the diffusion of light on a surface, for example painting on a wall. This could allow security cameras to bypass corners or even through frosted glass.

In the medical field, many light-based devices seek to collect data through skin and other tissues, such as a Fitbit capturing a person's pulse through his wrist. The scattering of light as it passes through circulating skin and blood cells, however, poses a problem for more advanced measurements. This technique can provide a way forward.

"We are also looking to see if this approach can be used to separate different aspects of light, especially color," said Gehm. "One could imagine using coded openings to get more information on a single image rather than using it to obtain a sequence of images."

Explore more:
Random scattering of light improves resolution of large-field optical microscope images

More information:
Xiaohan Li et al, one-shot memory effect video, Scientific reports (2018). DOI: 10.1038 / s41598-018-31697-8