[ad_1]
PITTSBURGH, September 11, 2018 – Researchers at Carnegie Mellon University have devised a way to automatically transform the content of one video into the style of another, allowing John Oliver's facial expressions to be transferred to those of a cartoon character. or to make a daffodil bloom like a hibiscus.
Since the data-driven method does not require any human intervention, it can quickly transform large amounts of video, making it an advantage for film production. It can also be used to convert black-and-white movies to color and to create content for virtual reality experiences.
Publicity
"I think there are a lot of stories to tell," said Aayush Bansal, a PhD. student at the CMU Robotics Institute. Film production was his main motivation to help design the method, he explained, making it possible to produce films more quickly and cheaply. "It's a tool for the artist that gives them an initial model that they can then improve," he added.
The technology also has the potential to be used for so-called "fake deep" videos, in which the image of a person is inserted without permission, which makes it appear that the person has done or said things out of his or her mind. character, recognizes Bansal.
"It was a revelation to all of us on the ground that such fakes would be created and have such an impact," he said. "Finding ways to detect them will be important to going forward."
Bansal will present the method today at the European Computer Vision Conference (ECCV 2018) in Munich. Among his co-authors are Deva Ramanan, associate professor of robotics at CMU.
The transfer of content from one video to another style relies on artificial intelligence. In particular, a class of algorithms called Generative Accident Networks (GAN) have allowed computers to more easily understand how to apply the style from one image to another, especially when they do not. have not been carefully matched.
In a GAN, two models are created: a discriminator that learns to detect what is consistent with the style of an image or a video, and a generator that learns to create images or videos corresponding to a certain style. When both work competitively – the generator trying to fool the discriminator and the discriminator evaluating the efficiency of the generator – the system finally learns how the content can be transformed into a certain style.
A variant, called cycle-GAN, completes the loop, much as if one translated English into Spanish, then Spanish into English, and then assessed whether the speech translated twice had a further meaning. The use of cycle-GAN to analyze the spatial characteristics of images has proven effective in transforming one image into another.
This spatial method leaves much to be desired for video, with artifacts and undesirable flaws in the complete cycle of translations. To mitigate the problem, researchers have developed a technique called Recycle-GAN, which integrates not only spatial but also temporal information. This additional information, taking into account changes over time, further limits the process and produces better results.
The researchers showed that Recycle-GAN can be used to turn Oliver's video into what appears to be comedian Stephen Colbert and return to Oliver. Or a video of John Oliver's face can be turned into a cartoon character. Recycle-GAN not only allows you to copy facial expressions, but also the movements and cadence of the performance.
Effects are not limited to faces, or even to bodies. The researchers demonstrated that the video of a blooming flower can be used to manipulate the image of other types of flowers. Clouds that quickly cross the sky on a windy day can be slowed down to give the impression of a quieter time.
Such effects could be useful for developing autonomous cars capable of sailing at night or in bad weather, Bansal said. It can be difficult to get a video of night scenes or stormy weather in which objects can be identified and tagged, he explained. For its part, Recycle-GAN can transform easily obtained and labeled day scenes into night or thunderstorm scenes, providing images that can be used to drive cars under these conditions.
More information and videos are available online at https: /
Source link
