What crazy technology does Adobe come from? Start by moving stills

Adobe's artificial intelligence already introduces features that simplify the creative process, but a glance at the next step for Adobe Sensei could push technology beyond time-saving tools. During Adobe Max 2019, Adobe unveiled several insights into what software engineers will develop with the help of the company's artificial intelligence, Adobe Sensei. Whether it's creating an animated photo in one click, converting a recorded voice into a musical instrument or designing an animation that responds to tweets in real time, creative people might soon have new wacky tools in Creative Cloud.

Project in motion

Project Moving Stills converts a still image into animation, but unlike a cinematograph, Moving Stills creates realistic 3D camera movements in one click. Demonstrated on stage, the software uses A.I. to move around the scene, creating an effect that looks more like a videographer moving in the picture than a boring 2D slideshow transition.

Adobe Sensei understands how objects are organized in a 3D space, Adobe explains, and uses this information not only to create a 3D motion effect, but also to determine the best effect for that scene. The setting can be applied in one click, but the software also includes a handful of tools to change the 3D motion, from moving in the scene to panning up and down. The software also allows a custom camera movement by selecting a starting and ending point.

Mobile still images can also apply effects to multiple effects at once, creating a photo slideshow that makes objects appear in images as if they were taken with a 3D camera instead of traditional zoom-in-picture mirrors on images fixed. (Sorry, Ken Burns.)

The popularity of cinemagraphs and GIFs has inspired several new tools such as Plotagraph, but instead of animating an object in the photo, Moving Stills seems to be taking a step forward in the photo. As a preview, Adobe has not yet communicated the moment when Moving Stills will be launched, or even whether it will launch autonomously or as part of an existing software.

Kazoo Project

You do not know how to play an instrument or sing, but you want to create your own sound? A.I. may soon be able to help with that. Project Kazoo is a program that transforms the recordings of your voice into notes on an instrument – or even in notes of a soprano singer.

Demonstrated on the stage at Max by Zeyu Jin of Adobe, the software first takes a recording and organizes this audio as notes on a scale, noting the note and duration. Users then select an instrument from a drop-down menu and the program re-reads those notes in that instrument. In addition to working with the voice, the program can also transform the recording from one instrument to another. A transpose slider also adjusts the audio to a higher or lower pitch.

Besides helping the uninitiated to create music to create their own sound, Adobe explains that the prototype can also be used to create effects for cartoon characters, such as to imagine the sound of sound. a cartoon violin.

Bounty brush

What if Photoshop brushes could paint the movement? This is the idea of ​​Brush Bounty, a tool that allows animators to incorporate effects that would take time or even be impossible to animate. Adobe's Fabin Rasheed introduced Max's Brush Bounty project to Max, including rain-painted brushes, wind blown hair, a starry sky or a glowing orb.

In addition to saving animators from the individual creation of each raindrop and each strand of hair, Brush Bounty can also link these animations to locations or even tweets. Adding a hashtag to the project allows the animation to react in real time to all tweets using this hashtag. For example, on stage, Rasheed created a superhero with a glowing orb of increased size and intensity at each tweet of #BrushBounty.

The animation can also be related to the location of the viewer, for example by matching the weather in the animation to the location. Another glance showed the tool changing the direction of the wind depending on how the viewer was holding the smartphone, changing the direction of the wind depending on the movement of the smartphone.

Files can be exported as videos or GIFs, with Web elements for interactive animations. Like the other previews, Adobe did not share the moment of launching the tool.

Fastmask Project

For video editors, creating masks is a tedious task, especially in scenarios with enough movement for auto-hide options to not work. Project Fastmask is an alternative powered tool that hides moving subjects, and works even after they exit the frame.

The person (or animal) is hidden by placing a handful of endpoints in the first frame. Clicking on Propagate will then adjust these limit points for the next image, continuing to the end of the clip and leaving a well-hidden character for further adjustments.

Smooth Operator Project

Vertical videos are a headache for creatives that shoot a video on multiple platforms (IGTV, we're watching you). Project Smooth Operator uses A.I. to automatically crop horizontal videos to vertical videos – without leaving the subject behind.

Smooth Operator uses A.I. to analyze the video and determine the most important parts. The tool will then keep these items in the frame using the selected aspect ratio. If the subject moves, the crop will be similar to the actual pan. Besides the horizontal conversion to vertical, the tool can also reduce the aspect ratio.

Even more impressive, the demonstration included a video with two different subjects, a dog and its owner playing to search. Smooth Operator panes between the two subjects, deciding where to base the action of the video.

Fontphoria Project

Getting the right font is often a struggle for designers, but A.I will soon be able to generate a font from an image, including characters not in the image. Fontphoria generates a font based on a text image, such as a handwritten photo or the font of a vintage poster. Similarly, the program can also generate fonts when you open a document, but do not use all fonts, apply the characteristics of existing characters to create a full font.

A lens mode will allow Fontphoria to preview the font on existing text, using augmented reality to replace the existing text with the new font. Another feature allows you to apply custom changes to all characters at once, instead of manually applying special effects to each letter.

The list of Adobe previews also included Fantastic Fold, a packaging design program, Project Model Morph for manipulating 3D objects in Adobe Dimension (possibly), and Project Good Bones, a tool for editing vector graphics. taking into account their shape. Adobe also provided a preview of Project Waltz, which uses a smartphone to take photos or videos from a 3D project.

In preview, Adobe did not specify what would happen or even how the tools will arrive: some will probably be autonomous programs, while others may be integrated into existing programs.