AI ‘DALL-E’ generates image from anything you describe

It can also draw and combine multiple objects and provide different views including cutouts and interiors of objects. Unlike older text-to-image programs, it even infers details that are not mentioned in the description but would be necessary for a realistic image. For example, with the description “a painting of a fox sitting in a field in winter,” the officer was able to determine that a shadow was needed.

“Unlike a 3D renderer, whose inputs must be specified unambiguously and in all detail, DALL · E is often able to ‘fill in the blanks’ when the caption implies that the image must contain some detail that does not exist. ‘is not explicitly stated, ”according to the OpenAI team.

OpenAI also leverages a capability called “zero-shot reasoning”. This allows an agent to generate a response from a description and a marker without any additional training, and has been used for translation and other tasks. This time, the researchers applied it to the visual domain to perform both image-to-image and text-to-image translation. In one example, he was able to generate an image of a cat from a sketch, with the indication “exactly the same cat at the top as the sketch at the bottom”.

The system has many other talents, such as understanding how phones and other objects change over time, capturing facts and geographic landmarks, and creating images in photographic, illustrative and even clipart styles.

For now, DALL-E is quite limited. Sometimes it delivers what you expect from the description and other times you just get some weird or crappy pictures. As with other AI systems, even the researchers themselves don’t understand exactly how it produces certain images due to the nature of the system’s black box.

Yet, if developed further, DALL-E has vast potential to disrupt areas such as photography and illustration, with all the pros and cons that that entails. “In the future, we plan to analyze how models like DALL · E relate to societal issues such as the economic impact on certain work processes and occupations, the potential for bias in model results and the longer term ethical challenges implied by this technology ”. the team wrote. To play DALL-E yourself, check out the OpenAI blog.