His eyes also change: his eyelashes thicken and his eyelids become shimmering silver, then plum, then orange. Suddenly, his eyebrows thicken.
We watch her transformation live on her iPhone, where she looks ready for a fun night. Sitting next to me in real life, however, her makeup is much more subtle.
A shakeup for the makeup world
Augmented reality, or AR technology that covers virtual images in real-world live video, has been available for years in a handful of mobile applications – like children's games or an IKEA application that lets you rearrange virtual furniture in your real home. But with the exception of Pokémon Go, for example, consumers have not understood it.
Virtual makeup, one of the first triumphs of technology, also helps AR to position itself and sell real-world products.
The AR makeup illusion is aided by improvements in the underlying technology such as facial function and finger tracking. The operating systems for smartphones from Google and Apple now offer advanced AR tools, and the latest phones feature much more powerful front cameras than those of a few years ago. . The application makers have also done a meticulous job of matching the actual colors and textures of the beauty products to what appears on the screen of your smartphone.
The latest novelty, at least for companies looking to make money, is the opportunity to try, for example, Orly Nail Polish in Seize the Clay or Urban Decay Moondust Eyeshadow in Intergalactic, and to immediately buy these same products over the phone.
It's a good idea to try makeup via AR for the companies that make and use this technology, because the idea of trying before you buy is so deeply rooted in the makeup buying experience.
Hoping that the AR eyeliner can help the bottom line
Virtual lipsticks and steaming eyeshadows are popular in applications, but do they translate into increased makeup sales? It is hard to find reliable data, but several companies claim that it encourages users to play with different colors and can also convince them to open their wallets.
Lubomira Rochet, Head of Digital at L & # 39; Oreal, said that users typically spend more time on an application or website incorporating makeup or skin care features, and that those who do try to buy a product 10% more often than those who do not.
This, she says, "has a huge impact on the bottom line".
Prama Bhatt, senior vice president of digital and electronic commerce at Ulta Beauty, said tens of millions of colors had been tested and users of virtual tools to test an article were more inclined to add this product to their purchases. online. bag and buy it.
"Like other industries, especially in the beauty industry, the ability to discover, sample and try is a crucial part of the journey," Bhatt said. "So it makes a lot of sense from the point of view of the companies but also from the point of view of the experience."
It may soon become something consumers expect to be able to do before spending money on a makeup purchase, said Brendan Witcher, an analyst at Forrester Research, which covers digital business strategy. But if it can help you find something new or see how many products will come out, he does not know how useful it will be to find the makeup you really like.
"I will not go to augmented reality to buy the same lipstick that I've been using for three years," he said.
Augmented history of virtual metamorphoses
Technology has taken a long time to come here. When Aarabi launched ModiFace in 2006, the iPhone had not even been announced yet. There was no app store, even less application that you could use to try dozens of shades of lipstick in minutes.
Aarabi is a professor at the University of Toronto. He is studying artificial intelligence for facial tracking. With a colleague, he started by creating a website allowing users to download a still image of themselves and exchange lips, eyes and nose with those of someone else (you can change your lips with those of Angelina Jolie, for example).
It was "just for fun", but Allergan, a pharmaceutical company whose products include anti-aging products like Botox and Juvederm, asked them to create a tool that allows users to upload a photo of their face. and see how anti-aging products improve. their characteristics. It was a tool on a website? Another early partnership was a skin patch tool for Garnier, a hair care and skincare brand owned by L & # 39; Oreal.
"The more we entered, the more we understood that it was necessary to simulate virtual transformations and try in the field of beauty," Aarabi said.
Last March, L'Oréal acquired ModiFace, which offers several of its own virtual metamorphosis and licensing technology applications to other companies. Sephora now uses ModiFace to perform virtual transformations in its applications and on tablets installed in stores.
For years, it was impossible to give a realistic AR makeup to a live video stream from the face of a smartphone user. ModiFace started performing make-up simulations on photos, in 2009, but did not deploy live video functionality in any app until 2013. And when Sartori started working on GlamST in his Native country, Uruguay, in 2014 she had planned to store touchscreen kiosks, rather than as a mobile application.
Laggy lipstick and other opportunities for improvement
As impressive as seeing an AR-eyed liner that stays on your eyelids when you turn your head, there are still noticeable differences between virtual makeup and reality.
Presenting a virtual make-up on a real, real face – which can be twisted, flashing, smiling or frowning – involves a complicated hardware and software tango. In a nutshell, an app must detect a person's face in every frame of a video taken by the front camera of the phone, superimpose an image (for example, an eyeshadow and a blush) over- on top of the good parts and show it to the user, all in a few milliseconds. If the setting is incorrect, the digital lipstick will drag behind your lips as you turn your face or the eye shadows will dance over your eyelids.
Virtual products do not always match their real-world counterparts either. In order for the colors to be as close to reality as possible, these companies must first photograph samples of cosmetics under different lighting and on different skin tones, paying attention to aspects such as finishing, covering and the texture.
"We're really trying to say: What you're trying here is very similar to what you can try in real life," Sartori said. "This piece is very important to me because if you are shopping online and want to see different shades, when the product arrives, you want it to be similar."
To get an idea of the effectiveness of this process, I invited Sartori to show me makeup – real and virtual – in person.
In our office, she tried a multitude of virtual lip colors, which seemed generally realistic and sometimes seemed a bit wrong with her real lips. Then she compared them to samples of the same makeup on her arm. The colors rendered numerically correspond to the real ones; The metallic-looking Kylie Cosmetics lip gloss twinkled exactly as you wanted, while a MAC Cosmetics matte lipstick looked choked and thick.
Looking beyond the blush for hair and shoes
Technology has a long way to go. Foot tracking is always tricky – it does not work well when you move your foot – and there were fewer than a dozen 3-D shoe models (each in different patterns) in the application in mid-February.
This gives an idea of what the future might look like when smartphones and the technologies that underpin Augmented Reality continue to improve.
Beauty companies also hope to use AR to make virtual transformations even more realistic. They work on delicate tasks such as figuring out how to make sequins properly and make extreme hair color changes look normal.
They also think about how to customize product suggestions, in the hope of allowing you to try virtually and buy even more. But it can be even more difficult than showing you lipstick or sneakers that seem to be on your feet.
Aarabi said, "It remains a challenge."