Niantic opens its AR platform so others can make games like Pokémon Go

Niantic Labs, the San Francisco-based game developer responsible for creating the massively successful augmented reality game Pokemon Go plans to open the AR platform behind its products to third-party developers .

At a meeting with reporters yesterday at his headquarters, CEO John Hanke gave a detailed overview of this technology – what Niantic calls his Real World platform. This is the driving force behind the AR experiments in Ingress Pokémon Go and the next title Harry Potter: Wizards Unite that this company is developing alongside Warner Bros. And Niantic says it improves all the time.

Alongside Hanke, other leaders of Niantic explain how the Real World engine uses a mix of computer vision, motion detection depth and object in real time. recognition. These are all artificial intelligence techniques that the company has perfected over the years while advancing its basic mapping, geolocation and social features that underpin applications like Pokémon Go .

To advance its RA techniques and make the Real World platform more robust, Niantic is acquiring a London-based vision-and-machine-learning start-up called Matrix Mill, whose members will now work at Niantic's first London office. The acquisition of Matrix Mill will allow Niantic to continue to build on AR's social experiences and follows the acquisition in February of a company called Escher Reality that helps now Niantic to develop multi-platform AR experiences that can involve multiple people in the same interactive digital space. In a series of demos, Niantic showed some of the new and experimental capabilities of its Real World platform, thanks in part to Matrix Mill's sophisticated feats in machine learning and other leveraging 39 AR experience shared skills that Escher offers. One of them involved a new AR visual technique. Niantic calls the occlusion, which allows virtual creatures like a 3D Pikachu to blend more realistically into real-world environments. This involves using automatic learning techniques to form a neural network capable of analyzing a real-time live scene with dynamic parts so that people and objects hide virtual creatures and hide them when necessary.

"Imagine, for example, that if our platform is able to identify and contextualize the presence of flowers, then it will be able to conjure up the tiny bee pokemon, Combee. see and contextualize a lake, he'll be able to bring up the duck pokemon, Psyduck, "Hanke explains in a blog post. "Recognizing objects is not limited to understanding what they are, but where they are." One of the main limitations of the AR currently is that AR objects can not interact in ways that in a 3D space Ideally, AR objects should be able to blend in with our reality, moving seamlessly behind and around real world objects. "

Another Niantic demo showed how, with the talent and the technology that she has acquired from Esther, she can develop applications several people interact in a shared AR environment, regardless of the type of device used. To do this, Niantic claims to have developed a low latency AR networking technique that prevents a smartphone from communicating with a server before establishing a connection with a nearby user. Instead, the device network allows everyone to communicate directly with one another via cell forwarding, which allows for lower latency connections and more immediate interactions with other players. The company built a demo game called Neon to show the technique:

In the future, Hanke says that he wants the Niantic Real World platform to work as Amazon Web Services for the cloud computing. In other words, the application makers will be able to leverage the power of its platform from anywhere in the world to develop their own experiences and services that use the technology and AR tools. Niantic has not yet released any concrete information on revenue sharing, or if the company would take a cut from applications developed using its technology. But the developer plans to publish more information on the Real Word platform and all the features of the API in the coming months.

Niantic has set up a website where developers can get more information about its Real World platform and how to apply for it. "Because we are so excited about the opportunity of an advanced RA, we want others to use the Niantic Real World platform to build innovative experiences that connect the physical and digital worlds of the world." A way we have not yet imagined, "Hanke said." We will select a handful of third-party developers to start working with these tools later this year. "