How AR's Facebook glasses could become real thanks to Oculus VR technology

The Facebook Oculus has seen many changes recently, with departures from key leaders of the mobile VR team, Nate Mitchell and Max Cohen. But Facebook's standalone VR headset released this spring, Oculus Quest, remains this year's best VR hardware achievement … and could point to Facebook's next destination in the RA.

Facebook is now openly discussing the fact that Oculus Insight is the technology that will drive future headsets as they move from virtual reality to augmented reality. Insight is computer vision and tracking based on AI in the Rift S and Quest helmets, which uses cameras in headphones, motion sensors and infrared LEDs in controllers to map real spaces into virtual reality and make everything smooth and fluid. Facebook is not the only company to use this type of tracking in virtual reality, but it is becoming increasingly clear that Facebook is also seeing this technology as a stepping stone to AR, and will be part of that. which does AR glasses long promised by Facebook technical work.

Facebook's technical lead Joel Hesch, who led the development of Oculus Insight, spoke to CNET about the potential of tracking technology.

A mix of virtual reality and virtual reality

Sometimes, Oculus Quest feels like a gateway to a mix of virtual and real, especially when you set a virtual space limit and you enter a bubble of the game world: Oculus camera tracking Insight is good enough to feel like you. to repaint borders in the real world. Monitoring could then be used for larger scale experiments.

Hesch referred to last year's Oculus Connect 5 conference as an indicator, evoking a demo of tracking the size of an arena last year, during which players were running around the world. A giant arena with obstacles and were playing a version of the VR shooter and death, Dead and Buried. "We had real world objects, those boxes that you could hide behind, lined up with objects from the virtual world.This kind of experience gives an idea of ​​what we are considering, how do we mix up content from real world and virtual world, and how do we think about multi-user experiences, with more than one person in the space ".

Reading in progress:
Look at this:

We took Oculus Quest on vacation

5:18

It remains to be seen if Facebook will allow Oculus Quest to be more customizable for companies conducting geolocation experiments, or LBE (thematic attractions resembling theme parks like the vacuum). ("The option is not hidden" for decision makers who wish to personalize Oculus Insight's follow-up, Hesch says, but admits that the priority works for "as many cases as possible. ") Some companies, like MANUVR, are already finding a way to mix Oculus Quest with larger arenas using Microsoft's Azure Spatial Anchors. Facebook has already improved tracking on Oculus Rift S and Quest via a system update. Hesch said: "We are very committed to a long track record of improvements".

A gateway to a Facebook AR headset

In the Facebook blog, "The future of this technology lies in the use of all-day, portable anti-glare goggles that are spatially aware.This will require the execution of SLAM (simultaneous localization and mapping). ) with even greater constraints, including a further reduction in latency and power consumption, as little as 2 percent of what is needed for SLAM on an HMD (head-mounted screen). " the anti-glare glasses could also mean changing the inputs of the physical controllers of Oculus Quest into something else.

Hesch did not specify whether the future AR AR helmets would also use controllers such as Oculus Rift S and Quest, but he stated that it was possible that Oculus VR or AR could possibly recognize hand gestures. . "In theory, you have four If you look at the scientific literature, there are a number of applications for computer vision, perception, that you can use with normal camera data. Do not talk about anything specific about the road map, but you can use your imagination with what are the different possibilities that come up when you have camera data available. "

With the AR headsets, the inputs could end up being more manual, much like what Microsoft HoloLens and Magic jump already incorporate. "If you're thinking of typing in general for virtual and augmented computing experiences," said Hesch, "what should this look like? We're really trying to identify what makes the most sense for users in the use cases they are tackling .. and we hope this will evolve over time.We are committed to exploring all possible options for monitoring the inputs and understanding what we can bring to the products that will enhance the user experience. "

Does Facebook Portal and other devices intersect?

Oculus Insight's authorized space tracking will not be focused solely on virtual reality: Hesch claims that elements of this computer vision technology are already being used in Facebook phone RAs. "For what we are doing in AR on mobile today and in Quest and Rift S, many of these common building blocks such as the Insight system actually work across multiple platforms, and we see this as a real opportunity to leverage our benefits. " investments when we look at new form factors and new products that we can market. "Hesch mentions including space-based AI:" We believe that space-based AI is a fundamental element for all platforms. "

Facebook Portal, the videoconferencing tablet owned by Facebook's AR / VR division, does not yet use Oculus Insight technology. But, Hesch suggests that "if you look at the cross-platform intersection between portable devices and a base station, there is definitely a potential for collaboration / overlap".

All other announcements on the next Oculus direction will likely Oculus Connect 6 at the end of September, which is only a month away. We will see how many Facebook AR plans are unveiled.