Oculus Insight: how Facebook has unplugged virtual reality and opened virtual worlds to everyone

Virtual reality can be a daunting task – configuring sensors or transmitters, connecting cables, and hardware configuration can take so long that it may seem unnecessary to start a fast gaming session. It's too easy to leave the dust of this expensive VR headset, even if you have invested in a powerful gaming PC.

The release of Oculus Go in 2018 has made it possible to change that, by offering a fully autonomous virtual reality experience, wirelessly and no need for additional hardware. Simply fasten the helmet and run.

It was impressive and much more affordable than other VR headsets available at the time, but much more limited than more powerful and captive helmets. More importantly, his movement system was "uniquely oriented," which meant you could turn your head to look around the environment, but could not move.

All this changed with the release of Oculus Quest in May this year. The Quest is always fully wireless, but unlike Go, it offers six degrees of freedom, all without the need for external hardware in your room.

How is it possible? TechRadar has been talking with Anna Kozminski, head of Facebook's AR / VR software in Zurich, Switzerland, to learn how Oculus cuts the thread of virtual reality.

Follow-up of the interior

First, a quick explanation: six degrees of freedom means that your body can move in three dimensions along the X, Y and Z axes. Moving up and down along the Y axis is called "lifting", coming and going on the along the X axis is called "breaking", and moving left and right along the Z axis is called "sway". You can also turn to another axis. A movement between X and Y is called "pitching", a movement between X and Z is called "yaw" and a movement between Z and Y is called "rolling".

"To make this seem natural, we had to follow the user's movements extremely accurately in real time"

If you want to create a truly immersive virtual reality experience, it's essential. "With a virtual reality experience, you want it to be as fluid as in real life. You do not want to be distracted because the bill is late or flickering, "says Kozminski.

"To make everything natural, we had to follow the user's movements with extreme precision in real time, so when you move your head and your hands, these movements are perfectly represented in the virtual world."

Oculus Quest is the first consumer device offering full six degree of freedom tracking, made possible by the Oculus Insight system. It uses a technology called simultaneous localization and mapping (SLAM), which uses the inputs of various sensors to determine the position of an object in a constantly updated virtual map. He uses these objects as reference points (much like using landmarks to steer you) and anchors them to points in the virtual world.

These points are observed several times to compensate for the drift (where small deviations of measurement add up with time, making the map less accurate). The virtual zone encompasses your entire field of vision. The system must react as quickly as you can. Tracking errors or offsets are very visible.

"This tracking from six degrees of motion has already been done, but with many external sensors in the room, which was a significant barrier to entry," Kozminski said. "You can not share the device, you can not take it back to a friend. [Our goal was to] its level of immersion, but make it more convenient for more people to use it. "

Make VR portable

To achieve this, Oculus had to move the stack of software that performs all the calculations on the helmet itself. This represented a serious challenge. how to put this type of complex system on a lightweight object, portable and powered by a rechargeable battery?

"There was a lot of difficulty developing the architecture to use as little energy as possible and leave a lot of space on the device for game content," she said. "Oculus Insight is a gaming platform, but if there's no content to enjoy, it's debatable. [We had to] Break the trend that it can only work on powerful PCs and allow developers to create cool games and experiences. "

"We had to collect thousands and thousands of datasets, presenting all the different lighting and scenery settings"

Of course, a room in your house may be totally different from your friend's, so the system must be able to handle a wide range of conditions. "The algorithm is very sensitive to the type of environment in which it is used," Kozminski said. "The lighting conditions, the size of the space, the amount of texture on the walls and surfaces – but we can not predict how people will develop their living room. Maybe they want to have subdued lighting [for example].

"We had to collect thousands and thousands of datasets, presenting all the lighting and scenery settings, then replaying them on Facebook's servers and simulating the actual use of the headset so to be able to calculate metrics. It was a multi-year effort. "

(Image credit: Facebook Technologies)

The work was worth it, though, and the hardware developers were very impressed by the creatives' achievements. We interviewed Kozminski about his favorites until now.

"I think there are two," she said. "One is external use in general and the other is an internal project. When it comes to external uses, I'm a big fan of 360-degree videos, in addition to games. When producers use 360-degree cameras in an environment like a rainforest or refugee camp and deliver a video message, it's really compelling. You must be in the place of someone else to experience them. The video content is really convincing because it gives the impression that you really are there, and gives you that first-person perspective that empathizes with.

"Virtual reality can be a solitary experience if you use it yourself and our goal is to create meaningful connections"

"We did a demonstration at OC5 last year – a proof of concept for future applications of Oculus, with six users participating in a Western-style shootout in an arena. We co-located several players. Today, you can experience multiplayer experiences, but there is no indication that it is actually next to you. In this demo, we shared the same map on all devices in this collocated room.

"Virtual reality can be a lonely experience if you use it yourself and our goal is to create meaningful connections. This demo was a demonstration of what we could do in the future to share experiences shared across regions – whether playing or hanging out. "

Break the divisions

Kozminski and his Facebook colleagues are not resting on their laurels. "Now that we have shipped the Quest, our team is writing new features that we publish almost every month," she said.

However, it's not just about games – Oculus is a Facebook project, so it's no surprise that the ultimate goal is connecting people. And to do this, we will have to lower even more the barrier at the entrance

Kosminski explains, "I have family in Canada, and with these virtual reality experiences, we have multiplayer games, but in the future, we wish to bring together two very distant people one. on the other hand in the same experience. and make them feel that they are in the same place. I want to get to a point where I can put on a helmet and go out with my brother to Canada, as if we were in the same space – whether in RV or RA.

"Oculus Insight technology is what we will use to power anti-reflective glasses in the future. We talked about investing in AR and ultimately, when we will have a lighter form factor, more comfortable to wear, that more people will use and we can bring more people into this community to experience digital experiences. Get to the anti-reflective glasses and dissolve these geographical divisions. "