I rode with virtual Nissan passengers with AR and 5G engines

Nissan's vision for the future of connected cars is as charged as possible with buzzwords. Leveraging 5G, AR and VR technologies to deliver advanced vehicle experience, the company's "Invisible-to-Visible" (I2V) technology allows not only HUD-type overlays, but also humanoid virtual badistants. by your side for conversation and relevant information locally.

Here is a practical diagram showing how it all works:

I have it? Yes, probably not. So I recently visited the Nissan Grandrive Test Center in Yokosuka, Japan, to see for myself how it would work. The automaker has just started testing the system in a moving vehicle equipped with 5G technology provided by NTT Docomo, the largest carrier in Japan. The test vehicle is based on a NV350 caravan.

Nissan's I2V system uses what the company calls "the metaverse", which does not refer to Persona 5The alternative dimension, but describes how virtual characters can appear in real-world vehicles thanks to augmented reality. In my demos, I wore a Meta 2 AR helmet that, although manufactured by a company that no longer exists, is very comfortable and offers a much wider field of view than Microsoft HoloLens.

I could see different badistants through the helmet, from a representative of the realistic Nissan RP at the PS2 level to a Cuboid cub named Nissan Bear to the extremely animated mascot of Unity Japan, Unity-chan. (Nissan says they used Unity-chan because it's free to get a license and the underlying software was written with Unity.)

The avatars were sitting in front of me at the back of the caravan or beside me in a separate static demonstration where I was in the driver's seat. The effect was reasonably convincing, even though the alignment of each seat was not perfect. Combined with the brightness of the Meta 2 image, it was hard to avoid the feeling that the images had been superimposed on my environment rather than being part of it.

The problem is that the virtual avatars I saw were actually real people, in a sense. While I was on the road, the speech and movements of the avatars were "interpreted" at the Grandrive Pavilion by two women wearing HTC Vive Pro VR helmets and – I swear I do not invent it – Crocs with attached Vive trackers. They could see a virtual representation of me through the cameras of the car and interact with my environment while talking live with me. It's just as strange and cool to see an augmented reality person shining by pressing a real button to open a real window.

All of this happens in real time with no perceptible latency, which is one of the advantages of 5G. Just in general, you do not want any shift risk when moving vehicles are involved. But the technology is still under test, a fact that came to light when Unity-chan tried to give me a donut, and then froze before my eyes when I heard a Windows error sound. She never resurrected. Nissan told me it was a problem with Docomo's 5G station, installed in a van parked next to the pavilion.

The use case of the AR badistant is clearly not an evolutionary concept, unless we are heading towards a strange future where cars on the road are badociated with employees. wearing Crocs clothes in VR cells pretending to be your friend. As a potential model for car chat, it's pretty cool. Over a long enough period, I do not think it's so ridiculous to imagine anyone who uses virtual reality to call a member of his family while he drives – otherwise nothing, that's all. 39 is the closest thing we will have from Star Wars-style hologram calls for a while.

Photography by Sam Byford