Facebook finally explains its mysterious new wearable on the wrist

It first appeared on March 9 as a tweet on the timeline of Andrew Bosworth, the little corner of the internet that offers a rare glimpse into the mind of a Facebook executive these days. Bosworth, who heads Facebook’s augmented and virtual reality research labs, had just shared a blog post outlining the company’s 10-year vision for the future of human-machine interaction. Then in a follow-up tweet, he shared a photo of a still invisible portable device. Facebook’s vision for the future of interacting with computers would apparently involve strapping something that looks like an iPod Mini to your wrist.

Facebook already has our social background and some of the most popular messaging apps in the world, for better or for worse. Every time the company dives into hardware, whether it’s a great VR headset or a video chat device that tracks your every move, it gets noticed. And that raises not only intrigue, but also questions: Why Does Facebook want to own this new IT paradigm?

In this case, the unanswered questions are less about the material itself and more about the research behind it – and whether the new interactions Facebook envisions will only deepen our bond with Facebook. (Answer: probably.) At a press briefing earlier this week, Facebook executives and researchers offered a glimpse into the technology. In simpler terms, Facebook tested new computer inputs using a wearable bracelet filled with sensors.

It is an electromyography machine, which means that it translates nerve signals from electric motors into digital commands. When it’s on your wrist, you can just flick your fingers across space to control the virtual inputs, whether you’re wearing a VR headset or interacting with the real world. You can also “train” it to sense the intention of your fingers, so that actions occur even when your hands are completely still.

This wrist cell phone has no name. This is just a concept, and there are different versions of it, some of which include haptic feedback. Bosworth says it could be five to ten years before the technology becomes widely available.

This all ties into Facebook’s plans for virtual and augmented reality, technologies that can sometimes leave the user with a distinct lack of agency when it comes to their hands. Put on a VR headset and your hands are completely gone. By taking a pair of controllers, you can play games or grab virtual objects, but then you lose the ability to take notes or draw accurately. Some AR or “mixed reality” headsets like Microsoft’s HoloLens have cameras that track spatial gestures, so you can use certain hand signals and the headset will interpret those signals… which sometimes works. So Facebook used this EMG laptop in its virtual reality lab to see if such a device could allow more precise hand-computer interactions.

But Facebook has visions for this wrist technology beyond AR and VR, Bosworth says. “If you really had access to an interface that allowed you to type or use a mouse, without having to type or physically use a mouse, you could use it anywhere.” The keyboard is a prime example, he says; this wrist computer is just another means of intentional typing, except you can take it anywhere with you.

Bosworth also suggested the kitchen microwave as a use case – noting that Facebook doesn’t, in fact, build a microwave. Appliance interfaces are all different, so why not program an appliance like this to just understand when you want to cook something for 10 minutes on medium power?

In the virtual demo that Facebook gave earlier this week, a gamer was shown wearing the wrist and controlling a character in a rudimentary video game on a flat screen, all without having to move their fingers at all. These types of demos tend to (pardon the pun) a move towards mind-reading technology, which Bosworth insisted it isn’t. In this case, he says, the mind generates signals identical to those that would move the thumb, but the thumb does not move. The device records a intention to move the thumb. “We don’t know what’s going on in the brain, which is full of thoughts, ideas and notions. We don’t know what’s going on until someone sends a signal over the wire.

Bosworth also pointed out that this wearable wrist is different from the invasive implants that were used in a 2019 brain-computer interface study that Facebook worked on with the University of California, San Francisco; and it’s different from Elon Musk’s Neuralink, a wireless implant that could theoretically allow people to send neuroelectric signals from their brains directly to digital devices. In other words, Facebook doesn’t read our thoughts, even though it already knows a lot about what’s going on in our heads.

Researchers say there is still a lot of work to be done in the area of ​​using EMG sensors as virtual input devices. Precision is a big challenge. Chris Harrison, director of the Future Interfaces Group at Carnegie Mellon University’s Human-Computer Interaction Lab, points out that every human’s nerves are a little different, and so are the shapes of our arms and wrists. “There is always a calibration process that has to happen with any muscle sensing system or BCI system. It really depends on where the computational intelligence is, ”says Harrison.

And even with the haptic feedback built into these devices, as Facebook does with some of its prototypes, there is a risk of visuo-haptic mismatch, where the user’s visual experience – whether in AR, VR or in real space – no matched haptic response. These sticking points can make these human-computer interactions all frustrating. areal.

Even if Facebook can overcome these obstacles in its research labs, there remains the question of Why Facebook – largely a software publisher – wants to appropriate this new IT paradigm. And should we trust him? That hugely powerful tech company that has a proven track record of sharing user data in “exchange for other equally or more valuable things,” as WIRED’s Fred Vogelstein wrote in 2018? A more recent report in MIT Technology Review highlights how a Facebook team united to fight “responsible AI” has been undermined by the relentless pursuit of leadership growth.

Facebook executives said this week that these new human-computer interaction devices will perform as much computation as possible “on the device,” meaning information is not shared in the cloud; but Bosworth will not make any commitments on how much data may ultimately be shared with Facebook or how that data will be used. The whole thing is a prototype, so there is nothing substantial to separate yet, he says.

“Sometimes these companies have piles of cash that is large enough to invest in these huge R&D projects, and they will take a loss on such things if that means they can be in the forefront in the future,” says Michelle Richardson, chief data officer. and Privacy Project at the Nonprofit Center for Democracy and Technology. “But with businesses of any size, any product, once it’s built, it’s so hard to overhaul it. So anything that can start the conversation about this before the devices are built is a good thing. “

Bosworth says Facebook wants to lead this next paradigm shift in IT because the company sees such technology as fundamental to connecting people. If anything, this past year has shown us the importance of connecting – of feeling as if you were in person, said Bosworth. He also seems to believe that he can gain the requisite trust by not “surprising” customers. “You say what you do, you set expectations, and you meet those expectations over time,” he says. “Confidence comes on foot and sets off again on horseback.” Pink AR glasses, activated.

This story originally appeared on wired.com.