Researchers hacked robot vacuum cleaner to record speech and music remotely

A team of researchers has demonstrated that popular robotic household vacuum cleaners can be hacked remotely to act like microphones.

The researchers – including Nirupam Roy, an assistant professor in the Department of Computer Science at the University of Maryland – collected information from the laser navigation system of a popular robot vacuum and applied signal processing techniques and deep learning to recover speech and identify TV programs being played. in the same room as the appliance.

Research demonstrates the potential for any device that uses light sensing and telemetry (Lidar) technology to be manipulated to collect sound, even if it doesn’t have a microphone. This work, which is a collaboration with Assistant Professor Jun Han of the University of Singapore, was presented at the Association for Computing Machinery Conference on Embedded Networked Sensor Systems (SenSys 2020) on November 18, 2020.

“We welcome these devices into our homes, and we don’t think about it,” said Roy, who holds a joint appointment at the University of Maryland’s Institute for Advanced Computer Studies (UMIACS). “But we’ve shown that even though these devices don’t have microphones, we can reuse the systems they use for navigation to spy on conversations and potentially reveal private information.”

Home robot vacuum Lidar navigation systems emit a laser beam around a room and detect the laser’s reflection as it bounces off nearby objects. The robot uses the reflected signals to map the room and avoid collisions as it moves around the house.

Privacy experts have suggested that maps created by vacuum robots, which are often stored in the cloud, pose potential privacy breaches that could give advertisers access to information about things like the size of the house, which suggests income level, and other lifestyle-related information. Roy and his team wondered if the lidar of these robots could also pose potential security risks as sound recording devices in users’ homes or businesses.

Sound waves cause objects to vibrate, and these vibrations cause slight variations in the light bouncing off an object. Laser microphones, used in espionage since the 1940s, are able to convert these variations back into sound waves. But laser microphones rely on a focused laser beam reflecting off very smooth surfaces, such as glass windows.

A vacuum Lidar, on the other hand, scans the environment with a laser and detects the light returned by objects of irregular shape and density. The broadcast signal received by the vacuum sensor provides only a fraction of the information needed to recover sound waves. Researchers were unsure whether a robot vacuum’s lidar system could be manipulated to function as a microphone and whether the signal could be interpreted into meaningful sound signals.

First, the researchers hacked into a robot vacuum to show that they could control the position of the laser beam and send the detected data to their laptops over Wi-Fi without interfering with the device’s navigation.

Then they carried out experiments with two sound sources. One source was a human voice reciting numbers played over computer speakers and the other was audio from a variety of TV shows played through a TV sound bar. Roy and his colleagues then captured the laser signal detected by the vacuum’s navigation system as it bounced off a variety of objects placed near the sound source. Items included a trash can, cardboard box, takeout box, and polypropylene bag – items one can normally find on typical flooring.

The researchers transmitted the signals they received through deep learning algorithms that were trained to match human voices or to identify musical sequences from TV shows. Their computer system, which they call LidarPhone, identified and matched spoken numbers with 90% accuracy. He also identified TV shows from a minute of recording with over 90% accuracy.

“This type of threat is perhaps more important than ever, considering that we all order food by phone and meet by computer, and often share our credit card or bank information.” Roy said. “But what’s even more worrying to me is that it can reveal a lot more personal information. This type of information can tell you about my lifestyle, the number of hours I work, other things I do. And what we watch on TV can reveal our political orientations. This is crucial for someone who might want to manipulate political elections or target very specific messages to me. “

The researchers point out that vacuum cleaners are just one example of a potential vulnerability to lidar-based espionage. Many other devices could be exposed to similar attacks such as smartphone infrared sensors used for facial recognition or passive infrared sensors used for motion detection.

“I think this is important work that will educate manufacturers about these possibilities and inspire the security and privacy community to find solutions to prevent these types of attacks,” Roy said.

More information:
The research paper, “Spying with Your Robot Vacuum Cleaner: Eavesdropping via Lidar Sensors”, Sriram Sami, Yimin Dai, Sean Rui Xiang Tan, Nirupam Roy and Jun Han, was presented on November 18, 2020 at the Association for Computing Machinery , SenSys 2020.