Technology has quickly infiltrated almost every aspect of our lives, but the way we interface with our devices is still less than ideal. From hunching over our computer screens (because if you’re anything like me, it’s pretty much impossible to maintain good posture for more than a few minutes at a time) to constantly looking down at our phones (often while walking, driving, or otherwise being in motion), the way our bodies and brains interact with technology is not exactly seamless. Just how seamless we want it to become is debatable, but a Google project is exploring those boundaries.
Google’s Advanced Technology and Projects lab—ATAP—focuses on developing hardware to “change the way we relate to technology.” Its Project Jacquard developed conductive yarn to weave into clothing so people could interact with devices by, say, tapping their forearms—sort of like an elementary, fabric-based version of the Apple watch. The lab has also been working on a project called Soli, which uses radar to give computers spatial awareness, enabling them to interact with people non-verbally.
In other words, the project is trying to allow computers to recognize and respond to physical cues from their users, not unlike how we take in and respond to body language. “We are inspired by how people interact with one another,” said Leonardo Giusti, ATAP’s Head of Design. “As humans, we understand each other intuitively, without saying a single word. We pick up on social cues, subtle gestures that we innately understand and react to. What if computers understood us this way?”
Examples include a computer automatically powering up when you get within a certain distance of it or pausing a video when you look away from the screen.
The sensor works by sending out electromagnetic waves in a broad beam, which are intercepted and reflected back to the radar antenna by objects—or people—in their path. The reflected waves are analyzed for properties like energy, time delay, and frequency shift, which give clues about the reflector’s size, shape, and distance from the sensor. Parsing the data even further using a machine learning algorithm enables the sensor to determine things like an object’s orientation, its distance from the device, and the speed of its movements.
The ATAP team helped train Soli’s algorithm themselves by doing a series of movements while being tracked by cameras and radar sensors. The movements they focused on were ones typically involved in interacting with digital devices, like turning toward or away from a screen, approaching or leaving a space or device, glancing at a screen, etc. The ultimate goal is for the sensor to be able to anticipate a user’s next move and serve up a corresponding response, facilitating the human-device interaction by enabling devices to “understand the social context around them,” as ATAP’s Human-Computer Interaction Lead Eiji Hayashi put it.
Whether or not we want our devices to eventually become more like extensions of our bodies, it’s likely the technology will move in that direction. Analyses from 2019 through this year estimate we check our phones anywhere from 96 to 344 times per day. That’s a lot of times, and a lot of interrupting what we’re doing to look at these tiny screens that now essentially run our lives.