Researchers at Cornell University in New York have developed a first-of-its-kind wristband-mounted miniature camera that is capable of tracking an entire body’s posture in 3D.
Dubbed BodyTrak, the device, which also features a customised deep neural network, is the first wearable that can track the full body pose with a single camera. According to Cheng Zhang, an assistant professor of information science at Cornell, if integrated into future smartwatches, BodyTrak could be a game-changer in monitoring user body mechanics in physical activities where precision is critical.
Advertisement
‘Since smartwatches already have a camera, technology like BodyTrak could understand the user’s pose and give real-time feedback,’ Zhang said. ‘That’s handy, affordable and does not limit the user’s moving area.’
BodyTrak is the latest body-sensing system developed by SciFiLab – a group based in the Cornell Ann S Bowers College of Computing and Information Science that has previously developed and leveraged similar deep-learning models to track hand and finger movements, facial expressions and even silent-speech recognition.
Advertisement
The secret to BodyTrak is not only in the five-pence-sized camera on the wrist, but also the deep neural network behind it. This deep neural network – a method of AI that trains computers to learn from mistakes – reads the camera’s rudimentary images or ‘silhouettes’ of the user’s body in motion and virtually re-creates 14 body poses in 3D and in real time.
In other words, the model accurately fills out and completes the partial images captured by the camera, said Hyunchul Lim, a doctoral student in the field of information science.
Advertisement
‘Our research shows that we don’t need our body frames to be fully within camera view for body sensing,’ Lim said. ‘If we are able to capture just a part of our bodies, that is a lot of information to infer to reconstruct the full body.’
Maintaining privacy for bystanders near someone wearing such a sensing device is a legitimate concern when developing these technologies, said Zhang and Lim. According to the researchers, BodyTrak mitigates privacy concerns for bystanders since the camera is pointed toward the user’s body and collects only partial body images of the user.
They also recognise that today’s smartwatches don’t yet have small or powerful enough cameras and adequate battery life to integrate full body sensing, but could in the future.
Advertisement
The research has been published in the Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technology.