The human hand is a marvel of biological engineering, featuring 34 muscles, 27 joints, and over 100 tendons and ligaments that allow for intricate movements and gestures. However, replicating this level of dexterity in robots has proven to be a significant challenge for researchers. A team from the Massachusetts Institute of Technology (MIT) and the University of Southern California is making strides in this area with a novel approach that utilizes ultrasound imaging to enhance robotic hand control.
Led by mechanical engineering professor Xuanhe Zhao, the team has developed an innovative wristband equipped with a miniaturized ultrasound “sticker” that adheres to the skin using a hydrogel. This device captures real-time ultrasound images of the wrist’s underlying muscles and tendons as the wearer moves their hand. An artificial intelligence algorithm, trained on meticulously labeled ultrasound images, translates these images into the corresponding positions of the fingers and palm. According to Gengxi Lu, a lead author of the related research paper, “The tendons and muscles in your wrist are like strings pulling on puppets, which are your fingers.” This insight allows the device to monitor the state of the hand with remarkable accuracy.
In practical demonstrations, the wristband has enabled users to wirelessly control a robotic hand, mimicking their gestures in real-time. Users have successfully manipulated the robot to play a simple tune on a piano and even shoot a mini basketball into a desktop hoop. Additionally, the wristband can interact with virtual environments, allowing users to pinch fingers together to resize objects on a computer screen. The research team aims to further miniaturize the device, currently comparable in size to a smartphone, and expand its training data set to include a diverse range of hand sizes and gestures. The ultimate goal is to create a wearable device that facilitates intuitive and precise control of humanoid robots and virtual objects, potentially revolutionizing applications in design, gaming, and even surgical procedures. As Zhao emphasizes, this technology represents a significant advancement in tracking dexterous hand motion through wearable imaging.
Source: Ultrasound imaging turns a robot hand into a skillful mimic via MIT Technology Review