Catherine Tianhong Yu 于天泓

Closed-loop sensorimotor training systems can improve learning by sensing movement and delivering real-time feedback, yet most are built as fixed implementations tied to a single task, even though the core technology (inertial and tactile sensing, vibrotactile cueing, rule-based logic) remains the same. We present Sensorimotor Stickies, a reconfigurable on-body platform that treats sensing and vibrotactile feedback as modular stickies that can be patched onto the body as needed. The platform includes miniaturized adhesive modules for IMU sensing, optional tactile sensing, and vibrotactile actuation; low-power firmware and BLE infrastructure for raw streaming and motor control without task-specific rewrites; and a companion mobile app that provides a shared body-centered model for placement, calibration, and feedback authoring. Together, these components enable reconfiguration across training scenarios, user needs, and feedback setups. We evaluate the platform through technical characterization, configured application demonstration, practitioner-mediated configuration sessions, and an end-user study, demonstrating technical feasibility, reconfiguration breadth, and end-user configurability for first-time setup, calibration, and within-task feedback reconfiguration.

Tactile gloves digitize contact and force during hand-object interactions, enabling robotics applications in dexterous manipulation, teleoperation, and learning from demonstration. To preserve hand dexterity and capture the nuances of natural interactions, these gloves and the integrated tactile sensors are designed to be soft, flexible, and comfortable. However, such flexible sensors are sensitive not only to contact forces but also unavoidably to hand pose changes, resulting in pose-related artifacts (PRAs). PRAs are especially problematic in the low-force range, resulting in misdetections or late-onset detections of contact, which raises the minimum detectable force (MDF) of the glove. In this work, we characterize the PRAs in relation to pose and force. Building on these insights, we introduce a glove-agnostic algorithmic framework that leverages hand pose information, which is increasingly available, to mitigate PRAs without glove modifications. Our pose-aware force estimation model augments tactile-to-force pipelines with a residual prediction branch that explicitly accounts for pose-induced sensor deformations. We validate our approach across 3 glove designs and 15 users, reducing MDF by 10.4%, 12.2%, and 18.3%, with consistent improvements across all evaluated metrics. This method provides a practical path to improving the usability of tactile gloves in data collection and diverse robotic applications.

Smart clothing has exhibited impressive body pose/movement tracking capabilities while preserving the soft, comfortable, and familiar nature of clothing. For practical everyday use, smart clothing should (1) be available in a range of sizes to accommodate different fit preferences, and (2) be washable to allow repeated use. In SeamFit, we demonstrate washable T-shirts, embedded with capacitive seam electrodes, available in three different sizes, for exercise logging. Our T-shirt design, customized signal processing & machine learning pipeline allow the SeamFit system to generalize across users, fits, and wash cycles. Prior wearable exercise logging solutions, which often attach a miniaturized sensor to a body location, struggle to track exercises that mainly involve other body parts. SeamFit T-shirt naturally covers a large area of the body and still tracks exercises that mainly involve uncovered joints (e.g., elbows and the lower body). In a user study with 15 participants performing 14 exercises, SeamFit detects exercises with an accuracy of 89%, classifies exercises with an accuracy of 93.4%, and counts exercises with an error of 0.9 counts, on average. SeamFit is a step towards practical smart clothing for everyday uses.

We present Ring-a-Pose, a single untethered ring that tracks continuous 3D hand poses. Located in the center of the hand, the ring emits an inaudible acoustic signal that each hand pose reflects differently. Ring-a-Pose imposes minimal obtrusions on the hand, unlike multi-ring or glove systems. It is not affected by the choice of clothing that may cover wrist-worn systems. In a series of three user studies with a total of 36 participants, we evaluate Ring-a-Pose’s performance on pose tracking and micro-finger gesture recognition. Without collecting any training data from a user, Ring-a-Pose tracks continuous hand poses with a joint error of 14.1mm. The joint error decreases to 10.3mm for fine-tuned user-dependent models. Ring-a-Pose recognizes 7-class micro-gestures with a 90.60% and 99.27% accuracy for user-independent and user-dependent models, respectively. Furthermore, the ring exhibits promising performance when worn on any finger. Ring-a-Pose enables the future of smart rings to track and recognize hand poses using relatively low-power acoustic sensing.

Seams are areas of overlapping fabric formed by stitching two or more pieces of fabric together in the cut-and-sew apparel manufacturing process. In SeamPose, we repurposed seams as capacitive sensors in a shirt for continuous upper-body pose estimation. Compared to previous all-textile motion-capturing garments that place the electrodes on the clothing surface, our solution leverages existing seams inside of a shirt by machine-sewing insulated conductive threads over the seams. The unique invisibilities and placements of the seams afford the sensing shirt to look and wear similarly as a conventional shirt while providing exciting pose-tracking capabilities. To validate this approach, we implemented a proof-of-concept untethered shirt with 8 capacitive sensing seams. With a 12-participant user study, our customized deep-learning pipeline accurately estimates the relative (to the pelvis) upper-body 3D joint positions with a mean per joint position error (MPJPE) of 6.0 cm. SeamPose represents a step towards unobtrusive integration of smart clothing for everyday pose estimation.

We propose Skinergy for self-powered on-skin input sensing, a step towards prolonged on-skin device usages. In contrast to prior on-skin gesture interaction sensors, Skinergy’s sensor operation does not require external power. Enabled by the triboelectric nanogenerator (TENG) phenomenon, the machine-embroidered silicone-textile composite sensor converts mechanical energy from the input interaction into electrical energy. Our proof-of-concept untethered sensing system measures the voltages of generated electrical signals which are then processed for a diverse set of sensing tasks: discrete touch detection, multi-contact detection, contact localization, and gesture recognition. Skinergy is fabricated with off-the-shelf materials. The aesthetic and functional designs can be easily customized and digitally fabricated. We characterize Skinergy and conduct a 10-participant user study to (1) evaluate its gesture recognition performance and (2) probe user perceptions and potential applications. Skinergy achieves 92.8 percent accuracy for a 11-class gesture recognition task. Our findings reveal that human factors (e.g., individual differences in skin properties, and aesthetic preferences) are key considerations in designing self-powered on-skin sensors for human inputs.

Tactile sensing is essential for robots to perceive and react to the environment. However, it remains a challenge to make large-scale and flexible tactile skins on robots. Industrial machine knitting provides solutions to manufacture customiz-able fabrics. Along with functional yarns, it can produce highly customizable circuits that can be made into tactile skins for robots. In this work, we present RobotSweater, a machine-knitted pressure-sensitive tactile skin that can be easily applied on robots. We design and fabricate a parameterized multi-layer tactile skin using off-the-shelf yarns, and characterize our sensor on both a flat testbed and a curved surface to show its robust contact detection, multi-contact localization, and pressure sensing capabilities. The sensor is fabricated using a well-established textile manufacturing process with a programmable industrial knitting machine, which makes it highly customizable and low-cost. The textile nature of the sensor also makes it easily fit curved surfaces of different robots and have a friendly appearance. Using our tactile skins, we conduct closed-loop control with tactile feedback for two applications: (1) human lead-through control of a robot arm, and (2) human-robot interaction with a mobile robot.

A scarf is inherently reconfigurable: wearers often use it as a neck wrap, a shawl, a headband, a wristband, and more. We developed uKnit, a scarf-like soft sensor with scarf-like reconfigurability, built with machine knitting and electrical impedance tomography sensing. Soft wearable devices are comfortable and thus attractive for many human-computer interaction scenarios. While prior work has demonstrated various soft wearable capabilities, each capability is device- and location-specific, being incapable of meeting users’ various needs with a single device. In contrast, uKnit explores the possibility of one-soft-wearable-for-all. We describe the fabrication and sensing principles behind uKnit, demonstrate several example applications, and evaluate it with 10-participant user studies and a washability test. uKnit achieves 88.0percent/78.2percent accuracy for 5-class worn-location detection and 80.4percent/75.4percent accuracy for 7-class gesture recognition with a per-user/universal model. Moreover, it identifies respiratory rate with an error rate of 1.25 bpm and detects binary sitting postures with an average accuracy of 86.2 percent.