Physical Inertial Poser

Physical Inertial Poser (PIP)

Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors

Abstract

Motion capture from sparse inertial sensors has shown great potential compared to image-based approaches since occlusions do not lead to a reduced tracking quality and the recording space is not restricted to be within the viewing frustum of the camera. However, capturing the motion and global position only from a sparse set of inertial sensors is inherently ambiguous and challenging. In consequence, recent state-of-the-art methods can barely handle very long period motions, and unrealistic artifacts are common due to the unawareness of physical constraints. To this end, we present the first method which combines a neural kinematics estimator and a physics-aware motion optimizer to track body motions with only 6 inertial sensors. The kinematics module first regresses the motion status as a reference, and then the physics module refines the motion to satisfy the physical constraints. Experiments demonstrate a clear improvement over the state of the art in terms of capture accuracy, temporal stability, and physical correctness.

More Live Captures

PIP captures jumping faithfully with large contact forces

PIP performs well on daily activities

PIP captures a challenging dancing motion

The ground reaction forces change with upper-body movements

PIP captures the global movements with little drifts

The gound reaction forces switch with the contacting foot

PIP captures running faithfully with litter jitters

PIP captures physically correct motions with plausible contact forces

PIP captures some free-styles

Acknowledgments

This work was supported by Beijing Natural Science Foundation (JQ19015), the NSFC (No.61727808, 62021002), the National Key R&D Program of China 2018YFA0704000. This work was supported by THUIBCS, Tsinghua University and BLBCI, Beijing Municipal Education Commission. This work was partially supported by the ERC consolidator grant 4DReply (770784). We thank Notiom for the extensive support on inertial sensors, and Liuqing Yang, Liangdi Ma, Siyuan Teng, Wenbin Lin for the help on live demos. Feng Xu is the corresponding author.

Citation

@InProceedings{PIPCVPR2022,
  author = {Yi, Xinyu and Zhou, Yuxiao and Habermann, Marc and Shimada, Soshi and Golyanik, Vladislav and Theobalt, Christian and Xu, Feng},
  title = {Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2022}
}