United Human Pose: Integrating Domain Knowledge and Machine Learning

You are here

Inside Signal Processing Newsletter Home Page

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

News and Resources for Members of the IEEE Signal Processing Society

United Human Pose: Integrating Domain Knowledge and Machine Learning

Sarah Ostadabbas

PhD Thesis: United Human Pose: Integrating Domain Knowledge and Machine Learning

By: Shuangjun Liu, Augmented Cognition Lab (ACLab), ECE Department, Northeastern
University Advisor: Sarah Ostadabbas

With the great endeavor of computer vision community, 2D human pose estimation has achieved considerable success in recent years, from the introduction of single-person pose estimation models such as the convolutional pose machine and stacked hourglass models  to multi-person pose estimation networks such as OpenPose. These successes are greatly owed to the recent accessibility of large-scale well-annotated 2D human pose datasets, whose creation was possible due to the facts that (1) 2D human poses during certain daily activities are fairly easy to capture/collect and (2) annotating 2D poses via crowd-sourcing is a straightforward and affordable task. However, such convenience no longer holds when it comes to poses during specific activities such as sleeping or when advanced pose regression tasks such as 3D human poses are targeted. Starting with a general purpose human pose estimation goal, it is evident that the existing large-scale human pose datasets are all collected in an ``easy-to-access'' fashion, and do not contain data from applications where data collection or labelling is expensive (i.e. Small Data domains). This is very common for medical applications where data collection and/or labeling is expensive, individualized, and protected by very strong privacy or classification laws.

In order to address the ``Small Data'' problems especially when inference models with deep structures are used, there are two basic approaches to reduce data needs during model training:  (1) incorporate domain knowledge in the learning pipeline through the use of data-driven or simulation-based generative models, and (2) decrease inference model learning complexity via data-efficient machine learning. This thesis study is unfolded around addressing small data problems in the context of human pose estimation by leveraging the existing research and filling in key gaps with original work.

A specific human pose estimation problem in this domain, in-bed human pose estimation is extensively studied with solutions in an increasing order of feasibility, that make use of (1) conventional non-deep inference model, (3) fine-tuning already trained deep model, and (3)  building and training a pose model from scratch.  In order to address the small data challenge in a more general way, data augmentation approaches via both 2D and 3D haven also been studied including: (1) a pose guided approach to augment existing 2D human figure images while preserving their background, and (2) a semi-supervised data augmentation approach via 3D graphic engine and test its effectiveness against real human data.

SPS on Twitter

  • DEADLINE EXTENDED: The 2023 IEEE International Workshop on Machine Learning for Signal Processing is now accepting… https://t.co/NLH2u19a3y
  • ONE MONTH OUT! We are celebrating the inaugural SPS Day on 2 June, honoring the date the Society was established in… https://t.co/V6Z3wKGK1O
  • The new SPS Scholarship Program welcomes applications from students interested in pursuing signal processing educat… https://t.co/0aYPMDSWDj
  • CALL FOR PAPERS: The IEEE Journal of Selected Topics in Signal Processing is now seeking submissions for a Special… https://t.co/NPCGrSjQbh
  • Test your knowledge of signal processing history with our April trivia! Our 75th anniversary celebration continues:… https://t.co/4xal7voFER

IEEE SPS Educational Resources

IEEE SPS Resource Center

IEEE SPS YouTube Channel