United Human Pose: Integrating Domain Knowledge and Machine Learning

You are here

Inside Signal Processing Newsletter Home Page

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

10 years of news and resources for members of the IEEE Signal Processing Society

United Human Pose: Integrating Domain Knowledge and Machine Learning

By: 
Sarah Ostadabbas

PhD Thesis: United Human Pose: Integrating Domain Knowledge and Machine Learning

By: Shuangjun Liu, Augmented Cognition Lab (ACLab), ECE Department, Northeastern
University Advisor: Sarah Ostadabbas

With the great endeavor of computer vision community, 2D human pose estimation has achieved considerable success in recent years, from the introduction of single-person pose estimation models such as the convolutional pose machine and stacked hourglass models  to multi-person pose estimation networks such as OpenPose. These successes are greatly owed to the recent accessibility of large-scale well-annotated 2D human pose datasets, whose creation was possible due to the facts that (1) 2D human poses during certain daily activities are fairly easy to capture/collect and (2) annotating 2D poses via crowd-sourcing is a straightforward and affordable task. However, such convenience no longer holds when it comes to poses during specific activities such as sleeping or when advanced pose regression tasks such as 3D human poses are targeted. Starting with a general purpose human pose estimation goal, it is evident that the existing large-scale human pose datasets are all collected in an ``easy-to-access'' fashion, and do not contain data from applications where data collection or labelling is expensive (i.e. Small Data domains). This is very common for medical applications where data collection and/or labeling is expensive, individualized, and protected by very strong privacy or classification laws.

In order to address the ``Small Data'' problems especially when inference models with deep structures are used, there are two basic approaches to reduce data needs during model training:  (1) incorporate domain knowledge in the learning pipeline through the use of data-driven or simulation-based generative models, and (2) decrease inference model learning complexity via data-efficient machine learning. This thesis study is unfolded around addressing small data problems in the context of human pose estimation by leveraging the existing research and filling in key gaps with original work.

A specific human pose estimation problem in this domain, in-bed human pose estimation is extensively studied with solutions in an increasing order of feasibility, that make use of (1) conventional non-deep inference model, (3) fine-tuning already trained deep model, and (3)  building and training a pose model from scratch.  In order to address the small data challenge in a more general way, data augmentation approaches via both 2D and 3D haven also been studied including: (1) a pose guided approach to augment existing 2D human figure images while preserving their background, and (2) a semi-supervised data augmentation approach via 3D graphic engine and test its effectiveness against real human data.

SPS on Twitter

  • The virtual conference is powered by leading companies who have continued to show their support for our t… https://t.co/Jz7mMCqqhJ
  • The virtual conference begins today! Join us all week long for an exciting week of inspiring, innovative… https://t.co/xQY7lS2nmw
  • Learn what's new with Society activities during the SPS Town Hall! Join SPS President Ahmed Tewfik on Wednesday, 28… https://t.co/l5mKvqaskO
  • DEADLINE EXTENDED: The application deadline for PROGRESS at has been extended to Tuesday, 20 October. Det… https://t.co/pO5s3ief9t
  • On Wednesday, 28 October, join Stefania Sardellitti for a new SPS Webinar, “Joint Optimization of Radio and Computa… https://t.co/yGKACShQzu

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar