Releasing SLP Dataset: To Learn How We Have Been Sleeping

Releasing SLP Dataset: To Learn How We Have Been Sleeping

The positions we take in our sleep could reveal important factors related to our physical/mental well-being as well as affect the symptoms of conditions such as sleep apnea, bedsores, or even carpal tunnel syndrome. Moreover, patients are usually required to maintain specific poses after certain surgeries to get a better recovery result and during pregnancy, pregnant women are recommended to avoid certain sleeping postures that could cause harm to the fetus. Therefore, long-term monitoring and automatically detecting in-bed poses are of critical interest in healthcare. Currently, besides self-reporting by patients and visual inspection by the caregivers, in-bed pose estimation methods mainly rely on the use of pressure mapping systems, which are expensive and hard to maintain.

Our goal in this research is enabling the use of advanced computer vision techniques in order to tackle challenges faced when sleep pose monitoring is desired. Although in the last decade, computer vision has been extremely successful in employing deep learning for many visual perception tasks, its use is still hindered when it comes to practical medical applications. This shortcoming is due to the factors that are shared in the majority of the medical applications, including: (1) lack of enough sample data and the cost of data gathering and labeling, (2) patient’s privacy concerns, (3) patient’s safety and comfort consideration, and (4) accessibility and unobtrusiveness of the data collection/monitoring technologies.

Assistant Professor Sarah Ostadabbas and her team in the Augmented Cognition Laboratory (ACLab) at Northeastern University received a grant from the National Science Foundation to build a dataset of sleep positions and use it to train computer vision algorithms to accurately recognize the pose of a sleeping person in a bed. They recently shared both the dataset and their algorithms on the ACLab website, to allow other researchers to use and expand on their work. This first-ever large scale dataset on in-bed poses are called “Simultaneously-collected multimodal Lying Pose (SLP)” (is pronounced as SLEEP). The lead PhD student of this project, Shuangjun Liu has combined different sensing technologies with machine learning techniques to monitor a sleeper’s position even under a full cover and in total darkness. His work could make the sleep monitoring easier for healthcare providers and less invasive for patients.

To build the dataset, the team recruited more than 100 people from the Northeastern community to come to their lab and lie down in various sleeping positions on a hospital bed or a regular twin size bed. The researchers collected several types of data from each participant, with and without a sheet or blanket, using a sensor mat under the person to take the contact pressure readings, an overhead depth sensor, and two cameras that took RGB and infrared images. Each session took about two hours. At the end of it, we have compiled SLP dataset with more than 14,000 pose samples that are fully labeled. ACLab would also like to acknowledge PhD student XiaoFei Huang, and Ms students Zhun Deng, Zhilan Li, Cheng Li, and Zhongnan Su, who contributed significantly in the SLP data collection and labeling.

By effective monitoring human pose under darkness and blanket, our SLP dataset and its associated tools can directly be employed in several healthcare applications such as long-term sleeping behavior studies.