Data Challenges

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Data Challenges

(with available data challenges)
Past Challenges


Image, Video, and Multidimensional Signal Processing

2019

This challenge is the 4th annual installment of International Challenge on Activity Recognition, previously called the ActivityNet Large-Scale Activity Recognition Challenge which was first hosted during CVPR 2016. It focuses on the recognition of daily life, high-level, goal-oriented activities from user-generated videos as those found in internet video portals.

Habitat Challenge is an autonomous navigation challenge that aims to benchmark and accelerate progress in embodied AI. In its first iteration, Habitat Challenge 2019 is based on the PointGoal task defined in Anderson et. al. We will have two tracks for the PointGoal task:

  1. RGB track: input modality for agent is RGB image.
  2. RGBD track: input modalities for agent are RGB image and Depth.

The aim of this challenge is to solicit original contributions addressing restoration of mobile videos that will help improve the quality of experience of video viewers and advance the state-of-the-art in video restoration. Although quality degradation of videos occurs in various phases (e.g., capturing, encoding, storage, transmitting, etc.), we simplify the problem in this challenge as a post-processing problem.

The SUMO Challenge targets the development of algorithms for comprehensive understanding of 3D indoor scenes from 360° RGB-D panoramas. The target 3D models of indoor scenes include all visible layout elements and objects complete with pose, semantic information, and texture. Algorithms submitted are evaluated at 3 levels of complexity corresponding to 3 tracks of the challenge: oriented 3D bounding boxes, oriented 3D voxel grids, and oriented 3D meshes. SUMO Challenge results will be presented at the 2019 SUMO Challenge Workshop, at CVPR.

2017

Tremendous progress has been achieved in the way consumers and professionals capture, store, deliver, display and process visual content. Emerging cameras and displays allow for the capture and visualization of new and rich forms of visual data. A new activity of the JPEG Standardization Committee, called JPEG Pleno, intends to provide a standard framework to facilitate capture, representation and exchange of plenoptic content in omnidirectional, depth-enhanced, point cloud, light field, and holographic imaging modalities.



Information Forensics and Security

2022

In collaboration with WIFS, TU Delft CYS is organizing a competition in the domain of signal processing, combining biometrics and secure computation. Specifically, competitors are tasked to create a private odor-based access control system that matches encrypted human samples with permitted encrypted samples on an external database. 

Associated SPS Event: IEEE ICIP 2022 Grand Challenge

The perceptual quality of images/videos in the context of video surveillance has a very significant impact on high-level tasks such as object detection, identification of abnormal events, visual tracking, to name a few. Despite the development of advanced video sensors with higher resolution, the quality of the acquired video is often affected by some distortions due to the environment, encoding and storage technologies, which can only be avoided by employing of intelligent post-processing solutions.

2020

Steganography is the art and science of hiding data within media while its counterpart, steganalysis, refers to all methods that aims at detecting media used to transmit hidden data. Those two form a cat-and-mouse game since steganography aims at modifying the media to remain as stealth as possible and steganalysis aims at fight back by improving detection accuracy.  Most of the work in those fields are based on specific dataset and application in the “real world” of current state-of-the-art steganalysis techniques is hardly possible.

2010

The 2nd BOWS Contest (Break Our Watermarking System) was organised within the activity of the Watermarking Virtual Laboratory (Wavila) of the European Network of Excellence ECRYPT (http://www.ecrypt.eu.org/) between the 17th of July 2007 and 17th of April 2008. 



Machine Learning for Signal Processing

2022

Associated SPS Event: IEEE ICASSP 2022 Grand Challenge

The L3DAS22 Challenge aims at encouraging and fostering research on machine learning for 3D audio signal processing. 3D audio is gaining increasing interest in the machine learning community in recent years. The range of applications is incredibly wide, extending from virtual and real conferencing to autonomous driving, surveillance and many more.

2021

A new Challenge on self-awareness in heterogeneous multi-robot systems has been organized within the first International Conference on Autonomous Systems (ICAS 2021). The research field of this competition is the unsupervised anomaly detection through self-aware autonomous systems, which is an active topic involving IEEE Signal Processing Society, also through the Autonomous Systems Initiative, and Intelligent Transportation Systems Society.

3D audio is gaining increasing interest in the machine learning community in recent years. The field of application is incredibly wide and ranges from virtual and real conferencing to game development, music production, autonomous driving, surveillance and many more. In this context, Ambisonics prevails among other 3D audio formats for its simplicity, effectiveness and flexibility.

2020

The Clarity Project will be organising a series of machine learning challenges for advancing hearing-aid signal processing and speech-in-noise perception modelling.

2019

In this competition you can take on the role of an attacker or a defender (or both). As a defender you are trying to build a visual object classifier that is as robust to image perturbations as possible. As an attacker, your task is to find the smallest possible image perturbations that will fool a classifier.

In many real-world machine learning applications, AutoML is strongly needed due to the limited machine learning expertise of developers. Moreover, batches of data in many real-world applications may be arriving daily, weekly, monthly, or yearly, for instance, and the data distributions are changing relatively slowly over time. This presents a continuous learning or Lifelong Machine Learning challenge for an AutoML system.

In this competition, you are tasked with developing a controller to enable a physiologically-based human model with a prosthetic leg to walk and run. You are provided with a human musculoskeletal model, a physics-based simulation environment OpenSim where you can synthesize physically and physiologically accurate motion, and datasets of normal gait kinematics. You are scored based on how well your agent adapts to the requested velocity vector changing in real time.



Signal Processing for Communications and Networking

2023

In wireless communications, the pathloss (or large scale fading coefficient) quantifies the loss of signal strength between a transmitter (Tx) and a receiver (Rx) due to large scale effects, such as free-space propagation loss, and interactions of the radio waves with the obstacles (which block line-of sight, like buildings, vehicles, pedestrians), e.g. penetrations, reflections and diffractions.

2022

Associated SPS Event: IEEE ICASSP 2022 Grand Challenge

Localizing the root cause of network faults is crucial to network operation and maintenance. Significant operational expenses will be saved if the root cause can be identified agilely and accurately. However, this is challenging for human beings due to the complicated wireless environments and network architectures.

2021

Associated SPS Event: IEEE ICASSP 2021 Grand Challenge

In today’s digital age, network security is critical as billions of computers around the world are connected with each other over networks. Symantec’s Internet Security Threat Report indicates a 56% increase in the number of network attacks in 2019. Network anomaly detection (NAD) is an attempt to detect anomalous network traffic by observing traffic data over time to define what is “normal” traffic and pick out potentially anomalous behavior that differs in some way.



Speech and Language Processing

2023

Spoken Language Understanding (SLU) is a critical component of conversational voice assistants, requiring converting user utterances into a structured format for task executions. SLU systems typically consist of an ASR component to convert audio to text and an NLU component to convert text to a tree like structure, however recently, E2E SLU systems have also become of increasing interest in order to increase quality, model efficiency, and data efficiency.

The MADReSS SPGC targets a difficult automatic prediction problem of societal and medical relevance, namely, the detection of Alzheimer’s Dementia (AD). Dementia is a category of neurodegenerative diseases that entails a long-term and usually gradual decrease of cognitive functioning.

Verbal communication in noisy environments can be hard. Speech enhancement using head-worn microphone arrays, such as hearing aids or augmented reality devices offers the opportunity to make it easier. However, the highly dynamic nature of the listening situation presents some challenges.

The advent of spoken language processing (SLP) technologies on meeting transcripts is crucial for distilling, organizing, and prioritizing information. Meeting transcripts impose two key challenges to SLP tasks.  First, meeting transcripts exhibit a wide variety of spoken language phenomena, leading to dramatic performance degradation.  Second, meeting transcripts are usually long-form documents with several thousand words or more, posing a great challenge to mainstay Transformer-based models with high computational complexity.

The LIMMITS’23 challenge on LIghtweight, Multi-speaker, Multi-lingual Indic Text-to-Speech Synthesis is being organized as part of the Signal Processing Grand Challenge track at ICASSP 2023. As a part of this challenge, TTS corpora in Marathi, Hindi, and Telugu datasets will be released. These TTS corpora are being built in the SYSPIN project at SPIRE lab, Indian Institute of Science (IISc) Bangalore, India.

The 5th DNS Challenge aims to motivate development of DNS models with great speech quality in presence of reverberation, noise and interfering (neighboring) talkers. DNS has gained momentum given new trends of hybrid and remote work in a variety of daily-life scenarios. Improving speech quality reduce meeting fatigue and improve clarity of communication. 

The Multimodal Information Based Speech Processing (MISP) 2022 Challenge aims to extend the application of signal processing technology in specific scenarios, using audio and video data. We target the home TV scenario, where 2-6 people communicate with each other with TV noise in the background. Our new tracks focus on audio-visual speaker diarization (AVSD), and audio-visual diarization and recognition (AVDR).

The ICASSP 2023 Speech Signal Improvement Challenge is intended to stimulate research in the area of improving the speech signal quality in communication systems. The speech signal quality can be measured with SIG in ITU-T P.835 and is still a top issue in audio communication and conferencing systems. For example, in the ICASSP 2022 Deep Noise Suppression challenge, the improvement in the background (BAK) and overall (OVRL) quality is impressive, but the improvement in the speech signal (SIG) is statistically zero.

This signal processing challenge is designed to get the latest advancements in speech enhancement applied to hearing aids. 430 million people worldwide require rehabilitation to address hearing loss. Yet even in developed countries, only 40% of people who could benefit from hearing aids have them and use them often enough, because they believe that hearing aids perform poorly.

2022

Associated SPS Event: IEEE ICASSP 2022 Grand Challenge

Recent development of speech signal processing, such as speech recognition, speaker diarization, etc., has inspired numerous applications of speech technologies. The meeting scenario is one of the most valuable and, at the same time, most challenging scenarios for speech technologies.

Listening in noisy reverberant environments can be challenging. The recent emergence of hearable devices, such as smart headphones, smart glasses and virtual/augmented reality headsets, presents an opportunity for a new class of speech and acoustic signal processing algorithms which use multimodal sensor data to compensate for, or even exploit, changes in head orientation. 

Associated SPS Event: IEEE ICASSP 2022 Grand Challenge

MISP Challenge 2021 has been accepted as a Signal Processing Grand Challenge (SPGC) of ICASSP 2022!Please refer to more details of ICASSP 2022 SPGC.

Associated SPS Event: IEEE ICASSP 2022 Grand Challenge

Noise suppression has become more important than ever before due to the increasing use of voice interfaces for various applications. Given the millions of internet-connected devices being employed for audio/video calls, noise suppression is expected to be effective for all noise types chosen from daily-life scenarios.

2021

Associated SPS Event: IEEE ICASSP 2021 Grand Challenge

Text-to-speech (TTS) or speech synthesis has witnessed significant performance improvement with the help of deep learning. The latest advances in end-to-end text-to-speech paradigm and neural vocoder have enabled us to produce very realistic and natural-sounding synthetic speech reaching almost human-parity performance. But this amazing ability is still limited to the ideal scenarios with a large single-speaker less-expressive training set.

2018

The 5th CHiME Speech Separation and Recognition Challenge (CHiME-5). The new challenge will consider the problem of distant multi-microphone conversational speech recognition in everyday home environments. Speech material was elicited using a dinner party scenario with efforts taken to capture data that is representative of natural conversational speech.

2016

This 4th CHiME Speech Separation and Recognition Challenge (CHiME-4) challenge revisits the datasets originally recorded for CHiME-3, i.e., Wall Street Journal corpus sentences spoken by talkers situated in challenging noisy environments recorded using a 6-channel tablet based microphone array. CHiME-4 increases the level of difficulty by constraining the number of microphones available for testing.

2015

The Multi-Genre Broadcast (MGB) Challenge is an evaluation of speech recognition, speaker diarization, dialect detection and lightly supervised alignment using TV recordings in English and Arabic. The speech data is broad and multi-genre, spanning the whole range of TV output, and represents a challenging task for speech technology. In 2015, the challenge used data from the British Broadcasting Corporation (BBC). It was an official challenge of the 2015 IEEE Automatic Speech Recognition and Understanding Workshop. 

2012

One year ago the 2011 PASCAL CHiME Speech Separation and Recognition Challenge considered the problem of recognising speech mixed in two-channel nonstationary noise typical of everyday listening conditions. Following the success of this challenge we are now organising a new challenge that, while keeping the same setting, extends the difficulty along two independent tracks: a larger vocabulary size and a more realistic mixing process that accounts for small head movements made while speaking.

2011

In 2006 the PASCAL network funded the 1st Speech Separation challengewhich addressed the problem of separating and recognising speech mixed with speech. We are now launching a successor to this challenge that aims to tackle speech separation and recognition in more typical everyday listening conditions. The challenge employs noise background that has been collected from a real family living room using binaural microphones. 



Autonomous Systems

2021

A new Challenge on self-awareness in heterogeneous multi-robot systems has been organized within the first International Conference on Autonomous Systems (ICAS 2021). The research field of this competition is the unsupervised anomaly detection through self-aware autonomous systems, which is an active topic involving IEEE Signal Processing Society, also through the Autonomous Systems Initiative, and Intelligent Transportation Systems Society.



Data Science

2020

Translational utility is the ability of certain biomedical imaging features to capture useful subject-level characteristics in clinical settings, yielding sensible descriptions and/or predictions for individualized treatment trajectory. An important step in achieving translational utility is to demonstrate the association between imaging features and individual characteristics, such as sex, age, and other relevant assessments, on a large out-of-sample unaffected population (no diagnosed illnesses). This initial step then provides a strong normative basis for comparison with patient populations in clinical settings. Detailed information. Website.

 

 

Pages

SPS Social Media

IEEE SPS Educational Resources

IEEE SPS Resource Center

IEEE SPS YouTube Channel