IEEE SPS New Course: Transformer Architectures for Multimodal Signal Processing and Decision Making

IEEE Signal Processing Society Launches New Course:
Transformer Architectures for Multimodal Signal Processing and Decision Making

The Education Board of the IEEE Signal Processing Society (SPS) has introduced a new short course on the IEEE Learning Network (ILN): Transformer Architectures for Multimodal Signal Processing and Decision Making.

Transformer neural architectures have emerged as the model of choice in natural language processing (NLP) due to their high performance in various tasks, such as machine translation, sentiment analysis, and text summarization. Transformers aim to replace hand-crafted features with general-purpose neural architectures powered by data-driven training. Their success has been extended to multimodal learning, decision-making, and even computer vision. Past and current advancements suggest great potential for Transformers and significant opportunities for the signal processing (SP) community. This 7-hour course aims to provide timely insights into these cutting-edge developments.

The course material was prepared by award-winning international experts in Transformers, Dr. Chen Sun, assistant professor at Brown University, and Dr. Boqing Gong, research scientist at Google. The content was edited by the IEEE SPS Education Board's Content Production Committee, including Drs. Panagiotis (Panos) Markopoulos, Stefania Colonnese, Kejun Huang, and Mayur Dhanaraj.

In this course, students will be given the opportunity to:

• Familiarize themselves with self-attention and other building blocks of Transformers, the vanilla Transformer architecture, and its variations.
• Explore Transformers’ applications in computer vision and natural language processing, including ViT, Swin-Transformers, BERT, and GPT-3.
• Understand supervised, self-supervised, and multimodal self-supervised learning algorithms for training a Transformer.
• Acquire visualization methods to inspect a Transformer.
• Learn advanced topics such as related neural architectures (e.g., MLP-Mixer), applications in visual navigation, and decision Transformers.

This professionally produced video course, the first in a series, provides a practical guide to understanding and applying Transformers in speech or computer vision projects. It offers comprehensive insights into the foundational concepts and advanced techniques of Transformer architectures, equipping students with the knowledge and skills to leverage these powerful models in their own work. The IEEE SPS Education Board is currently working on delivering more high-quality short courses on various topics of SP interest.

Roxana Saint-Nom, Chair of the IEEE Signal Processing Education Board, expressed her enthusiasm for the new course. "As the Chair of the IEEE Signal Processing Education Board, I am proud to lead this effort, which reflects our commitment to providing curated, top-quality educational content to researchers, students, practitioners, and IT developers," said Saint-Nom. "The Signal Processing Society will continue to create in-depth technical content, with the Education Board ensuring it is accessible through the Resource Center and the ILN. We are dedicated to advancing knowledge and fostering innovation within the community," she added.

Upon completion of the course, students will earn a Certificate and 1 CEU/10 PDH credits, enhancing their professional credentials and demonstrating their expertise in this cutting-edge technology.

The course can be found on IEEE ILN.

All interested individuals are encouraged to take advantage of this opportunity to expand their knowledge and skills.