SPS Webinar: MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer
Date: 14 May 2025
Time: 8:00 AM ET (New York Time)
Presenter(s): Dr. Wei Tang
Based on the IEEE Xplore® article:
"MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer"
Published: IEEE Transactions on Image Processing, July 2022.
Download article: Original article will be made publicly available for download on the day of the webinar for 48 hours.
Abstract
Multimodal medical image fusion, an effective way to merge the complementary information in different modalities, has become a significant technique to facilitate clinical diagnosis and surgical navigation. However, existing deep fusion models generally depend on convolutional operations, which fails to preserve global context information. To compensate for this defect and achieve accurate fusion, the presenter will propose a multiscale adaptive Transformer to fuse multimodal medical images termed MATR. She will introduce an adaptive convolution for adaptively modulating the convolutional kernel based on the global complementary context. To further model long-range dependencies, an adaptive Transformer is employed to enhance the global semantic extraction capability. Their network architecture is designed in a multiscale fashion so that useful multimodal information can be adequately acquired from the perspective of different scales. Moreover, an objective function composed of a structural loss and a region mutual information loss is devised to construct constraints for information preservation at both the structural-level and the feature-level. Extensive experiments on a mainstream database demonstrate that the MATR outperforms other state-of-the-art methods in both visual quality and quantitative evaluation. She also extend the proposed method to address other biomedical image fusion issues, and the pleasing fusion results illustrate that MATR has good generalization capability.
Biography
Wei Tang received the bachelor’s degree in biomedical engineering from Wannan Medical College, Wuhu, China, in 2018, the master’s degree in biomedical instrument from Hefei University of Technology, Hefei, China, in 2021, and the Ph.D. degree in artificial intelligence from Wuhan University, Wuhan, China, in 2024.
She is currently an Assistant Professor in the School of Computer Science and Technology at Tongji University, Shanghai, China. Her current research interests include artificial intelligence, computer vision, image processing, information fusion, and biomedical image analysis.
Dr. Tang has first-authored several refereed journal articles, including IEEE Transactions on Image Processing, IEEE Transactions on Multimedia, IEEE Transactions on Circuits and Systems for Video Technology, and IEEE Transactions on Computational Imaging.
