SPS Webinar: An Anomaly Detection Framework with Compressed Transformer Architecture for Tiny ML
Date: 18-December-2025
Time: 10:00 AM ET (New York Time)
Presenter: Dr. Luca Barbieri
Based on the IEEE Xplore® article under the title “A Tiny Transformer-Based Anomaly Detection Framework for IoT Solutions”
Published: IEEE Open Journal of Signal Processing, November 2023.
Download article: Original article is open access and publicly available for download: ARTICLE LINK
Abstract
Transformers have recently revolutionized the artificial intelligence landscape, providing state-of-the-art performances in several tasks including anomaly detection. The latter feature is fundamental to monitor the status of industrial systems and promptly raise alarm whenever malfunctions are detected. However, transformers often entail huge computational complexity and memory requirements that are hardly available in tiny, low-computation, low-energy and battery-powered devices, calling for dedicated countermeasures to reduce energy consumption and memory footprints.
This webinar presents a compression framework tailored for transformer-based anomaly detection tools making them viable to be run on energy-limited devices. The presenter will describe how the anomaly transformer can be compressed using knowledge distillation, substantially reducing the number of training parameters of the model, hence its computational and memory complexity. Simulation and experimental results show the effectiveness of the developed compression framework, comparing its results against other baselines.
Biography
Luca Barbieri received the M.Sc. degree in telecommunication engineering and the Ph.D. degree in information technology from Politecnico di Milano, Italy, in 2019 and 2023, respectively.
Dr. Barbieri is currently a researcher with Nokia Bell Labs, Germany. From 2023 to 2024, he was a postdoctoral researcher at the Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB) of Politecnico di Milano. In 2022, he was a visiting researcher at the King's Communications, Learning & Information Processing (KCLIP) lab at King's College London, UK. His research interests lie at the intersection between unlicensed spectrum communications and AI/ML strategies.
