Skip to main content

Adaptive Progressive Training With Hierarchical Prompt Loss for Weakly Supervised Video Anomaly Detection

By
Chen Zhang; Guorong Li; Laiyun Qing; Xinfeng Zhang; Qingming Huang

Video anomaly detection aims to identify abnormal events in untrimmed videos and is crucial for intelligent surveillance. Recent advances leverage vision-language models such as CLIP to incorporate category-level textual prompts, enabling both coarse-grained binary detection and fine-grained multi-class categorization. Despite their effectiveness, these dual objectives introduce two challenges: the lack of explicit semantic structure among learnable prompts weakens fine-grained discriminability, while the optimization conflict between the fast-converging binary detection task and the continuously evolving multi-class categorization task leads to unstable training dynamics. To address these limitations, we propose a weakly supervised video anomaly detection framework that improves both the semantic quality and the optimization robustness. First, we introduce a Hierarchical Prompt Loss that imposes structured semantic constraints on prompt embeddings, preventing abnormal subclasses from collapsing and ensuring that the normal prompt is well separated from all abnormal categories. Moreover, an Adaptive Progressive Training (APT) strategy is devised to dynamically regulate the learning process. Specifically, APT employs a Gradient-Aware Decoupling mechanism which monitors gradient norm and directional consistency of the two tasks to detect persistent task conflicts. Following conflict detection, the mechanism decouples the training objectives by stopping updates to the shared feature learning module and transitions the model to a Dependency-Guided Refinement scheme, where the binary detection task is first stabilized before multi-class refinement proceeds. Extensive experiments across various benchmarks demonstrate that our method outperforms existing approaches on both binary anomaly detection and fine-grained categorization.

Read on IEEE Xplore