Enhancing Real-Time Semantic Segmentation With Routing Attention and Boundary-Guided Fusion

Real-time semantic segmentation requires accurate prediction under strict latency constraints. Existing CNN-Transformer dual-branch methods combine a high-resolution CNN branch for spatial details with a low-resolution Transformer branch for semantic context, but they still suffer from limited context coverage in efficient attention and unreliable cross-branch fusion caused by feature-space inconsistency and spatial misalignment. We propose RBNet, a real-time semantic segmentation network with two novel designs for efficient attention and cross-branch feature fusion. To improve efficient contextual modeling, we introduce Head-wise Bi-Level Routing Attention, which enables each attention head to adaptively select relevant regions and aggregate more diverse context. To improve cross-branch interaction, we design an Alignment and Boundary-Guided Fusion module with Boundary Guidance and Interior Suppression loss to align CNN and Transformer features, correct spatial misalignment, and learn boundary-aware gates for reliable fusion. Experiments on Cityscapes and CamVid demonstrate that RBNet achieves a favorable balance between segmentation accuracy and inference speed.