MoPDO: Mixture of PDE-Derived Operators for Parameter-Efficient Vision Fine-Tuning

Parameter-Efficient Fine-Tuning (PEFT) has become a widely used paradigm for adapting large vision models to downstream tasks. However, many existing PEFT methods rely on low-rank or linear transformations, which may restrict the flexibility of feature adaptation when downstream visual distributions differ from those seen during pre-training. In this paper, we propose MoPDO, a Mixture of PDE-Derived Operators for parameter-efficient vision fine-tuning. MoPDO reformulates visual feature adaptation as a lightweight mixture of PDE-derived operators, enabling frequency-domain modulation with structured spatial responses beyond standard low-rank adaptation. As an initial exploration, we instantiate MoPDO with three representative PDEs: heat diffusion, wave propagation, and Poisson’s equation. These PDE-derived operators induce different spatial response patterns in the frequency domain, while a lightweight layer-wise routing mechanism learns to balance their contributions across transformer blocks. To implement them efficiently, we derive a unified formulation based on the Discrete Cosine Transform (DCT), making MoPDO easy to insert into Vision Transformers as a plug-and-play adapter with only a small number of trainable parameters. Using MAE pre-trained ViT-B, MoPDO improves PEFT accuracy by up to 2.1% on VTAB-1K image classification with a comparable number of trainable parameters. Extensive experiments across supervised and self-supervised vision backbones demonstrate the effectiveness, efficiency, and adaptability of MoPDO, suggesting PDE-derived operators as a practical direction for parameter-efficient vision model adaptation. Code is publicly available at github.com/feufhd/MoPDO