The proposed presentation will address a critical challenge in modern AI: adapting foundation models to individual user needs efficiently. As generative AI systems with large numbers of parameters become ubiquitous, personalization is essential for practical deployment across diverse applications. This talk will be highly relevant to ICASSP attendees, bridging signal processing, machine learning, and efficient algorithm design.
The technical content will cover parameter-efficient fine-tuning techniques, with particular focus on Low-Rank Adaptation (LoRA). I will explain how freezing foundation models and training only low-rank adapters enables cost-effective personalization. The presentation will detail LoRA implementation strategies, including for text, speech and image generation applications, demonstrating how these methods achieve state-of-the-art results with minimal computational overhead.
Beyond single-task personalization, a key frontier is enabling models to handle multiple specialized tasks efficiently. The talk will introduce advanced adapter merging techniques that address this challenge by combining multiple task-specific LoRA adapters into unified models. I will present three complementary research contributions from our lab. 1) Compositional Multi-tasking (EMNLP'25), which merges adapters for complex operations like translated summarization in a single inference pass. 2) LoRA.rar (ICCV'25), which uses hypernetwork-based merging to combine subject and style adapters for image generation with 4000x speedup. 3) D2C (ICASSP'26), a data-driven clustering method that identifies suitable groupings of task-specific adapters using minimal number of examples. It then merges adapters within each cluster to create compact multi-task adapters deployable on resource-constrained devices.
Such research direction is motivated by the practical limitations of current foundation models: full fine-tuning is computationally prohibitive, while ""one-size-fits-all"" models fail to capture individual user preferences and task-specific requirements. The need for efficient, personalized AI is particularly acute in resource-constrained environments where memory and compute are scarce, yet users demand sophisticated multi-task capabilities.
This talk will inspire the signal processing community by showcasing how efficient parameter adaptation techniques can democratize access to powerful AI systems. Attendees will gain practical insights into LoRA-based personalization, understanding adapter merging strategies, and applying these methods to real-world applications. The presentation will balance theoretical foundations with empirical results, providing both researchers and practitioners with actionable knowledge for advancing personalized generative AI.
