IEEE Signal Processing Society

Large language models(LLMs) have demonstrated increasingly powerful capabilities for reasoning tasks, especially in text. The project aims to explore and advance these capabilities in reasoning across multiple data modalities, including but not limited to text, speech and audio. The integration of multiple modalities can lead to more robust and general systems capable of understading and reasoning about the world in a more human-like manner. The project will involve fine-tuning pre-trained models and developing self-supervised learning techniques to adapt LLMs for multimodal tasks.