This paper proposes a Model-Based Online Learning (MBOL) framework for waveform optimization in integrated sensing and communications (ISAC) systems. In particular, the MBOL framework is proposed to enhance the ISAC performance under dynamic environmental conditions. Unlike Model-Free Online Learning (MFOL) methods, our approach leverages a rich structural knowledge of sensing, communications, and radio environments, offering better explainability and sample efficiency.