Facilitating Logical Flaw Detection for Autonomous Driving Systems Through LLM-Empowered Oracle Generation

Currently, safety accidents and traffic violations related to ADS are frequent, posing a severe threat to road safety, which are largely caused by logical flaws within ADS. Unlike traditional software vulnerabilities, the logical flaws in ADS typically need to be identified through the detection of abnormal driving behaviors (e.g., traffic violations) that occur in scenarios. However, there is currently a lack of reliable automated methods to construct such violation oracles, hindering more comprehensive detection of ADS flaws. To address this gap, we propose LawSentry, an automated violation oracle generation framework to facilitate logic flaws detection in ADSs. Given that traffic laws provide the authoritative basis for EGO driving behaviors, LawSentry is highly motivated to generate executable violation oracles derived from these laws. Leveraging the potential of large language models (LLMs) in complex reasoning tasks and code generation, we introduce two key techniques, namely LLM-empowered traffic law understanding and LLM-empowered oracle code generation. Given a structured traffic law described in natural language, LawSentry leverages LLMs to automate the formalization of these laws and convert them into the corresponding oracle code, supported by a library of highly reusable APIs. In our evaluation, we successfully generated 55 violation oracles for traffic laws from four different countries and regions, which are executable on mainstream ADS simulation platforms (i.e., LGSVL and Carla). These generated violation oracles achieve a 95.5% accuracy in detecting traffic violations. Compare with the raw LLM-based approach (i.e., the approach without our prompt designs) and a learning-based baseline tool, LawSentry improves the 44.0% executable rate and 54.0% $\sim ~61.6$ % detection accuracy. All these significant improvements can be observed across different LLM families (e.g., GPT, DeepSeek, and Llama series). Furthermore, with the help of these oracles, we discovered $7+7$ previously unknown flaws on two stable versions of an industry-ADS (i.e., Apollo 7.0 and Apollo 8.0).