CBGF: Callback Coverage Guided Fuzzing

Abstract

Callback functions are widely used across programming languages, libraries, and operating systems. While offering flexible software design, these mechanisms introduce inherently complex execution flows that can serve as potential attack vectors for security vulnerabilities. Despite the continuous reporting and active exploitation of callback-related vulnerabilities in real-world environments, traditional coverage-guided greybox fuzzing approaches typically instrument the entire codebase uniformly without prioritizing callback functions, thereby insufficiently capturing callback-specific execution flows and limiting the effectiveness in detecting these vulnerabilities. We propose Callback Coverage Guided Fuzzing (CBGF), a novel approach that leverages user-defined callback functions to strategically insert instrumentation code, enabling precise monitoring of callback execution flows. The proposed instrumentation approach facilitates the comprehensive identification of callback execution paths while maintaining seamless compatibility with existing fuzzing frameworks. Our evaluation using CPython demonstrated the effectiveness of CBGF, by discovering seven bugs, including five callback-related vulnerabilities.