A Generic Machine Code Instrumentation Library

Dynamic machine code instrumentation is a powerful technique for analyzing and modifying running programs. This is crucial for performance monitoring, debugging and security analysis. However, with the lack of source code, creating robust instrumentation tools is challenging. It requires a deep understanding of the target architecture and calling conventions. Existing solutions are often platform-specific or lack the desired flexibility. This impacts companies such as Dynatrace that rely on such tools for application monitoring.
This thesis presents a generic C++ library, designed to simplify the dynamic machine code instrumentation process. The goal is a simple interface that allows users to attach entry and exit hooks to functions. The library handles disassembling of the target function, relocation of original instructions while preserving semantics and the generation of trampoline code. The initial target architecture is x86 with the ability to extend the library to support other architectures.
The resulting library lowers the barrier for developers, particularly in the context of Dynatrace, by providing a reusable and maintainable solution. The library simplifies the creation of tools that rely on instrumentation, and establishes a foundation for instrumenting further natively compiled languages such as Rust. This work ultimately aims to enhance the efficiency in development efforts and to improve capabilities for monitoring