LLM-assisted Metamorphic Testing of Embedded Graphics Libraries

Modern applications increasingly rely on embedded systems that incorporate visual interfaces developed utilizing socalled embedded graphics libraries. Verifying these embedded graphics libraries is challenging due to hardware dependencies and the lack of reference outputs. The lack of reference outputs is tackled in Metamorphic Testing (MT) by constructing two Firmware (FW) versions with distinct implementations that maintain the same input-output relationships. These relations are known as Metamorphic Relations (MRs). However, the development of these MRs remains a tedious and challenging task.
In this paper, we present a novel approach for generating MRs for MT of embedded graphics libraries using Large Language Models (LLMs). Because directly creating MRs with simple prompts is too complex for the LLM, we employ proven prompting strategies to develop our LLM-assisted MR pipeline. Strategies include role prompting, least-to-most prompting, zero-shot prompting, constraint-based prompting, and style prompting. In our experiments, we verify a widely used embedded graphics library. We compare our results with an existing manual approach and demonstrate that LLM-assisted MRs nearly doubles coverage and identifies additional bugs.