Metamorphic Testing for Processor Verification: A RISC-V Case Study at the Instruction Level

Metamorphic Testing (MT) has been shown to be a very effective technique in the Software (SW) domain. MT does not require a reference model to compare against for testing but instead relies on Metamorphic Relations (MR) to derive the expected result from relationships between several calls to thefunction under test. An example of an MR is the expectation that the sum of an arbitrary list of integers remain unchanged regardless of it being sorted or reversed. Thus, a key requirement for applying MT effectively is availability of MRs specific to the domain at hand. In this paper, we propose MT to the domain of processor verification. As a case study, we consider the RISC-V Instruction Set Architecture (ISA) and provide MRs tailored for RISC-V. For evaluation purposes, we propose an efficient on-the-fly MT framework that integrates the MRs with an Instruction Set Simulator (ISS). We measure the quality of those MRs by the number of mutations they kill, also referred to as mutation analysis. Our experiments demonstrate the effectiveness of the MRs to kill all mutations, which confirms our research question that MT is also a suitable technique for the domain of processor verification.