Benchmarks and Performance Analysis of the LogicGuard Framework

This paper presents benchmarks, performance measurements and analysis for LogicGuard framework. The specification and data used to perform the benchmarks are all artificial and were designed to show various complexity levels. For this purpose, a set of parameters was defined to generate specifications and sample data, which include quantifier depth, search direction, windows size and delay. A set of tools was designed and implemented to perform the benchmarks and the analysis of the results. In addition to the LogicGuard software, a tool to generate specifications based the performance parameters, a tool to process the measurements and lastly a tool to analyse, extrapolate and plot the results were developed. The results presented show the resource cost in terms of processing time per message, number of instances and memory usage. To be able to capture the performance of LogicGuard specification as accurately as possible, the external function used for monitoring was designed to return as fast as possible such that the time needed for external processing can be neglected. Further analysis using polynomial extrapolation is applied to understand the behaviour of the framework.