Using Large Language Models and Law-Based Rules over Extracted Knowledge Graphs for the Analysis of VAT Chain-Transaction Cases in Austrian VAT Law

Legal decisions must be reliable and unambiguous, which is why they must be made on an explainable and logically founded basis. Yet, this is precisely where large language models (LLMs) still have their difficulties due to their tendency to hallucinate. As a result, the EU AI Act classifies AI applications in the legal domain as high-risk, requiring additional governance and safeguards. In this paper, we present a novel approach that combines the strengths of LLMs in processing natural-language text and the possibilities of rule-based decisions into a robust application that produces logic-based and law-based results in the context of VAT chain-transaction cases in Austrian VAT law. The framework required for the structured preparation of the decision basis is formed by a knowledge graph, which is also used to generate a graphical representation. The application was tested with a total of 167 practical cases and an overall accuracy of 94% in solving cases of VAT chain transactions was achieved. Despite the 6% inaccuracy, the findings of this work make a valuable contribution to the practical application of LLMs in the field of VAT chain transactions, as even the visual representation of a complex issue brings efficiency gains for solving a chain transaction. In addition to the automatic resolution of chain transactions, the visual representation is therefore a valuable artifact resulting from the developed application.