A Cloud-Native Lakehouse Architecture for Using Knowledge Graphs in Aeronautical Information Management

A knowledge graph (KG) represents real-world entities as well as their properties and relationships in a machine-readable format, which can be employed in decision support systems across various domains. For example, in ATM, a KG can contain information about various aspects of the current state of the air traffic network, including infrastructure, important events, and flight trajectories. Traditional monolithic KGs face scalability challenges as the volume, variety, and velocity of the data increase, limiting real-time responsiveness. However, many applications require access only to subsets of a KG rather than to the entire KG at once. For example, a pilot briefing for a particular flight within Central Europe does not need information about the entire European air traffic network. Based on this observation, we propose a cloud-native data lakehouse architecture for KG management, optimized for ingesting and indexing large volumes of a variety of data arriving at high velocity. The main contribution of this paper is the design of a modular and scalable architecture for data ingestion and the on-demand generation of contextualized KGs from the ingested data. We further provide a proof-of-concept implementation using open-source technologies, demonstrated on a real-world use case of decision support in ATM. A comparison against a traditional monolithic pipeline shows that the proposed architecture achieves superior ingestion rates, with horizontal scaling further increasing the throughput.

Keywords: Knowledge graph management system, data lakehouse, big data, microservice architecture, contextualized knowledge graphs