Multimodal self-supervised pretraining of causal sequence models for seizure detection and prediction

Epilepsy affects approximately 1% of the global population, with seizures that often occur without warning and vary widely between individuals. Reliable detection and early prediction remain critical challenges for improving patient safety, clinical response, and quality of life. Recent advances in deep learning - particularly the development of transformer-based foundation models - offer new opportunities to model complex biosignals and context information in a unified framework. However, existing approaches often focus on single modalities or task-specific architectures, limiting generalizability and clinical adaptability.
This thesis proposes a modular, multimodal model pretrained via self-supervised learning on a large clinical dataset of EEG and accompanying physiological signals, enriched with temporally aligned clinical markers and textual annotations. The model adopts a causal sequence transformer architecture designed to integrate EEG, ECG, SpO2, segment metadata, and contextual text into a shared tokenized input space. Training is guided by next-token prediction over a continuous embedding space, with auxiliary multitask objectives - such as time-to-seizure regression, segment classification, and marker reconstruction - serving to regularize and enrich the learned representations.
Experimental results demonstrate that the pretrained model captures generalizable and semantically rich features, achieving strong performance on both seizure detection and prediction tasks using linear probing and downstream evaluation. Input ablation confirms that contextual signals - especially time-past-seizure and clinical markers - considerably enhance performance. While not the primary focus, marker inference from biosignals proves feasible through similarity-based decoding in sentence embedding space, indicating representational depth and potential for generating annotations.
In conclusion, this work provides evidence that multimodal causal pretraining can yield transferable representations for seizure-related applications, supporting both interpretability and downstream adaptation. The approach bridges biosignals and clinical context, paving the way for generalizable seizure monitoring systems. Future research will need to extend this work to additional datasets, data acquisition and modality settings, and refine event-based evaluation strategies.