Evaluating LLMs on Formal Models Coursework

Beschreibung

The use of Large Language Models is on the rise in educational
settings such as creating teaching material, helping with
assignments or solving exercises. At the same time these new
possibilities bring new challenges in regards to the understanding
of the capabilities of LLMs. To offer a glimpse into the current
capabilities of LLMs in regards to solving coursework, we developed
a set of questions from our Bachelor's course Formal Models, which
allows us to rate and compare the generated answers of LLMs. We
developed an evaluation template to rate the responses, which allows
us to not only compare the end result of given answers, but also
intermediate steps and understanding of given tasks. The evaluations
are presented on an interactive website, which allows us to offer
new insights and enables a faster response to newly released models.

Zeitraum	02 Aug. 2025
Ereignistitel	ThEdu'25
Veranstaltungstyp	Workshop
Ort	Stuttgart, DeutschlandAuf Karte anzeigen
Bekanntheitsgrad	International