Handling Structural Mismatches in Real-time OperaTracking

Algorithms forreliable real-time score following in live opera promise a lot ofuseful applications such as automatic subtitles display, orreal-time video cutting in live streaming. Until now, such systemswere based on the strong assumption that an opera performancefollows the structure of the score linearly. However, this israrely the case in practice, because of different opera versionsand directors' cutting choices. In this paper, we propose atwo-level solution to this problem. We introduce areal-time-capable, high-resolution (HR) tracker that can handlejumps or repetitions at specific locations provided to it. We thencombine this with an additional low-resolution (LR) tracker thatcan handle all sorts of mismatches that can occur at any time,with some imprecision, and can re-direct the HR tracker if thelatter is `lost' in the score. We show that the combination of thetwo improves tracking robustness in the presence of strongstructural mismatches.