sound/tracks: Real-Time Synaesthetic Sonification of Train Journeys.

Peter Knees, Gerhard Widmer, Tim Pohle

Research output: Chapter in Book/Report/Conference proceedingConference proceedingspeer-review

Abstract

Travelling on a train and looking out of the window at the moving scenery reveals a composition of “visual music”with its own tempo and rhythm, its own colours and harmonies. The project sound/tracks aims at capturing these visual impressions and translates them into a musical composition in real-time - producing an immediate and unique soundtrack to the train journey based on the passing landscape. To this end, the outside impressions are captured with a camera and translated into instantaneously played back piano music. The immediately added sound dimension allows for reflection of the visual impression and deepening of the state of contemplation. For the resulting compositions, the passing scenery can be considered the score. “Re-transcription” of this score to an image gives a panoramic overview over the complete journey and exhibits some interesting effects caused by the movement of the train, such as compression and stretching of passing objects. In addition to intensifying the experience of a train journey, sound/tracks permits to persistently capture and archive the fleeting impressions of journey and composition and allows for re-experiencing the trip both visually and acoustically at a later point.
Original languageEnglish
Title of host publicationProceedings of the ACM Multimedia 2008 - Interactive Arts Program, Vancouver, BC, Canada.
Number of pages2
Publication statusPublished - 2008

Fields of science

  • 102 Computer Sciences
  • 102001 Artificial intelligence
  • 102003 Image processing
  • 102015 Information systems
  • 202002 Audiovisual media

Cite this