Scenario - Sonification

In the following there are various brief scenarios about sonification which is an umbrella term for the process of describing both real and virtual movement, images, spatial relations etc. with systematic sounds. For instance, when you notice that your car is making a funny noise and then realise that there might be something wrong with it you are using a sonification application. Here's a few examples of future potentials of sonification. Excuse us if the two first ones are a bit, er, business minded.


A sonification application could be used by a stock trader to track the movement of a particular stock -- say, Poutun Makkara -- against the some general index of stock-exchange. Application would give Poutun Makkara stock a certain tone on a computer, with the tone modulating in volume according to the number of shares trading hands. A second tone could be created for the general indec of stock exchange, with a different timbre that plays in tandem with the Pouttu's stock. Variances in the two tones, which are adjusted in real time as stock market data comes in, could be used to alert the trader to unusual activity.


Another good example where this could be used is in any industry with process controls. A chemical company could create a program that monitors the flow and temperature of certain chemicals and gives out certain tones. When something abnormal happens, the sounds coming from the system may alert a technician a lot faster than visual gauges or graphs. In particular, the technician is free to concentrate on other activities, relying on background audio to keep him or her informed about ongoing processes.


Sonification could also help us moving in virtual space. Think about an educational multimedia application about history of art. Let us think there is three main dimensions on it time (history), technical-artistic info and specific-general info. To make navigation easier we could make rythm of the bacground music to match the history dimension: the longer ago the faster the beats. Technical-artistic dimension could be matched to harmony: the stronger the harmony, the more the topic under discussion would concearn artistic (opposed to technical) topics. Specific-general dimension could me matched to the level of melody. The more specific the information on about history of art would be, the stronger the melody. Another version of this scenario would be helping the blind to hear where things (or rather their virtual representations are and what do they look like: Colours could be the tone of voice, volume the distance of the object inquestion, frequence wheter its on the left or right etc.


Check also out what these Sonification Designers at Computer Aided Surgery, Inc. are workig on:

"Our goal at Computer Aided Surgery, Inc. is to develop a special kind of sonification we refer to as "Tactical Audio". Sounds (really more like computer music) are employed to assist or enable the user surgeon to perform specific surgical placement tasks (specifically, blind needle stick into the body for neurosurgery) by hand (with sophisticated computer tracking) without ever having to take his eyes off the patient.

My inspiration came from my experience as a musician playing a fretless musical instrument such as a slide trombone and a violin. In these instances, the user positions their instrument (bow, fingers, slide?) precisely and accurately by ear in one dimension. Playing a note harmoniously, or 'in tune' requires accurate manual placement. Musicians do this quite well. Modern musicians can play western, diatonic music on the Theremin (e.g.: http://www.nashville.net/~theremin/) by moving their hands in space to achieve well defined notes. With surprisingly less training than you might expect, many people have learned to play scales and western music on the Theremin. With DARPA funding, we have built the worlds most expensive a 6 dimensional (X,Y,Z translations and Azimuth, Elevation, and Roll poise) digital

Theremin for surgical navigation application. In the Tactical Audio concept we turn the musical paradigm around: instead of using the hands to produce musical notes in tune, we use musical sounds to provide feedback to the user to produce precise and accurate surgical instrument placements. Indeed, we foresee the development from this emerging technology of "Applied Music" many uses in areas beyond medicine. We have an engineering application for musical principles completely beyond aesthetic enjoyment by an audience!

Dr. D. B. Karron

Chief Technical Officer, Computer Aided Surgery, Inc. (CASI)"