invisibility/corporeality

by Scott deLahunta

pdf (12 Kb)

The process of computation is invisible in the simplest sense that the labor of the software programmer or engineer is largely taken up in the 'writing' of an instruction that tells the computer hardware and connected peripherals how to execute (perfectly) an operation. (1) This writing and subsequent rewriting/editing is part of the creative process whereby something gets 'made' in terms of digital technologies. While some might decide to write code (or simply 'coding') for code's sake - generally this activity is done in order to enable something else to happen or get made. Where the primary creative activity is in relationship to the 'second order' activity can only be determined on a case/field by case/field basis... but it will vary (i.e. there is variation in this regards between the music field and dance field)

The other day I was having a discussion with someone about dance making and interactive systems and the 'transparency' issue as regards the receptivity of an audience to the aspects of the work that might be invisible. What is being considered *invisible* in this context is the mapping from input to various forms of output - and this mapping is essentially the consequence of someone providing the instructions for the computer, telling it what to do. Input in the case of BigEye occurs through the analysis of performer movement/action in a video image that then generates a stream of 'movement tracking' data. Thus performer movement/action is used to trigger some sort of event (sonic, visual, robotic, etc.) in the space around or in some proximity to the performer. The connection between the performer action that activates the stream of data and the output event is determined by 'mapping' the input to the output in the computer in some way. Christopher Dobrian (U. of CA, Irvine) expresses it simply: "The interpreted data provides information about the speed, direction, and location of moving objects in the video image, and that information can be used to provide input control data to music-generating software". This is essentially what is referred to as an interactive system. (2)

Mapping is at the heart of the creative process as regards these systems - which Marcelo Wanderley (researcher based at IRCAM) in a detailed presentation on interactive systems at ISEA in December 2000 pointed out. In a paper entitled *Towards a model for interactive mapping in expert musical interaction* (2000), Wanderley and Ross Kirk review the ways "performer instrumental action can be linked to sound synthesis parameters". (3) Their precise definition of 'mapping' uses the word to refer to the "liaison or correspondence between control parameters (derived from performer actions) and sound synthesis parameters. Within this they do not include in the concept of mapping the "actions related to data preparation, such as segmentation, scaling, limiting, etc." In the paper, they point out that generally two main 'mapping' directions can be derived from an analysis of past work: a) the use of generative mechanisms (e.g. neural networks) to perform mapping; and b) the use of explicit mapping strategies. For Wanderley and his fellow researchers in the field of electronic music, mapping is clearly a topic of immense creative interest and focus of artistic practice. However, it is the manifestation of mapping that enters the field of perception of the viewer/listener, not the mapping itself. Once completed, the instructions that comprise the mapping itself are relegated to the invisibility of computation. How this invisible mapping works or how it might work is of interest primarily to those who are engaged in its construction. [These last few sentences form the core of a polemic - which is completely refuted by the work of medical researcher Greg Kramer - see note 4 below]

There might be two directions for artistic work with interactive systems - One: towards allowing everyone audience/user access to all facets of the systems - input, mapping and output. Today, dance performances using interactive systems tend to allow an audience access only to the output, installations allow access to the input and the output - so why not include exposure to the mapping itself?

Another direction is towards the user/performer who might work with (practice) these systems in order to enter into a realm of higher level skill and 'virtuosic' activities within them. Combining input measurement that responds to a higher level of detail in performer action with more complex mappings. Wanderley/Hunt conclude their article by stating that "complex mappings cannot be learned instantaneously, but then again, we have never expected this from acoustic instruments". Assuming that the reference to learning can be seen as a reference to training - it begs the questions, where in the dance field do we discuss and debate notions of dance 'learning' (training/technique) overlapping with the development of interactive systems? There are a handful of practioners/artists (i.e. troika ranch/palindrome) whose efforts over time are accumulating richness and depth through personal determination and diversification // but their activities are focussed on artistic output, not training.

To return to this concept of the invisibility of computation - in relationship to interactive systems. I am curious about the long term outcome of creative activity that is proportionately shifting its centre of labor from the physical spaces to the virtual spaces. Any dance artist working with interactive systems will tell you that so far the amount of work involved in 'getting the technology' to work is immense and seems disproportionate to the amount of work done in the studio, perspiring and flexing. A shift away from the physical is by consequence in aesthetic terms a shift away from the formal and towards the conceptual... could we see audiences who develop a taste for mapping coming better prepared to watch/contemplate dancing in interactive systems?

Perhaps there is a future that holds both possibilities - where the invisibility of computation is displaced by its contemplation in the context of corporeality. Where 'interactive systems' are infiltrated increasingly by sweating/flexing bodies who spend more time in them sweating and flexing... while an audience/user's understanding of performance may increasingly see this body as one that is intimately involved with the invisible electro/digital spaces within which it performs.

NOTES

1) Writing is arguably not the best descriptive metaphor for software programming - building is preferred as often coding more often requires the reuse or reassemblage of previously written code.

2) For a very useful and straightforward breakdown, I recommend Dobrian's website 'video motion tracking for musical input' http://www.arts.uci.edu/dobrian/motiontracking/default.htm

3) You can download the Wanderley/Ross PDF here:
http://www.ircam.fr/equipes/analysesynthese/wanderle/Gestes/Externe/Hunt_Towards.pdf

4) For a completely radical and fascinating look at 'mapping' - take a look at this article on *Mapping a single data stream to multiple auditory variables* by Greg Kramer on the sonification of data from Radionuclide Ventriculography (RVG)-a non-invasive means for obtaining the blood volume change of the left ventricle. The article details how by working with complex mappings rather than a 'one to one' mapping auditory representations of a 'diseased heart' for example can become much more compelling to listen to - don't miss downloading the .wav files http://www.santafe.edu/~icad/ICAD96/proc96/kramer.htm

[Scott deLahunta began in the arts as a dancer and choreographer. Since 1992, as a partner of Writing Research Associates (WRA), he has organised several international workshop/symposia projects in the field of performance including recently the third session of Conversations on Choreography at the Institute for Choreography and Dance, Cork, Ireland. He taught theory and composition classes from 1994-1998 at the School for New Dance Development, Amsterdam and returns there to teach on occasion. In 1996, WRA organised Connnecting Bodies, the first conference in the Netherlands on the theme of the overlap between dance and emerging technologies. From February-May 1999, Mr. deLahunta was a guest professor with the Department of Dramaturgy, Aarhus University, Denmark where he was also co-organiser of the Digital Theatre Experimentarium, a project investigating the relationship between motion capture, animation and live performance. From 1998 to 2000, he was a consultant for the Laban Centre London on dance and technology applications and implementation. He is frequently invited to give presentations and contribute to publications on the overlap between dance and new media technologies at both small and large scale events such as DEAF (Rotterdam) and ISEA2000 (Paris). He is currently a Research Fellow and guest lecturer at Dartington College of Arts, UK where he is conducting research, also supported by the Arts Council England, into the conditions for collaborations between performing arts and applied science practitioners. In Autumn 2001, the WRA initiative *Software for Dancers* will conduct the first in a series of research labs/thinktanks looking to develop new software tools for performance artists.]