Liminal spaces between digital and analog

Audiovisual laptop performances has matured during the past 10 years into a well defined art practice, with a set of conventions and boundaries. Mia Makela (a.k.a Solu), a practitioner of live cinema based in Barcelona, identifies in live laptop-based performance the following basic elements [1]:

1. Space: the location where the performance occurs, shared between the public and the performer/creator.
2. Time, since real-time performance is an event that takes place in a specific moment in time and has a certain duration.
3. The performer/creator, who operates the computer (and other instruments as well) in order to create the images (and perhaps sounds) that define the performance.
4. Projection: the area used to display the images as well as the means of projecting these images.
5. Public: the audience that participates in the performance (and affect it by its own presence and potential explicit participation)

These elements are very broad, and provide an general framework where to situate very different types of digital audiovisual performances. There is an extra element that could also be added to this list, which is the “digital audiovisual substance” that Golan Levin refers to in his thesis essay. He considers specifically this “substance” in the context of his own work with painterly interfaces for live performance:

“To meet the goals stated above, I introduce a new interface paradigm for audiovisual performance instruments. This metaphor is based on the idea of an inexhaustible, extremely variable, dynamic, audiovisual substance which can be freely “painted,” manipulated and deleted in a free-form, nondiagrammatic context. According to this scheme, a user creates gestural, painterly marks in a two-dimensional input field, using an electronic drawing device such as a Wacom tablet or mouse. These marks are treated as the input to digital signal analysis algorithms, filtering algorithms, and computer simulations. The outputs of these algorithms are then visually interpreted by a graphics synthesizer, and also sonified by an audio synthesizer.” [2]

However, the concept of a digital substance could be extended to all varieties of laptop-based digital performance. This substance is, ultimately, the stream of bits that is processed by the computer in different ways at each single moment during the execution of the performance tool. This stream can originate in a video file, a live feed from a camera or in a generative algorithm. But irrespective of the origin, the stream can be reinterpreted and reprocessed an infinite number of times (the computer is, after all, a universal Turing machine, and by changing the code, the meaning of the bits change as well). Understanding bits as pixels allows to compose multiple layers of image effect kernels onto a source video stream. The result could be entirely unrecognizable, and nevertheless is made of the same substance (which can be further modified or casted under a different representation) as the original video.

This substance remains in the digital space, even when projected onto the physical world. The projection is “harmless” and “safe”, in a way, since it is completely contained and bounded inside the computer. We do manipulate this substance, but only from the outside, and even if we also provide its sources (live video), once they become digital substance, they are in a way out of reach. There is a division between the physical or analog realm and the digital plane. The public can be surprised, amazed or disgusted by the projected image, but they know it is just the result of a process taking place inside the computer.

The role of the live performer needs to be discussed here, since the last paragraph seems to be ignoring her altogether. This is a delicate issue, in fact, and it relates with the question: “how does the public know that what they see is either a truly live visual performance or just a DVD playing a movie?”. Some visual performers stress the live characteristic of their act by adding a second screen or projection showing him or her operating the laptop. Perhaps it can argued that a live performance succeeds as such (and not just as a visual piece) when there is no need to stress its “liveness”, when the public “connects” with the performer in such a way that there is something else that doesn’t exist entirely inside the computer, nor it is completely outside of the digital domain. Besides the basic 6 elements that constitute a digital audiovisual performance (the first five pointed out by Mia Makela, with the 6th being Levin’s digital substance), we can suggest the existence of an aditional element that crosses the boundaries between the digital and analog (physical). The existence of this seventh element is contingent to the success of the performance as a live event, unique in time and space.

I will argue that this 7th element can still be generated within the boundaries of “conventional” audiovisual practice such as VJing, but it requires a highly talented performer in order to reach this element, and break the boundaries of the genre from with its own accepted tools. The performative skills of the artist alone are able to break the separation between the plane of the digital image, the space of the physical bodies and the sphere of subjective perception of the audience. I will also argue that this 7th element can be introduced in an explicity way, by creating highly intertwined relational systems between the real/analog space and the digital plane. The intent of this approach is not to replace the skilled/talented laptop performer by an automatic system, but to point out possible ways in which the boundaries of the live audiovisual performance practive could (and are) being expanded.

As pointers to the nature of this element, let’s think in the following situations:

1. a girl in a club dances in front of an image of herself, captured with a camera installed in the dance floor, while playing with geometric figures that are digitally attached to her body and track her motions;
2. an audience performs with an interactive art piece that digitally augments their hands by projecting puppets that are controlled by their gestures. While these actions take place simultaneously in the real, “analog”, space, the digital image space (projection) and process space (computer memory, capture devices, etc), the resulting experience has a “tangibility” that escapes these three independent planes considered in isolation.

In which ways a tighter integration between the analog and the physical can generate this new space? By hiding the (digital) technology and accentuating the perception of the digital image as a “magic” occurrence. As an example of this approach, we can take the installation piece “3 minutes²” by the French art group Electronic Shadow:

“3 minutes²” (2004) is an installation mixing space and image, real and virtual and proposes a hybrid habitat metamorphosing endlessly around it is inhabitant. No screens, no visible interfaces, the two characters touch the walls, make movements, the habitation responds to them. The technology has become totally invisible and the effect of technologic becomes then magic. ..Parallel realities melt in one space-time, 3minutes are a space, 10 square meters are a space. 3 minutes² is a surface-time, a hybrid space living according to it is inhabitant’s rhythm, his trace, his electronic shadow.” (excerpt taken from the project’s website)

On the opposite end of the spectrum with regards to the way technology is shown (or not shown) and its relationship with the actual message of the piece (the technology is the message, paraphrasing Marshall MacLuhan), we find works such as the series of performance pieces by film maker Pierre Hébert and composer Bob Ostertag, “Living Cinema”:

“Although technologically intensive, Living Cinema does not celebrate technology but questions it, and its relation to the bodies of performers and the world around us, and, of course, garbage. Today’s cutting edge technology is of course tomorrow’s garbage. This paradox is not banished to the shadows as an unspoken embarrassment, but is rather the starting point of the entire project. Ostertag and Hébert sit on stage and try to sense out of science and garbage — a more explicit rendition of the situation we all live in every day.” (excerpt taken from Bob Ostertag’s website)

The use of multiple media sources, and in particular “traditional” media such as pain, watercolor, crayons, as well as physical devices and materials (machines, found objects, etc), is a direction already adopted by many artists, such as the collective AVCENTRALEN, video artist Sue C. and media artist Julien Maire.

I’d like to briefly focus the discussion into the technique of rotoscoping, widely used in traditional animation since the early twenty century. As noted in the wikipedia entry, “…Rotoscope output can have slight deviations from the true line that differ from frame to frame, which when animated cause the animated line to shake unnaturally, or “boil”. Avoiding boiling requires considerable skill in the person performing the tracing, though causing the “boil” intentionally is a stylistic technique sometimes used to emphasize the surreal quality of rotoscoping, as in the music video Take on Me…” (and specially so in the feature film Waking life). Recent advances in computer graphics would allow to do realtime video rotoscoping during live performances.

These different examples try to illustrate the idea of a new, intermediate space between the digital and the analog. I think space is what makes a live performance using digital projections more than just the sum of its parts. The concept of “liminal spaces”, trying to describe this intermediate, in-between, space has been proposed in the context of digital theater:

“The conjunction of live performance and digital imaginery can produce a particular, hybrid form and experience akin to what Alain Virmaux has described in relation to Artaud’s film scenarios: “something which is neither theatre nor film, but partakes of the evanescent reality of dreams.” This sense of in-between-ness – a liminal space operating between the screen images and live performers- is often the essential kernel, what one might even call the “metatext” of digital theater production. Margaret Morse, Elizabeth Grosz and Sarah Rubidge have each argued that the idea of the space “in between” is central to the artistic forms and spectator experiences of (respectively) video installations, architecture and immersive artworks. What is fundamental is not so much the material object or environment, but rather the space it occupies and the dynamic spaces configured and exprienced by visitors and spectators in relation to it. As we will see, this also holds true for digital theater spectacles, where the stage space becomes transitional, always in a state of flux, and, in Elizabeth Grosz words, “always in the process of becoming but never realised… the space of the in-between.” [3]

I think this idea of liminal spaces captures the esence of the proposed 7th element in audiovisual performance, the in-between space that connects the digital and analog in live performance. Hence, here we propose the following extended list of basic elements of live audiovisual performance:

1. shared performance space
2. performance (real)time
3. performer/creator/artist
4. projection plane
5. public
6. audiovisual substance
7. liminal in-between space

A exceptic reader might point out that 6th and 7th are just fancy labels for “bits” and “subjectivity”, but this is left open for discussion.

References
[1] Live Cinema: Language and Elements. Mia Makela. MA in New Media Thesis, Media Lab, Helsinki University of Art and Design, 2006.
[2] Painterly Interfaces for Audiovisual Performance. Golan Levin. Master of Science in Media Arts and Sciences Thesis. MIT, 2000.
[3] Digital Performance: A History of New Media in Theater, Dance, Performance Art, and Installation, Steve Dixon, The MIT Press, 2007, pp 337