This was done with an older version of Andiamo, together with a client application running on the phone (which is also on the trunk, btw). This work might continue in more refined/enhanced versions, but at this time is to early to say.

On the other hand, the current UI doesn’t look that bad, below you have a few screencaps of Andiamo running on OSX:

Video Layer showing the clip selection interface

Drawing layer with parameters window and animation selector

Audio layer

But I should also note that even if it looks decent, many hard facts from limited performance experience indicate limitations of the current interface. The MIDI input, of course, is of great help but it should be better synchronized with the GUI elements

So what is next? I would like to use the program as it is now to experiment a little with the combination of digital gestures and analog animation captured with the camera. This was in fact one of the first motivations for the whole MFA project that lead to this code. Time is always the big problem, but without the pressure of public performance events and taking it more as an informal experimentation, things could flow more easily. Maybe.

However, the crude nature of the piece in terms of materials and visual quality might have still played well with its main themes (childhood, playfulness, tragedy, death), and even the technical and synchronization glitches somehow became part of it. At least for some of the spectators. For others, these issues were disappointing. In any case, Latent State did prompt a variety of responses ranging from rejection to unsettling feelings. Disturbing by its errors, or by its own internal logic and contrast of images?

Something could come out of this

On the other hand, the coding work for this particular performance resulted in a new version of Andiamo that could serve as more stable platform for further experimentation than earlier versions of the code. In this sense, the performance at Make Art had a very concrete and positive outcome.

Now the question is: more coding, more practice or more error?

Maybe these guys have an answer.

Update: Some video material from the event is here.

So, one week of coding later and Andiamo looks quite nice, with a completely redesigned interface for the drawing and video layers. This UI was built with the excellent library controlP5 by Andreas Schlegel. A MIDI controlled can be also used to input certain parameters such as video mixing, volume, line speed, etc. The MIDI support was possible thanks to the Midi Bus library. I also evaluated the proMidi library, but this one crashed when the Scene selector button on the nanoKontrol was pressed, which didn’t happen with the Midi Bus.

There are still no install packages, but the source code is available in the sourcerge’s svn. The only missing feature with respect to the versions used originally in Latent State is the video tracking. At this time I didn’t find any satisfactory solution for detection and tracking of feature points, and unfortunately the proGPUKLT library doesn’t work on my MacLinux box (it crashes with some mysterious graphics driver error, and no time to look for a fix right now). The most promising direction to add video tracking seems to use the JavaCV wrappers for opencv 2 being developed by Samuel Audet.

Andiamo offers a number of gesture types, most of them adapted from the original yellowtail code from Golan Levin. Now, I just discovered the GML4U library by Jerome Saint-Clair which adds support for something called the Graffiti Markup Language. I don’t know much about this yet, but it seems this language allows to define custom gesture types by using xml-based files, with full support for animation and various transformations. So it could be a good idea to incorporate the GML in the future.

The very term framework by itself establishes the notion of control (a frame or skeleton, a system to contain and determine overall shapes and directions). In a more specific context, software frameworks might define several key components of the creative process: workflow, interface, and aesthetic decisions and outcomes. On one hand, it can be argued that frameworks are needed to facilitate, or just to make even possible, the access to the creative and expressive affordances offered by the computer, specially to the non-technical users. On the other hand, successful frameworks for digital creation are sometimes criticized as constraining the users to specific interface paradigms that narrow down the possibilities of digital manipulation.

This connects with the idea of computer literacy: the knowledge of the internal aspects of  computer function (not so much from the hardware point of view, but from the conceptual/semantic perspective). How much computer literate do artists need to be? The use of specific frameworks represents a certain level of computer literacy that allows the manipulation of specific software paradigms and metaphors. To keep with the literary analogy, the artist/user becomes expert in expressing herself in one style of prose or poetry (perhaps recent american short fiction or 60′s historical novel).

But computers, by virtue of being universal Turing machines, can be accessed only through specific abstract constructs: programming languages being the most fundamental. But even the most low-level language imposes its own semantic restrictions and allowances. Henceforth, the knowledge of computer programming as an advanced form of computer literacy won’t avoid being locked down in some form of literary style.

But after all, why all of this should matter? A pragmatic answer could be: as long as the software in question (be high-level audiovisual framework or lower level programming language) allows to solve the problem at hand, we should be fine.

But since anything we can potentially think of solving is implicitly defined by the language itself, we would probably be better off by keeping in mind what is lost and what is gained when we adopt a certain framework or language. At least, we should make sure we can express with it the subjective metaphors and even the contradictions that are part of our expressive intentions.

As for Andiamo, it just doesn’t constitute by itself a developed-enough framework to take as a concrete example in favor of any position. Given my current take on these questions, I would abandon it in favor of more lower-level constructs that still allow for some degree of “reusability”. But at the very least, Andiamo has proved useful (although only for me) to start exploring these questions. In this sense, it has fulfilled its goals as a platform for personal creative experimentation.

This time  I’ll be using a Macbook Pro 13″ as the laptop for the performance.  To be consistent with the FLOSS guidelines of GOTO10, I installed Ubuntu 10.04 on this machine, following the steps I described here. The software for the performance is implemented with the Processing language, using as the starting point bits and pieces of the previous application I created for Latent State, Andiamo. Input hardware: Korg nanoKontrol MIDI controller, Wacom Intuos 3 graphics tablet, and Unibrain Fire-i firewire camera in addition to the built-in iSight webcam on the MacBook.

To my initial surprise, all the hardware in the MacBook was recognized (even bluetooth and video output, which tend to be problematic to get working properly under Linux). A semi-random list of notes regarding the hardware/software configuration:

* The Ubuntu 10.04 64 bits installation DVD comes with the kernel 2.6.32-24. After a software update, 2.6.32-25 gets installed as the default kernel. However, brightness control (through pommed) doesn’t work with 25, so I just keep booting with the older 24, in which case everything works fine.

* I’m using Processing 1.2.1 with OpenJDK, the later installed from the package manager. I copied /usr/lib/jvm/java-1.6.0-openjdk to <Processing folder>/java, so replacing the Sun Java VM. Processing doesn’t seem to be very fond of OpenJDK, but after the warning it runs nicely:

Also, use of OpenGL in Processing under 64 bits Linux requires some additional tinkering with the library files, as detailed here.

* The GLGraphics and GSVideo libraries are used to handle OpenGL-accelerated graphics, and video capture and playback with GStreamer. I’m introducing some improvements and bugfixes in these libs as I go along, so newer installation packages better reflecting what I’m using for Make Art will be available in the coming weeks.

* To properly access the Fire-i camera, I had to allow the use of video devices for my user. To to this: go to System|Administration|Users and Groups, then choose advanced settings, and then go to the user privileges tab. Scroll down the list, and check the “Use video devices” option. Logout and login, and now the Fire-i should work from GSVideo using a pipeline with the dc1394src element:

pipe1 = new GSPipeline(this, “dc1394src ! ffmpegcolorspace ! video/x-raw-rgb, width=640, height=480, bpp=32, depth=24″);

* MIDI input either with themidibus library, or proMidi.

* Tablet input with the proTablet library.

* A little bit of computer vision to be described later

As you can see in the picture, the input/output capabilities in the MacBook are fully taken advantage of!

The thesis is divided in two major sections, Theory and Practice, one Appendix, plus a final Conclusion. The Theory section deals with the artistic and conceptual antecedents for this work, and I used most of the previous posts in this blog as the starting point for this section. The Practice section describes in detail the creative processes and final outcome of the performance pieces carried out as part of the MFA thesis exhibition – one at UCLA’s Experimental Digital Arts (EDA) space and another at the Museum of Jurassic Technology – as well as the personal background that motivated their creation. I also posted the documentation for these performances in previous posts. The video files can also be downloaded directly as mp4 files from this folder.

The final Appendix section serves as a reference for the custom software tool specifically developed for these series of performances, called Andiamo, and the related libraries I programmed to achieve real-time video playback and filter effects in Processing.

So, the chance of doing Latent State at the MJT made me very happy and grateful to David and Alexis. The Museum is in fact a very appropriate location for my piece, given the theme of “Latent State” and also the techniques I’m using in it. We decided to stage the performance in the Borzoi Cabinet Theater, located in the second floor of the Museum.  Many of the exhibits in the second floor (the space paintings, the Tsiolkovsky sketches) were a particularly good match. However, the space was quite different from the original setting in the EDA performance. I had no space for the multiple props I used back then, and this time I was restricted to a single projector/screen. This constrains in fact made me to focus in the central elements of the piece, and also to rethink the piece in terms of a single attention point.

The physical arrangement also turned to be quite interesting from a conceptual point of view: the stage area (with the new mini moving panorama I built specially for this performance) was located right below the cinema screen. So the “shooting” of the movie occurred (almost) in the same space as the projection of the resulting image. I think this a great example of the this idea of  open and closed liminality discussed in a previous post.

Here is short video containing some excerpts from the performance:

and here there is an extended version which, although edited, shows the entire flow of the narrative:

Finally I want to thank Tiffany Pan for designing the flier and passes for the event, and Michael Kontopoulos for helping me out with transporting the gear from UCLA to the Museum and back.

The first performance piece that resulted from this exploration is called “Latent State” and took place in the EDA space on May 14th, in the context of the opening of the DMA MFA 09 show.  An edited video from the performance can be seen below:

This piece posed a significant challenge with regards to the transition from the themes I wanted to address with it to the actual implementation, since the process of its creation run parallel to the development of the instrument used to perform the piece. In one of the first posts in this blog, I commented on the great advantage of being the developer the tool as well as its user. Although I think this remains true (new visual ideas can be tested almost immediately, understanding of the limitations and capabilities of the software and hardware informs the directions ), it is also true that complex software development and artistic creation seem to require different thinking patterns and creative strategies. This poses the following question: can be the creator of a tool become a proficient performer of it? A positive answer is probably contingent on constant practice and the ability of intertwining in a successful manner the technical and artistic creativity.

The piece ended up structured in three parts, each one based on a different technique: real-time animation in the first part, performance in the second, and live film in the third.

I would like to thank Johanna Reed for her feedback during the production of the piece, as well as for ther participation during the performance as the “benign entity”.

I’d like to draw an improbable arch spanning thousands of years. Let’s first consider the cave paintings from pre-historical man.

Today we are able to see this remains of ancient practices, thanks to the technical resources available to us. But I think it is reasonable to ask if these drawing-paintings were meant to be looked at after their creation, or they rather represent the outcome of a one-time ritual, being the enactment of this ritual (drawing the prey) the sole goal of descending into the almost unreachable, unlit areas of the cave. It is usually said that the cave paintings are among the first artistic manifestations of humanity (which left physical trace), however these artworks are paintings that cannot be seen. The left object is not important, but the act of its creation is.

During centuries the visual arts (in the western world) were focused in the final object that results form the creative process. The 20th century witnessed a dramatic change in this focus, towards the processual nature of art (for instance fuxus and happenings during the 60′s). The following two examples can still be inscribed in the realm of object-oriented art, but show the increasing importance of the gesture and the ephemeral. Jackson Pollock’s action paintings are probably more about the movements and gestures that the artist made over the canvas than about the resulting, static painting. These paintings seem to exist at a crux between the object and the gesture.

In Le mystère Picasso, a documentary by H. G. Clouzot released in 1956, we can see, among other things, the transformation in Picasso’s painting between a fish, a cock and finally a human face. This transformation is probably more fascinating that the final painting itself. Maybe part of this fascination come from the fact that he is Picasso, but we can see nonetheless live painting (or drawing) as a potentially new art form that combines performance (and even ritual) with older traditions and techniques of visual arts.

Live drawing certainly is also closely connected traditional animation. However, the fact that the line is created in “real-time” in front of the audience creates a number of unique narrative and visual challenges and opportunities. For instance, the act of drawing the line becomes the animation itself, or at least part of it.

Digital media greatly facilitates the practice of live drawing. Real-time capture and digitalization of video and hand gestures allow to build systems for real-time animation. The collaboration between experimental animator Pierre Hebert and sound artist/composer Bob Ostertag is a good example of the possibilities of live drawing/animation:

With the widespread accessibility of laptop computers, graphic tablets and low cost cameras, the practice of digital live drawing can be found in different parts of the world (look for example at Argentinean visual artist Marcella Rapallo or NYC-based British illustrator Shantell Martin). It is also interesting to note how the form of live drawing is interacting with other (audio)visual performative practices such VJ’ing or live cinema (film).

]]> http://naute.wordpress.com/2009/05/31/live-drawing-and-return-to-performativity/feed/ 5 ac Traful-cave-paintings pollock-in-action http://naute.wordpress.com/2009/05/29/sheepwoman-live-fil/ http://naute.wordpress.com/2009/05/29/sheepwoman-live-fil/#comments Fri, 29 May 2009 06:14:33 +0000 ac http://naute.wordpress.com/?p=162 ]]> Sheepwoman is a live audiovisual performance piece by Sue-C and Laetitia Sonami. Sue-C is a visual artist whose work mixes video and laptop performance with traditional media like watercolor and drawing. Laetitia Sonami is an experimental musician and composer, who created her own unique performance tool to sculpt soundscapes in real time: the Lady’s Glove. Hence, in Sheepwoman Sue performed the majority of the visual elements of the piece, while Laetitia was in charge of the sounds and music (although there was some overlap between the two artists). I had the opportunity to see two consecutive performances of Sheepwoman at the Resonant Forms festival at the LACE gallery in Los Angeles.

Sheepwoman opens up for discussion many relevant questions in the context of live laptop performance. First of all, I’ll start by noting that Sheepwoman was introduced as a “live film”. There is a seemingly equivalent term that has been growing in popularity lately, that of “live cinema”. A broad definition of live cinema proposed by Mia Makela is “the simultaneous creation of sound and image in real time by sonic and visual artists who collaborate on equal terms and with elaborate concepts”. I tend to shy away from definitions and their attempt to tabulate and digest artworks with accepted labels. On top of that, Sheepwoman took place at a venue where experimental and avant-garde (wherever that word means today) works have been exhibited since the late 70′s. Isn’t one of the goals of experimental art to defy conventions and definitions? Nonetheless, I began this discussion precisely with the issue of defining the piece because I think that the choice of the term “live film”, instead of the more popular “live cinema” or even “audiovisual peformance”, was very conscious and relevant to discuss the nature of the piece.

If we ask ourselves what are the physical spaces required to the (non-live) construction of a film, the following list might suffice in most of the situations: a set or stage where the action is shot with a camera and sounds are recorded, the editing room where the different scenes are combined together into a linear montage, and the projection screen (plus the surrounding space for the audience ) where the film is finally shown.

Sheepwoman in fact retains all these traditional elements: there is a miniature set where the camera moves around and the actual, physical action takes place and its shot. The editing room is now the computer, where the digitalized filmed material is stored, combined and processed into a linear sequence of audio and video frames (adding perhaps some non-live materials). Finally, the images are projected onto the screen and sounds played back through the speakers in the room where the audience has gathered to see the piece.

The nature and scale of these elements (sets, sound stage, editing room, projection screen) have changed, but their roles remain the same. There is one key difference, though, which justifies the term “live film”: all these elements exist in the same space and are used simultaneously, as opposed to conventional film where they occur sequentially one after the other (shooting on the sets during the production phase, followed by post-production editing and finally, projection at the movie theater). These considerations lead to the following definition of live film:

“Live film is a form of cinema where the sets and sound stage, editing room and projection screen share the same space and time.”

This proposed definition does not just restates the obvious, but it tries to re-interpret traditional filmmaking concepts by confronting with ideas of simultaneity and spacial co-existence. The digital doesn’t have a central role in terms of the building blocks of what live film is or could be, but it constitutes nonetheless the enabling technology that allows for this simultaneity of shooting, editing and projection: the digital real-time.

This idea of temporal and spacial co-existence of the sets and the projection screen connects with a concept that I discussed earlier, that of closed vs open in-betweeness, or liminality. Both the sets and the screen inhabit the same place but are not the same,  in the sense that the projection doesn’t occur on the set like in many digital dance performances, where even the body of the dancers become the screen. The screen and the sets are clearly delimited, and hence the liminality is determined by the space between them. More importantly, these gap is not hidden from the audience, on the contrary it becomes a very important part of the piece, and the gaze moving back and forth between the screen and the “real” object being filmed creates dialogical and narrative layers which I think are unique to this art form.

There is another element which I found extremely important in Sheepwoman and would like to explore in my own pieces, and it is the role of the laptop performer. While during most of the performance both Sue and Laetitia were sitting in front of their tools and controls, Sue used a flashlight to illuminate parts of the set to create very dramatic moments at certain points during the piece. Given the size and distribution of the elements on the set, this action was much more physically involved, requiring her to sit on the table and move around the set structures. Sue become then much more visible to the audience, abandoning the usual, protected role of a laptop performer.

I felt that, in a strange way, she turned into an actor in the movie, connected to the ongoing narrative not only by the effects her actions with the flashlights created on the screen, but as some character looking for some lost object in the space conveyed by the sets.

I think that this type of physical involvements and actions create new performative and narrative possibilities that extend the limits of live cinema and live film into more theatrical expressions.