Digital Performance and the HyperMedia Studio: An Interview with Jeff Burke

 

Jason Farman

 

When the School of Theater, Film, and Television at UCLA added another area of study to its moniker—Digital Media—it sought to understand how new media can be incorporated into performance, influence performance, and ultimately become a performance. Part of this new agenda was the addition of the HyperMedia Studio, a research lab within the School that developed new technologies that could be applied across the fields of Theater, Film, and Television. Jeff Burke is the director of this studio and has been developing some of the most cutting-edge digital elements for live performance that are available to the performing arts. I had the chance to sit down with Jeff to discuss his history with the HyperMedia Studio, his thoughts on the convergence of digital media and performance, and the future direction of this important research group.

 

Jason Farman: When did the HyperMedia Studio get off the ground and what was your first large-scale production.

 

Jeff Burke: It would have been around 1997-98 when Fabian Wagmister started it, with the first large-scale performance in 1999 in Duisburg, Germany. This was a piece commissioned by the Wilhelm Lehmbruck Museum with Lynn Hersheman, who is a media artist out of UC Davis, called Time and Time Again. It was a series of artworks about the relationship between people, industry, technology, and communication in a region that saw its old industries of coal mining and steel mills fall apart as imports became cheaper. The piece distributed cameras in the coal mines and train stations that represented the old infrastructure and using the new, emergent telecommunications structures, we streamed these images into the museum space. As you moved through the space, sensors would detect your position. As you moved closer to the piece, the image would zoom in and as you moved away it would zoom out to aerial views of these old-industry spaces. Finally, there was a robot telepresent element that came from a lot of Lynn’s interest and work that placed a robotic doll in the space with cameras for eyes, streaming back out to the web this experience. The web interface allowed people to control what camera was being shown as well as where the doll was in the space. Conceptually, it had all of the components that show up both in Fabian’s later work and the Studio as a whole: awareness of the body as a means to explore media, the use of sensors to allow audience members and spectators to manipulate and interact with the media, and the networked connectivity aspect to the piece.

 

Jason Farman: I know your scholarly background is in Engineering. How did you make the leap into performance and join ranks with the HyperMedia Studio?

 

Jeff Burke: It’s actually not that much of a shift. I entered UCLA and did my undergrad and grad work in Engineering but along the way was doing the things I could do in the Theater Department. That was the best way for me to be doing both things at the same since there wasn’t really much in the way of cross-disciplinary stuff that was accessible at the time. Someone in the department introduced me Fabian, who, at that time had just started up the HyperMedia Studio. So I ended up working with him on a bunch of different projects starting in 1999. Though I didn’t initially think that I would get to do both theatrical work and engineering together, I had a faculty member here in the Department of Engineering who said, “You should look at these things together” and helped me design a graduate thesis project that tied together the performing arts and technology. That became the production of Macbett.

 

Jason Farman: How did you bring technology on to the stage for Mabett?

 

Jeff Burke: In this production of Ionesco’s text, we had the ability to locate the position the actors and tie their movements to control over lighting and sound. We looked to give the supernatural characters in the play some supernatural control over the environment by tracking their position through objects they carried. That set up our first large-scale performance experiment. All the while, we were looking to answer the question of how to make performance more flexible rather than more rigid. Because, especially at the time, high-end technology in performance tends to be about synchronizing the actors to some other track, whether video or some controlled machinery, things where they have to match their timing to the technology. So I was really interested in opening up the pallet to designers, but also considering how actors worked.

 

Jason Farman: Did the technology used in Time and Time Again get incorporated into your production of Macbett?

 

Jeff Burke: In starting with Time and Time Again and other pieces I had been working on with Fabian, we had a lot of interest in the use of sensors to give people more fluid access to media than they could get through the use of a mouse or a keyboard. Martin had provided their Lighting Director that was a sensing system used to track the position of performers. They had used it for rock musicians on stage. But at some point for us, just knowing where the actors were on stage wasn’t that interesting. So the next step was to process that data to know how the actors are moving—the types of things that directors might be interested in—is the actor’s movement strong or straight or curved? What were the characteristics of the actor’s position changes? The example I like the most of how it got used in this performance of Macbett was with the witches. We gave one of them the capability to conjure up wind in proportion to the movements of her staff for a spell-casting scene. We would turn this capability on and then, when she would move the right way in the scene, it would create wind according to her movements. The most successful part, I thought, was that because it was in proportion to her movements, she discovered ways to use slight movements of the staff outside of our original intent for the scene. She realized that she had control over the environment for that scene and she could use little wisps of wind to punctuate her statements or use it as a part of her character without having to think about triggering a cue or work with an operator. So, it changed her performance and gave her a new way of relating to the stage.

 
Production Photo of Macbett. Photo by Craig Schwartz.

Jason Farman: Based on this example, do you think actors inhabit stage space differently in the digital age?

 

Jeff Burke: There seems to be a change in the way that actors think about and rehearse in these spaces that are under their control. It puts them in a direct relationship with the director and the designers, and their presence becomes magnified and their control is magnified. They become, in some senses, a design collaborator. And that’s not so much about an expansion or contraction or change to the space as a real change in the dynamic of the creation of the performance.

 

In the case of media in general, it does bring a real change to the theatrical space. As soon as photographic elements were first introduced into performance in first part of the 20^th Century, they weren’t assumed to be manipulatable, but today we have a clearer understanding and very savvy audiences who know that media can be manipulated and still appear to be realistic. What we’re seeing is theater trying to capture in media some of the manipulation of reality that video and film have allowed. Reality, however, doesn’t seem to be the issue. The desire to create a photographic reality around the performance isn’t that exciting to me. This is limited theatrically. We default to a limited staging that presents these monolithic image surfaces (even if the images themselves are fragmented). We tend to think of visual media on stage as being these screens, and thus our formal experimentation of media on stage seems very limited right now. I’m not sure why. We have gone backwards to painted backdrop staging, just rethought from this point of view of projection. So we can do everything we want on the projection, but we’re forgetting that the solid surface has a reality for the audience and has an affect on the way the performance looks. We have missed 40 years of different kinds of staging that I hope we can rediscover and fracture media in a way that film can’t, and fracture the screen. This is also not very representative of how we encounter media elsewhere: many screens, very fragmented, very integrated into real space in devices we carry in our hands or in or cars.

 

Jason Farman: The screen interface does pervade our daily lives and seems to blend seamlessly with material interfaces. Does this blending of interfaces affect your ability to engage an audience who has experienced these multiple interfaces on a daily basis? Basically, I’m asking if the proliferation of interfaces affects audience interactivity?

 

Jeff Burke: We work a lot with people in engineering who are looking at the field of ubiquitous and pervasive computing, the idea that computers and computing devices will eventually be everywhere and that they will have these invisible interfaces and will disappear into the framework and the fabric of our lives. I’m a skeptic, not in the sense of how interesting those technologies are, but in the value and importance things disappearing and what that means. What we’ve focused on in performance, what is interesting to me, is that the audience becomes a professional user. So that rather than trying to simplify the relationship between the audience and the media, we try to open it in a way that there are many dimensions for navigating the given spheres of possibility within the media. The issue of interface thus is whether or not you are able to have a visceral connection with the media. Interface is finding these ways of giving dimension to the expressions of the body, whether it’s movement or voice, and translating those things into a similarly nuanced control over the media. Thus interface, for me, is having a continuum of control relationship rather than either the device trying to interpret something about your context or you having to make static decisions or use movements or techniques that have no physical relationship with anything going on anywhere.

 

Jason Farman: Another project you were involved with, called The Iliad Project, also seemed to rethink the notion of audience interactivity. You mentioned in an article that you wanted to “include audiences as an important element of the story.” How did you seek to accomplish this?

 

Jeff Burke: The audience interaction stuff that we’ve done has been more in the media instillation area. We’ve thought about some sort of dynamically shifting texts in theater and having implemented them in a full-scale theatrical piece. In those media instillations, the difference is that you have to give the audience/participants a sense of what’s available to them very quickly. With performers, the process allows discovery. In a museum context, you have so little time to acclimate people with what it is they’re interacting with and what the mechanisms for their interaction are. Thus, the key focus is how to not make the interface disappear but make the interface uncover itself so that it can disappear.

 

Jason Farman: With these forms of audience interactivity, do you see a connection to the tracking of the spectators to the emerging new media surveillance technologies? You used some of these techniques in the Iliad Project—can you discuss how that performance took place?

 

Jeff Burke: We protyped The Iliad Project, but what we wanted to figure out was how to create a dramatic structure that has an arc that is familiar enough to audience that they’re willing to sit through it and engage with it as if it was a play but also has a dynamic media construction. It was both a critique and exploration of the role of surveillance in creating the media that we see and the drama we see play out politically through the media. All of these things were played out in this performance. I think there are types of experimentation in performances that border on the audiences’ lives outside of the theater where there are kinds of interesting things that happen.

 

The audience would encounter the performance for the first time on the web. The website would give them information about the characters and the backstory—where is Helen, who took Helen, this inciting incident of the story. The second component was the financial transaction of them buying a ticket that linked them to a demographic—who they actually are. Then they show up to the performance. Maybe along the way they’ve been getting e-mails, giving them more information about what’s been going on leading up to the performance. They start to get engaged with what’s going on. Then they show up and get their ticket. That ticket has an RFID [Radio Frequency Identification] tag in it. Then they move to a gallery of photography about the Acheian cause. Now they’re in the first act of the play, which is their movement through the gallery of photographs about the horrors afflicted by the Trojans on the Acheians. What we would do is we would look at where people spend their time in the gallery. We would find out what people are looking at and slowly we would accumulate information. Then they would take their seats in something that feels like a typical performance. Through their RFID tag, we would know where people are sitting, so that actors could refer in their direction to them. A series of political arguments would unfold onstage and the media of that performance would be dynamically updated according to the particular audience. The idea is to incorporate as many of these little echoes, and we would slowly reveal the machinery as the performance continues. The idea is that the machinery gets inverted. You started out rooting for the Achieans, but slowly things get inverted and the audience sees the exact same machinery from a different point of view.

 

This technology is already in place in a popular media context. I mean, that’s what Amazon.com does, right? And so for it to happen in an entertainment context, it is already here but not done as often in a live presentation.

 

We protyped this all the way up until what we thought was a workshop stage, but haven’t been able to push it past that. Things tapered off around the end of 2002. It was about a year and half development process. We got a lot done and learned a lot but it didn’t make it into the performance stage. We each ended up moving on to other projects and haven’t had the opportunity to pursue it further. I think we would if we have a venue.

 

Jason Farman: One of your most recent projects was The Blogger Project, directed by Mel Shapiro. How did the HyperMedia Studio get involved in this production?

 

Jeff Burke: Mel came to us right around the beginning of the calendar year, and was interested in incorporating media into his next experimental show. He was interested in tying together three areas: the first four books of The Iliad, the Iraq War, and blogging. I could see the through line between them and we talked a lot about war, about how war is seen, about the role of media in war, about epic posturing in this kind of crazy alternation between sex and the battlefield in The Iliad and how it played out in all those arenas. When we talked about it, one of the things that came us is how much our media portrayal of war looks like a video game. That set us off in a direction of thinking about actually creating a video game that would contain all of the visuals of the performance. We would use this not only as our own organizational tool, but also as an experiment. For us, because this is a research university and we see ourselves as a research group (and because we naturally want to do new things) we’re always looking for new angles to better understand a certain areas of media that we don’t or find a tool that are more flexible than what we already have. So this seemed like a good opportunity to investigate the area of video game design, an area that we hadn’t spent that much time working with at this point. We picked what we thought was the best-in-class video game engine by Unreal and worked with them to get a licensing agreement in place where we could both use their tool to develop the engine for the production but also be licensed to write extensions for it and to have access to pieces of the code to make the updates that we wanted. Mostly, we wanted to explore how we could incorporate video into that game realm, since most of things that Mel wanted had a video component to them.

 

We ended up doing the thing that I typically don’t like: we had a massive screen for The Iliad section of the performance. It was 32 feet high and 24 feet wide with a platform cutting across the middle of it. The individual pieces of the screen below the platform were rotating doors so that actors could come in and out through the screen itself. Typically I’m not that excited about large screens, but in regards to the middle part of Mel’s show that was an absurdist Iliad with all of its scenes of war and carnage, we wanted media to have this huge, overwhelming presence. We did do things a little bit differently so it didn’t seem like performing in front of a movie screen, such as adding the doors in the media and creating an aspect ratio that was tipped on its side. So, it seemed as if the performers were performing in some other world.

 
Image of the Game World in The Blogger Project.

Jason Farman: Since this performance was called The Blogger Project, how was the media specificity of the blog conceptualized in order to translate onto the stage? How does a blog become performative?

 

Jeff Burke: Well, Mel had three sections in the show that were basically unconnected. He had a segment that was a series of blogs that people walked through as they entered. Those involved footage that we shot and live blogs that characters performed (such as Marie Antoinette, John Wayne Gasey, and other characters that the actors had developed from material on the Internet). The second section was the large-scale production of the first four books of The Iliad set in the game world. The third piece was a couple of real blogs of a soldier in Iraq and one from an Iraqi girl that were set in counterpoint with each other.

 

Reflecting back, one of the most interesting things about blogs is their anonymity. Even if the person identifies him or herself or has incredible amounts of personal information on there, it doesn’t matter because there is something about this mediated character of the Internet. There is something about the way the blogs are created that doesn’t exist in live blogs. So you’re seeing something that is apparently first-person that doesn’t have that veneer of being disconnected and it changes it a lot. In that case, it feels like a one-person show. The video segments of the performance seemed to capture that encased rant about something. It was nice in comparison to the live actors’ performance because it repeated over and over again. The fact that it still existed outside of that person actually having to be there was familiar and felt familiar like a blog. Seeing these run simultaneously felt like the web.

 

In a sense, having media and live actors in the same place put the actors up against something that ended up feeling more like a blog. Whereas if it had been ten all live actors repeating over and over again, I could see how the translation of the blog to the theatrical context. The third part of Mel’s show was a performance of the soldier’s blog and the Iraqi girl’s blog, performed by actors in the MFA program. The soldier’s blog was acted out by four male performers and the girl’s blog was performed by four women in a reading/documentary style. Structurally, I didn’t feel like it was connected to the specificity of the blog, but the parts that worked seemed to work because they were not overly edited, which characterized the material that people blog. But whether there was a tie to the media specificity, I don’t know and I’m not sure that was what he was getting at. I think it was a transposition to see what would happen instead of trying to create the equivalent to the blogging experience.

 

Jason Farman: Thinking of how online experiences can relate to theatrical ones, where do you envision online theatre going? Do you think the HyperMedia Studio will be involved in any performances that take place entirely in the online realm?

Jeff Burke: There are a lot of theoretical relationships between the media we are considering and the theatrical metaphors that show up in the works of people like Brenda Laurel and Janet Murray. You can clearly see experimentation in that area and this adoption of alternate personas, playing out of narrative scenarios, communal gathering to watch certain events or performances even in online spaces. For us, most of the focus has been on the relationship between the body and media. As far as dealing with online space and performance, I think the transition in our work here will be with non-professional performance that deal with a move toward a participatory scenarios in cities where we’re looking for people to document their own environments. The playback of such documentation would often have a theatrical component to it. Typically this would still involve physicality at the time of capture and at the time of presentation and communal audiences at the time of presentation. I would say as we go into those contexts, we are talking about a couple of different projects that use mobile and wireless devices in urban, public spaces for community exploration of their own identity and history. I would say that these are still going to have a physical component to them, but they will be similar to the online experience because they will be out of the theatrical venue and they will be driven by laypeople rather than theatrical professionals or those that follow the same organizational patterns that theater has at a university or an amateur theater anywhere. I think there is a lot in common structurally with how online performances become self-organized as these social networks that are around a game or narrative scenario. I think these are going to be very similar to online performances.

 

Jason Farman: What direction is the HyperMedia Studio taking at this time?

 

Jeff Burke: As I was alluding to, there is a slow and important shift in REMAP’s [Center for Research in Engineering, Media, and Performance] focus to be engaging with local communities and how the same interfaces to media and approaches to technology can be brought into the environment of communities documenting their own history and collective memories and do so outside of academic institutions but still locally in Los Angeles. We are already moving in that area. On one hand, we are working with Engineering to think about what the technologies are and find out how to leverage mobile devices. Also, we are rethinking what the research process should be to engage with these communities and not necessarily design a system until we are in the middle of this process. We are starting up a project with the National University of Singapore and Disney looking at participatory documentation as a part of cultural tourism (and the tools necessary for that). We are also taking a group to Bulgaria to be in residence and develop a new work over there. We are also looking to work with Mel for a production in 2008. So we have a couple of these production opportunities in line and figuring out how to incorporate the community-based research into these productions.

 

Jeff Burke is a researcher and lecturer in the UCLA School of Theater, Film and Television, where he directs the technological research of its HyperMedia Studio. Previously, he was on the faculty of the graduate industrial design program at the Art Center College of Design. He has co-authored, designed, programmed or produced performances and new genre installations exhibited in eight countries from 1999-2004, coordinating diverse teams for design and implementation. He has also managed, designed, or engineered systems for many theatrical productions. At UCLA, his interactive authoring course incorporates sensors, media control, programming and critical theory and has attracted students from education, engineering, computer science, architecture, theater, and film. He is currently a principal investigator of the Dept. of Film, Television and Digital Media's Advanced Technology for Cinematography project, funded by Intel Research, and NEA-supported research into authoring tools for media-rich ubiquitous computing environments.