Body Doubles:
Kinesthetics and Visual Display in the Aesthetics of Gallery-based Interactive Art

David McDowell

When viewing “static” visual art works such as paintings or sculptures in an art gallery, a spectator will often be aware of other viewers in the exhibition space and that reciprocally other viewers may occasionally be aware of him or her. However, a spectator of such works is less likely to be self-conscious of how he or she views other spectators, or how the process of navigation from work to work actually shifts his or her viewpoint in relation to the other spectators. Even though one’s viewing activity might be on display to other viewers, it is not part of the display of the works themselves. In contrast, interactive works installed in gallery settings often require interacting viewers to become part of such works in order to experience them and that while doing so will actually be part of the works’ display to other viewers. By regarding interactive works not only from the point of view of the private experience they may offer a participant-viewer but also from that of their public display, interactive art opens up for consideration the relationships between notions of space, the circuit of body, sensorium, technology, and the participant.[1]

1.

New media art often harnesses the capacity of the computer to run routines or programs to modulate the computer’s auditory and/or visual output. Interactive art requires input – from a participant – to trigger the implementation of programming constructs. From the computer’s point of view the input must register as an event – a change of state – that the computer is able to recognize; from the user/performer’s point of view the making of the input requires movement.

That movement may be simply a humble mouse click or the activation of more exotic input devices involving the use of sensors to detect movement. To be activated, sensors may require gross bodily activation, such as stepping on a pressure pad connected to a sensor, using a joystick-like device, making loud noises, or moving the body, to which sensors, the position of which can be tracked by detection devices, are attached. Other sensors are able to detect more subtle bodily perturbations: of the hand, of the eye, of pulse or the rise and fall of the chest during breathing. The computer will detect the individual input triggers but may be also instructed to register the frequency of input triggers and to react to that frequency in addition to or instead of the individual triggers.

Certain bodily motions require the volition of the participant. The participant will be conscious of the decision to make voluntary movements and at least some them as they are actually made (one might not register each individual mouse click or key press will playing a computer game, but one will be aware that mouse-clicking or key-pressing is happening when it does).

The motions needed to trigger events that the computer can register will, on this level, be purely instrumental. However, artists can make the need for these movements, and the fact that the user/performer will be conscious of making them, an integral component of the concept of a work and the experience it offers. Take a very well known work, such as Char Davies’s Osmose or Ephérème, for example: these present virtual reality (VR) environments are navigated by the “immersant” by moving his or her head and body and, through breathing, expanding and allowing to contract the chest/abdominal cavity.

Davies’s website makes a clear statement of how her works are intended to foreground embodied experience:

Immersive virtual space … can provide an intriguing spatio-temporal context in which to explore the self's subjective experience of "being-in-the-world" — as embodied consciousness in an enveloping space where boundaries between inner/outer, and mind/body dissolve.[2]

Davies surveys immersants – post-“immersion” – to gain feedback on the their experience of the works. The cumulative response confirms “the artist's belief that traditional interface boundaries between machine and human can be transcended even while re-affirming our corporeality.”[3] While not stated explicitly, the activity of navigating one’s body through the work is a very significant means of registering the corporeality of a body that, during immersion, cannot be seen by the immersant. To this extent such works are highly open to consideration in kinesthetic terms.

Not all bodily motions harnessed by an interactive work need to of the order of locomotion or muscular-skeletal articulation of head, limbs or hands and fingers. Movement is the primary sign of life; and this can be as subtle as a pulse or the rise and fall of the chest, airflow caused by breathing, or the dilation of the pupil of an eye. The detection of more subtle bodily movements within interactive works generally involves a direct attachment of sensors to the body, and perhaps even that the body be constrained to some degree.

The sensing of very subtle bodily animations may require that the body itself not engage in larger scale movement, in other words, that it be largely immobilized. Cardiomorphologies, by George (Poonkin) Khut, is an interactive work is which the heart rate and breathing depth and frequency of a participant is monitored and used to modulate visual and auditory output (Fig. 1).[4] The participant reclines on a couch/recliner chair. A heart rate sensor is held in either hand, a pressure sensor under the participant’s back detects movements associated with breathing. The participant’s body is not constrained as such, but brought to rest so that the more subtle vital movements can be monitored and so that, over a period of time and through the feedback provided by the visual and auditory output (seen and heard by the participant), he or she may become aware of variations in heart and breathing rate and the relation between these and state of mind. Khut’s website states that:

[h]is research focuses on the development of interactive sound and video environments in which listeners experience and learn to influence aspects of their own psycho-physiological (body-mind) processes, by learning to observe and differentiate changes in the sound, color and form of the artwork.[5]

 
Fig. 1. Installation view of Cardiomorphologies by George (Poonkin) Khut et al.
Image courtesy of the artist; photographer: Julia Charles

Perhaps having a less obvious kinetic component than most interactive artworks, Cardiomorphologies does, nevertheless, have a kinetic component that might be said to involve micro-movements. As such, this work may be also considered in kinesthetic terms. It would be easy to allow the amount of movement to be the measure by which an experience, and the artwork offering it, is considered more or less kinesthetic. However, “more kinetic” does not necessarily equate to the “more kinesthetic” (if this was the standard for kinesthetic works, then athletic or acrobatic acts of agility and endurance would be considered as primary kinetic activity).

When the input device of an interactive artwork is as mundane as a computer mouse or keyboard, the participant may not become particularly self-conscious of the movements he or she is making when using it. However, particularly in a public setting such as a gallery (in which, out of necessity, practicality, or artistic intent only one person might be able to experience a work at one time, as is often the case with interactive artworks that other gallery visitors can see and watch the person ‘undergoing’ the artwork) the participant often is quite aware that his or her presence and activity in the work are on display. Even when an immersant is wearing a VR headset and cannot see his or her public display, he or she may not lose full awareness of being “on show.” The artificial isolation of sensory streams from the spectatorial space may assist the participant in concentrating on his or her private experience, the installation view can, perhaps unintentionally, reveal part of the participant’s experience, thus leading to the participant’s (self-)conscious appearance in the work.

Likewise, even though Cardiomorphologies does not require that the participant move in ways of which he or she might become publicly self-conscious, the fact that, in the installation view (Fig. 1), the artist’s figure appears, situated well behind the participant and behind a console, indicates that there is a spectatorial space associated with the actual installation outside the participant’s field of view. This space would be known to the participant, who might then not entirely lose the awareness of being on view to unseen spectators. Additionally, the technical devices and the circuits, which connect input devices to monitors and processors and the other structural hardware that supports an interactive artwork, operate as an armature that encompasses the (necessarily) patient participant. That armature may not be fully on display – it may, for instance, include wireless connections – but it is a framework into which participant is situated and by which he or she may envisage him or herself being both embodied and enframed.

The installation view of Cardiomorphologies (Fig. 1) certainly completes that enframing by converting both the physical space of the installation (that of the experiential mapping of that space by the participant – a mapping that includes both the physical space and the bio-technical matrix that the participant is “within”) into pictorial space. The installation of Osmose or Ephérème at the Australian Centre for the Moving Image (ACMI), Melbourne in 2003/2004, definitely did turn the immersant’s body into and image for other gallery visitors, albeit as shadow.[6] The banner graphic of Davies’s website shows a photographic image of an “immersant,” either shadowed in this manner or against a backlit screen; on each side of this image in the banner is a VR “still.”[7] The silhouette of the immersant creates a distinctive impression, partly alien and partly human in an alien environment. The backlighting does not only accentuate the immersant’s form: it also accentuates the negative space surrounding it. Only the umbilical-like cabling, connecting the hardware that the immersant wears to the computer system driving the VR, interrupts the relative lack of modulation of this negative space. The cabling appears also as a visible restraint – like a leash – that holds the alien form in check. It also provides a reminder that the immersant must be physically “hooked up to” a technological matrix – operatively held by its armature – but, in terms of visual composition, it also connects the immersant’s figure to the frame created by the sides of the backlit screen. Because the (about-to-be) immersant enters the space where the immersion will take place from the public gallery space, and so can see the screen onto which his or her shadow will be projected, he or she is made well aware that he or she will be privately and publicly – even if partly de-personified – part of the work.[8]

In its public installation, the interactive artwork is not only the technical armature and the participant situated in it: it is also the space the work occupies, and is both a physical or architectonic space and a space of display. The (self-) consciousness the participant has in regard to an interactive artwork is not simply of one of being aware of one’s presence within the instrumental space of the artwork. The presence of the installation extends beyond its armature into its external surrounds, and that extension carries with it the externalized display of the participant. The negative space, produced by the backlighting, that surrounds Davies’s immersant is a pictorial analogue of this extensive space. The immersant is observer of his or her own experience as well as observed as a positive form against a negative surround. The negative and positive are the dual components of what is both installed and installation views – the depiction of a space of actively containing the “immersant,” for the purposes of show. In the installation view of Cardiomorphologies (Fig. 1) the spotlighting of the participant differentiates the participant from the much more mutely lit surrounding space of the installation. The fact that the installation view is photographed from the “point of view” of what the participant would being presumably viewing – the screen showing the video component of the installation – suggests that the unseen screen, and by association the space of the installation itself are effectively observant.[9] If “observant” implies sentience to the space the work, then it at least it can be said to be indirectly so when other viewers are in its presence observing a participant ‘undergoing’ the work. Further, because it contains a network of sensors and devices triggered by various inputs, the space of the work can thought of as technically – and literally – sensitive.

The observant (or sensitive) exterior space and the interior space of the armature are effectively two registers, into which the participant’s body (which includes the sense of embodiment and the sense of the exteriority of that embodiment), are dually received. In two ways then there can be said to be a doubling of the body. Firstly, there is the body as instrumental component of the circuit that also consists of the technological armature of hardware and software; on the other the body in which consciousness in embodied, a consciousness that both attends to the movements it makes as well as monitoring the reactions these give rise to. Secondly, on one side there is a body as active agent of its embodied consciousness that negotiates the artwork, and on the other side, the body as part of the public display of the artwork.[10]

 

2.

The “art” of which Michael Fried primarily disapproved in his famous essay, “Art and Objecthood,”[11] was a particular kind of late modernist sculpture. Artists producing examples of this sculpture, such as, Donald Judd and Robert Morris, set out explicitly to create large, singular forms (or wholes[12]), which insisted their existence as objects (and presences), and in doing so, claimed, not only the space they occupied, but also the space they presented themselves to be viewed in: the viewer’s space. Fried favored painting (and sculpture) that gave itself to a viewing that would absorb the viewer in his or her viewing. It is the space between viewer and work of art that Fried expects the work of art to, in effect, evacuate by drawing the viewer’s vision completely into it. Though Fried does not put it in these terms, the effect of these works can also be said to involve a doubling: that of the work’s presence to the viewer, and the (perhaps no longer viewer but) visitor’s presence to the work.

But even absorbed viewing will involve at least eye movement, even if the viewer is not conscious of that movement. The framing of a picture – whether it be a frame proper or simply the edges of a canvas (or work on other surface) – serves to contain eye movement. To that degree, the frame is a major part of the armature, one that also includes the work’s internal composition, which might hold the viewer’s attention within the work, and it is this armature that might temporarily cause the viewer to pause, to largely stop moving his or her body in relation to the work. However, treatment of composition and rendition of surface can elicit not simply a ‘seeing all at once’ but rather an exploratory looking that requires attentive movement of at least the viewer’s eyes and head to scan across and around the work, to allow for the development of the visual impression that the work makes as a whole as well an inspection of detail and the material means by which that impression is realized.

Considering interactive artworks in terms of Fried’s judgment on late modernist visual art may not be particularly appropriate since these artworks may not resolutely insist themselves as ‘visual’ art (or primarily as visual art, even if presented in an art gallery). However, Fried’s case does offer pointers that can be used in the consideration of interactive artworks, such as space of the work and the viewer’s/participant’s place in it. The issue of putting oneself in the space of the artwork can, again, be “illustrated” by installation views of interactive art works. Considering these illustrations in light of other aesthetic enquiries can help bring into focus certain issues for deeper regard, the conclusions of which might be then brought back to offer deeper aesthetic insight into interactive artworks themselves.

Gilles Deleuze, on his way to arrive at a philosophical articulation of a “logic of sensation,” which he does by a very close examination of the paintings of Francis Bacon, presents a specific case of how perceptual modalities can be thought of as operative in and operative for other perceptual modalities.[13] He does this by proposing that a characteristic feature of Bacon’s painting presents or offers “a tactile or ‘haptic’ view.[14] Deleuze explains his use of the term ‘haptic’ to mean the condition in which “sight discovers in itself a specific function of touch that is uniquely its own, distinct from its optical function.”[15]

This notion of the haptic, as used by Deleuze, does foreground a relationship between what are generally regarded as quite distinct sensory faculties: sight/vision and touch/feeling.[16] The haptic designates a way of seeing that would develop as a result of experiential discovery made jointly through sight and touch: the eyes seeing and the hands or body’s surface feeling what comes in the body’s reach, through the movement of either the body itself or the movement of that which is seen and felt being brought it into contact with the body. The individual sensory stimulations received, either simultaneously or sequentially, from something that can be both seen and felt are coupled in and through the combined sensory impression of the common source of the stimulation they create. Subsequent sensory stimulations are able to resonate with the memory of the experience of previous stimulations. Previous experience of seeing and feeling materials and objects enables the anticipation of the appearance from its touch or its touch from its appearance of a familiar material or object.

Deleuze makes it clear that this coupling is not of the order of mutual resonance: functions of senses (at least that of touch in relation to sight) are brought into play within one another. The haptic would also legitimately involve feeling – here meaning the faculty of touch not simply the sense of touch itself – discovering in itself a specific function of vision that is also uniquely its own. Feeling – the active experience of touch – functionally involves, whether it be subtle or gross, movement: movement of the body itself moving, itself being moved or something being moved ‘against’ it. That there is an acquired inter-coupling of senses is hardly surprising, for, whether in immediate reach or by moving itself into a position of sufficient proximity, a sensitive being can experience aspects of its environment through more than one sense, simultaneously or sequentially.

The inter-coupling of sensory functions that “haptic vision” entails inductively suggests that other senses share comparable couplings and, because movement very often facilitates or accompanies sensory experience, the proprioception of movement itself might also be inter-coupled. For example, even though the phrase itself does not acknowledge it, “eye-hand co-ordination” designates a coupling that very much involves movement – of both hand and eye.

Perception has been closely studied since at least the mid 19^th century. In the 20^th the relation of perception to the development of other cognitive and bodily competencies also received scientific attention. In the early 1960s, for example, an experiment used two kittens to examine the relationship between the development of the visual faculty and movement. One kitten – the ‘active kitten’ – was able to move in a constructed environment (the ‘kitten carousel’).[17] It was harnessed by means of wires and pulleys to a ‘gondola’ in which a second kitten – the ‘passive kitten’ – was restrained. The passive kitten experienced an equal amount of movement in the environment as the active kitten and was exposed to visual shifts in this environment relative to the active kitten’s movement. Only the active kitten developed correct motor responses associated with depth perception cues. It was concluded that there is a fundamental requirement that a kitten be able to experience the visual changes, which occur as a result of its self-motivated movement in its environment, in order for its visual faculty to develop fully.

The example of haptic vision and that of the relationship between self-motivated locomotion and the development of the visual faculty, while coming from quite separate fields of inquiry, both deal with how relatively independent faculties are correlated: sight and feeling in one case and sight and movement in the other. Movement is, either explicitly or implicitly, common to both, and while movement itself is not a sense, a sensitive being can proprioceptively sense its own body’s movement, whether that movement is the body moving itself or being moved. The “kitten experiment” also, indirectly, demonstrates that the ability to move one’s head and eyes but not “locomote” one’s body is insufficient for the development of a full, functional visual faculty. This may lead to a hypothesis that, once the normal visual faculty has developed, the self-motivated movement that had been a necessary part of its development be implicated in this faculty in a manner at least analogous to that discussed by Deleuze in regard to haptic vision: vision comes to have discovered, if not a kinesthetic one, then a proprioceptive function.[18]

Deleuze’s writings that deal with art do often restore the primacy of sense and make it a center for aesthetic and philosophical consideration. While the “kitten experiment” discussed above has no immediate relevance to understanding the experience of art, it does demonstrate a particular coupling of two ostensibly independent aspects of animate and sensory existence. This coupling, or cross-mapping[19], does parallel the sensory complex that Deleuze deals when discussing haptic vision. Movement – the physical movement of the perceiver of a work of art – may not be commonly regarded as part of the aesthetic experience of art. If movement is taken into account, at all, it may be simply be accepted as a practical matter, say, of walking around a sculpture to see it ‘in the round’, or moving through an architectural space or the space of an exhibition. But any art that has a visual component will be viewed with a moving eye; pictorial composition in visual art, for instance, is very much an means of establishing visual structures or patterns or rhythms the will influence a viewer’s eye movements during the viewing of a work.

If movement plays a role in the experience of – at least some – art, then it has a function in relation to that experience. The anticipatory experience of that functional movement, if not the function of movement itself, may, to paraphrase Deleuze, be ‘discovered’ within other senses, in ways that are uniquely their own, distinct from their own individual receptive functions. If sight, for instance, discovers a function of touch in itself, then it is not the discovery that invents that function; it is already there, implicated in sight, just as sight is implicated in touch. And implicated in both is – if not bodily movement directly, then – the proprioception of that movement.

 

3.

The haptic entails feeling (perhaps much more than simply touch – the tactile) and feeling entails moving, and haptic vision (feeling seeing) emphasizes that movement is integral to vision itself. Deleuze’s ultimate aim is to develop concepts that make it possible to deal with sensation philosophically, utilizing Bacon’s painting as a case study to the extent to which it “accumulates” or “coagulates” sensation.[20] In Deleuze’s usage “sensation” does not mean loosely “sensing,” or the sensing of just anything sensible; rather it has the force of a philosophical concept:

[I]t acts immediately upon the nervous system… Sensation is the opposite of the facile and the ready-made, the cliché, but also the “sensational”, the spontaneous, etc. Sensation has one face turned toward the subject (the nervous system, vital movement, “instinct,” “temperament”…), and one face turned toward the object (the “fact,” the place, the event). Or rather, it has no faces at all, it is both things indissolubly, it is Being-in-the-World, as the phenomenologists say: at one and the same time I become in the sensation, and something happens through the sensation, one through the other, one in the other.[21]

All artworks create circuits that involve the very sensitive and sensate being of their perceivers’ bodies. This happens importantly at a pre-rational level as well as, and not just solely or firstly, at the level of “embodied consciousness” (Davies) or “mind-body” (Khut). Consciousness comes into play, of course, as the very least requirement of being able to “aestheticize” the experience the viewing unleashes.

Interactive artworks, such as those considered above, couple the body to technical and sensory circuits in order to elicit and process responses to its movements. The space of the installation into which the participant (as embodied consciousness, as mind-body) is a spectatorial field is one in which the participant is on display and also functions as an operative field in which the participant is instrumentally sensed by the technical gadgetry of the work. This technical gadgetry may not be so definitely evident when sensing devices are not attached directly to the body, however, it can actually be difficult for the participant not to be aware that his or her movements are being “sensed.” This duality of being seen and sensed and seeing and sensing effects its own doubling of the body/sensorium, which is made be both output and input “device.”

Sensation, according to Deleuze, would seem at first to have a dual character (“two faces”), but this supposition immediately requires qualification, so that its absolute singularity (“no faces at all”) is fully recognized. The extent to which interactive artworks might achieve sensation, rather than just being “sensational” would be how successfully they achieve the experience of embodiment while circumventing any awareness of the body/sensorium’s doubling. The staging of interactive artworks in conventional gallery settings and their instrumentalization of the sensorium/body can make this a challenge.

The kinesthetic might be one way – one of many, considering the multiplicity of sensory and experiential, mutual and reciprocal, implications possible – to consider the active “discovery” of the function of senses and of sensory experience in one another and to what extent this may contribute to the overall achievement of a successful interactive artwork. Alternatively, without being part of a greater aesthetic achievement, interactive art’s kinesthetic impingement upon the sensorium and its exploitation of the body’s movements might result in a self-conscious doubling and coupling: as if the “active kitten” had come to see itself as merely the motor for the eyes of its immobilized twin..