Revo:oveR (2008) is a collection of installations created by the Digital Arts Research Collective (DARC) and commissioned for the grand opening of the new Taubman Museum of Art in Roanoke, VA. My contributions include Elemental interactive communal soundscape and interactive audio-visual content for the two sculptures titled Cyrene Reefs that were created in collaboration with conceptual artist Eric Standley. A brief video overview of three pieces can be seen here.

Elemental functions both as a standalone artwork and as a unifying aural component of the Revo:Over installation. Occupying entire exhibit space, the piece utilizes equidistantly spaced 4×3 speaker array suspended from the ceiling. Using infra-red (IR) spotlights and a homebrew IR camera the installation captures visitors’ traversal throughout the room and accompanies their motion with sounds of water ripples. The resulting aural landscape portrays a soothing image of navigating a calm waist-deep body of water. Apart from the fact that visitors’ motion generates no tactile feedback commonly associated with water, the surreal is further amplified by the aural cues that emanate above, rather than from below. As ripples propagate throughout the space, they are affected by the physical forces that define the world around us, refracting from the walls and slowly dissolving until all of their kinetic energy is drained. Occasionally, ripples clash against each other spawning a flurry of ever-changing sonorities, na aural fireworks marking spots where visitors’ paths may already have or are about to cross, or where an individual has caught-up with the wave they generated only a moment ago. The ensuing aural fabric serves as a foundation for a dynamic soundscape whose structure depends entirely upon visitors’ actions and interactions.

Cyrene Reefs can be seen as a pair of ornate musical instruments or sculptural metaphors of instruments that draw inspiration from the mythical story. While at the first sight the installations offer a relatively simple form of interaction, just like any other musical instrument their full potential (akin to virtuosity of playing a real musical instrument) can be uncovered only through patience and persistence. The first sculpture is populated by three arm-sized holes that can be explored by inserting hands and in return offering aural feedback that builds upon the metaphor of elements, namely fire, water, and air. The interaction with the ensuing soundscape is also accompanied by the visual feedback in a form of an ever-changing organic shape, a variant of nurb simulation found in derelicts and some of the more recent visualizations, populating the projection dome whose properties are affected by the aforesaid elements. The holes utilize IR proximity sensors allowing for accurate measurement of distance from objects near and inside the holes. As a result, the sounds of the elements that grow as the hand is inserted deeper into respective holes are accompanied by a seductive dynamic melody whose range and loudness grows proportionally to the concurrent presence of the three elements and whose character is designed to complement the Elemental soundscape.

The second sculpture, populated by five smaller hand holes offers audio-visual feedback that builds upon the metaphor of elements established in the first sculpture, in this case the element of earth. The interaction with the ensuing soundscpae is also accompanied by the visual feedback in a form of distorting camera projection found on both sides of the sculpture. Just like the first instrument, only through patience and persistence does the sculpture disclose all of its multisensory secrets. In this case, through the use of multiple hand holes, a wooden wind chime drone melody emerges encouraging improvisation with different sonorities ascribed to each of the hand holes. From a visual perspective, the two sides of the second sculpture are populated by homebrew infrared (IR) cameras that project their respective images on the opposite sides of the sculpture, suggesting that one can see directly through the large circular openings found on each side of the artifact. Inside this “semi-transparent” space resides a tiny iBot (eye-bot) that is capable of observing nearby visitors, often by uncomfortably staring at them for prolonged periods of time. In the event there are visitors on both sides of the artifact, the iBot will shift its attention from one side to the other as it sees fit, encouraging observers to seek its undivided attention. This sole “living organism” of the Cyrene Reefs stands as its endemic protector, while its seemingly simple behavior serves as a catalyst for spawning many different interpretations in visitors’ minds regarding its role within this virtual “techosystem.” Although an intricate hard shell separates the real world from the iBot’s (and whose visual manifestation is also found in the smaller red circle projections located on each side of the artifact), through the use of hand holes, visitors are able to momentarily “dip” their arms into the iBot’s world, generating aural feedback and consequently temporarily altering their own visual reflection in amusing and engaging ways.

Project highlights:

• Collaboration with a team consisting of a 3D animation artist, architect, conceptual artist, CS researcher, film artist, musician, and new media artist;
• A “techosystem” consisting of six interdependent pieces co-located within the same exhibit space;
• Retrofitting conventional webcam with filters and wide angle lens and coupling it with IR LED floodlights to produce a makeshift IR motion tracking system in a controlled dark environment with wall projections;
• Integration of MaxMSP’s Jitter and Unity3D game engine using µ interoperability toolkit. Use of Max to capture visitors’ motion and import data to Unity3D to run a 3D physics simulation that emulates exhibit space and translates visitors’ motion into water ripple metaphors. The resulting data is sonified through Max and diffused across a 12-channel ceiling-mounted 4×3 speaker array;
• Use of proximity IR sensors for the arm-based interaction in conjunction with a 3D projection on the smaller dome and two displays on the Cyrene Reef B sculpture;
• Algorithmic nurb visualization of visitors’ interaction with the arm holes on the dome (Cyrene Reef A) and the two displays on the (Cyrene Reef B);
• Use of blob tracking on both sides of the Cyrene Reef B and mapping resulting data so that the virtual avatar (iBot) appears to follow visitors’ location. The development of the iBot’s logic to make decisions which side to turn towards and when to disengage, and
• Utilization of GPU shaders for iBot’s reflective eye resembling camera objective and Gaussian blur of two camera feeds.

Early concepts of this two-year undertaking include arm pit bot with a custom scriptable animation algorithm and a HD video playback capability, and a three-screen frustum-enabled cave-like environment with a 12-channel audio diffusion and a 6-channel microphone array designed to attract flocking 3D entities to a sound based on its position.

Project Collaborators:
Ivica Ico Bukvic (Co-PI)
Carol Burch-Brown (VT)
Truman Capone (VT, PI)
Steve Harrison (VT)
Simone Paterson (VT)
Joy Rosenthal (VT)
Eric Standley (VT)
Dane Webster (VT)

Elemental and Cyrene Reefs shows:

• Taubman Museum of Art in Roanoke, Virginia, November 2008-March 2009.
• Kent Square Galleria, Blacksburg, Virginia, May 6, 2009.
• NIME conference at Carnegie Mellon, Pittsburg, Pennsylvania, June 2009.
• Experiential Galleria, Blacksburg, Virginia, September 23, 2009-June, 2010.