Weaving Conductive Threads

A moment from Repercussions and Reverberations at Purdue, with lights responding to the dancers’ movement.

Designers, dancers and digital tech came together for an experiment in control and choreography

An idea was born as my colleagues Kat Hickey and Renee Murray (both modern dance choreographers) and I were sharing beers at the 2015 Prague Quadrennial: What if we found a way to combine my interest in sensor and control technologies with their work in dance? Almost 18 months later we had the opportunity to present to audiences the first step in what promises to be a long collaboration amongst the three of us (and other artists, including Courtney Frederick and Rose Kaczmarowski in costumes, Mark Jamerson in music composition, and Allison Newhard and Megan Turnquist in lighting) with a piece entitled, Repercussions and Reverberations.

To start the journey, we all began brainstorming. What would a performance look like if the visual and audio environments responded to the movements of dancers? How would that impact the choreographic process? The design process? What would it mean for the audience experience? We wondered if it were possible to go beyond simple one-to-one relationships, in which repeated dancer movements would generate the identical response from lighting or sound equipment. What if, instead, the responses to the dancers’ movements were predictable only to a certain degree? What would it mean if the lighting and sound responded to aspects of the dancers’ bodies and movements that weren’t under direct control—for example, heart rate or breathing rate? And, finally, how can one incorporate body and movement sensors in a way that doesn’t require the costume to resemble some kind of cyborg out of a science fiction movie?

In June of 2016, our work began in earnest. We experimented with a number of materials, including conductive fabrics and threads, to find ways to incorporate sensor networks in costumes suitable for dance. We explored a number of different sensor and electronics technologies, from pulse sensors to accelerometers to gyroscopes. Additionally, we played with control protocols to link up with our existing lighting system, as well as to communicate with a proposed artificial intelligence algorithm (being developed by our colleagues in Purdue’s Polytechnic Institute) that would be able to compose music in response to sensor data. Our colleagues began developing movement vocabulary that could both test our ability to collect the different sensory data we wanted to collect, but also communicate themes about the way we connect with the world around us, with each other, and with and through the technology we surround ourselves with.

Pushing the envelope in as many directions as we were, we saw successes as well as some … opportunities for further development. Our computer technology colleagues let us know around September that the artificial intelligence algorithm would not be ready to use in time; also, incorporating a wired sensory network throughout a dancer’s costume is a challenge, as a costume needs to stretch, and wires break when stretched!

Still, we were quite excited to present the first fruits of our research in a 25-minute dance performance (two solos and a duet). Both dancers were outfitted with custom garments housing prototypes of wired sensory networks, communicating on a wireless network with custom software to process the data and communicate with a show control system and a DMX lighting system.

Each dancer was fitted with a three-axis gyroscope sensor connected to an Arduino microcontroller by means of an I²C data connection. (The I²C network was distributed throughout the costume.) Each microcontroller communicated to a control computer offstage via an Xbee radio over a self-healing Digimesh network, transmitting gyroscope readings approximately every 500 milliseconds. A computer running Q-Lab was used to both play back the audio composition created by Mark Jamerson as well as to transmit OSC (Open Sound Control) commands to the offstage computer via a wired TCP/IP network. These commands indicated what the control computer should do with the received data. On the control computer, a Processing sketch received the data transmitted from each dancer as well as the OSC commands, then formatted that data as DMX messages to be transmitted to the four Mac 2000 automated lighting fixtures to control color, pan and tilt. 

The resulting presentation demonstrated the potential of the technologies, as well as their coolness. Watching automated lights move in coordination with the rotational speed of the dancers was a real thrill, and the dancers shared that they felt an unprecedented level of connection with and power over the lighting of the event. Moving forward, the team will continue to explore fabric and wiring technologies to allow for deeper embedding of the sensory networks, providing data streams seamlessly to audio and lighting systems, and what it means to choreograph with different sensors and systems interacting simultaneously.