A molecular physics experience through movement at Stanford

Earlier this year, dS headlined at the Barbican, London’s hot multi-arts and conference venue. Now it’s coming west.

dS, short for danceroom Spectroscopy, is the world’s first large-scale, interactive molecular physics experience, and it was created by scholar, scientist and artist David Glowacki, a Royal Society research fellow at the University of Bristol, presently in residence at Stanford.

Glowacki is exhibiting an interactive installation based on his dS project at the Stanford Art Gallery this month. He is also collaborating with artist Camille Utterback, an assistant professor of art and art history, and composer and sound engineer Michael St. Clair, a lecturer in the Department of Theater and Performance Studies, to extend the system for a dance production at the Cowell Theater in San Francisco in this month.

The interactive danceroom Spectroscopy installation at the Stanford Art Gallery runs through Sept. 20. Visitors interacting with the dS setup will see real-time projections thrown up on the surrounding walls of their “energy avatars” and will be able to use them to manipulate a real-time atomic physics simulation, generating both graphics and sound.

On Sept. 11 at noon, Glowacki and Utterback will give a free talk in which Glowacki explains the scientific origins of dS and Utterback talks about her experience using it to develop live interactive graphics for the Cowell Theater production Dances of the Sacred and Profane.

The Cowell production, a world premiere, is a collaboration between Glowacki, Utterback, St. Clair, choreographer Mark Foehringer and a host of Bay Area artists, computer scientists and dancers. The productionwas developed around a selection of musical pieces, including works by Claude Debussy and Maurice Ravel, and uses the live video-tracking abilities of the dS system to reimagine the Impressionist exploration of passing time. The Cowell show runs Sept. 13–14 and 18–21.

Molecules in motion

The dS technology works by using a set of 3-D imaging cameras that communicate with a custom-built high-performance computer to interpret people as energy fields. The computer embeds people’s fields in an atomic physics simulation, with the net result that they can use their fields to steer the simulation. The result is graphics and sound, both of which are generated in real-time response to human movement.

The sonic component of dS is carried out using analysis techniques that are common to the field of molecular spectroscopy. The computer performs real-time analysis of how the simulated atoms vibrate; as participants move, they change the atomic vibrations and generate different sounds. However, for the Cowell production, St. Clair is not generating sound per se; rather, he is using the particle data to remix his musical source material, warping it in pitch, time and tempo.

“Not only is dS built from the same mathematics and algorithms that I use in my chemical physics research, but I developed it in close collaboration with a team of artists,” said Glowacki. “As a consequence, it’s as much a scientific research tool as it is a beautiful interactive artwork. Our human sensory organs cannot see the atomic world; nevertheless, scientific communication relies on how we imagine it to look, and this opens up fascinating aesthetic territory.”

He observes that dance is an art form with a highly developed vocabulary that is suited to discussing and analyzing dynamical systems: “I found that I had more in common with the dancers than I thought because we both have insights into the subtleties of dynamical systems. It’s actually been quite interesting to see how often chemists and biochemists use dance analogies to describe dynamical phenomena in their research.”

The range of purposes employed by Glowacki’s dS technology is particularly exciting to him. In addition to the artistic application, he is using it as a tool to educate K–12 schoolchildren and the general public about nanoscale molecular physics by giving them an interactive glimpse into the dynamics of the otherwise invisible molecular world.

“Having seen the extent to which people engage with dS and how they love playing with it, we’ve started investigating whether it can be developed as a digital platform where ‘citizen scientists’ can actually use it to help us conduct real scientific investigations to learn how biological molecules work,” he said.

Visually fusing science and art

Utterback had already begun to collaborate with choreographer Mark Foehringer on Dances of the Sacred and Profane when she learned of Glowacki’s work. “My artwork often uses computer vision systems to generate live imagery based on human movement in public places. The collaboration with Mark is the first time I’ll be creating imagery based on choreographed movements of trained dancers, not the movements of the general public.”

She was thrilled to learn of another Stanford colleague working on a high-speed 3-D tracking system suitable for dance. Glowacki generously offered to let her build on the existing dS system for Dances of the Sacred and Profane.

The original aesthetics of dS focused on clearly visualizing the particle behaviors and the people in the system. For Dances of the Sacred and Profane, Utterback worked with dS programmer and artist Phill Tew to extend the visual capabilities of the system to include different textures, different blending modes and the ability to layer video into the system. “The system is still fundamentally a visualization of how particles react to an energy field, but the new capabilities allow me to use it in a much more metaphorical and narrative way that supports the content of the Dances of the Sacred and Profane performance,” she said.

The performance morphs the more straightforward particle visualizations into other types of imagery. For example, in one scene, the particle movement controlled by the dancers feels very liquid and watery. This is used to create a visual reference to the lily ponds in Claude Monet’s paintings of his gardens at Giverny. Utterback said, “We blend a video of clouds under the particle simulation to increase the feeling of reflection. The particles become ripples on the water, which disrupt the reflections.

“In other words, we are building a whole visual world around the particle movement. What I love, though, is that while we are doing this visually for the performance, in reality our physical world is built around these particle dynamics. So my work continues to extend David’s goals for building the system in the first place – which is helping people appreciate the incredible dynamics that are present in the world all around us.”

See, hear and feel

St. Clair heard about dS long before meeting Glowacki on campus. “I loved his enthusiasm for blending the ways that performance and science think – and, even more so, the ways that performance and science see, hear and feel,” he said. “David has this very deep understanding – one that most physicists share, but many resist – that our choices about how to represent physical phenomena outside of the range of human perception are basically aesthetic rather than scientific. Being able to freely play with encompassing, aural representations of ambiguously space- and time-like phenomena was immensely appealing to me.

“Also, to be blunt: The technology is really cool. I love working with Camille, who has a brilliant visual sense and a kind of technical-intuitive aesthetic approach that I value a lot; and Mark’s precise and skillful formal modern dance vocabulary is a very familiar and comfortable basis for me to do compositional work from.”

The dS projects are closely related to St. Clair’s academic work in digital performance, design studies and pedagogy, and the technological and cultural relationships between broad genres of performance. In particular, he said, “It closely touches on my work with play and games – much of the sound design I’m doing here is more like game design than it’s like traditional musical composition, in that I’m building a physical environment with particular rules and affordances that produces output based on someone else’s play, in this case, dance, in the environment.”