Just Passing Through

An early GAN-based AI abstract audio-visual landscape. Music by The Josh Craig.

From the artist: "This piece grew out of my conceptual work in abstract color theory and a technical question: how would a GAN-based AI handle it? In 2020 I was a beta tester for Playform.io, an early AI platform for artists led by Professor Ahmed Elgammal of Rutgers University. A web-based platform lets artists upload their own training sets as adversarial pairs or style influences. The results were unpredictable, but every so often the tool gave me compositions I found genuinely amazing.

For Just Passing Through, I uploaded single-colored images into the system. Playform blended them in wonderful ways and output the results as video. I was fascinated by the way that AI blended the discrete color files into lush gradients and traversed 3D color space in unexpected ways. Color has been a long-term interest of mine in both traditional and especially digital media, where artists have unprecedented control and new types of tools for exploring color relationships.

I brought that video into After Effects and processed it through many plugins that distorted the space and layered in effects.

I sent a near-final version to my musical collaborator, The Josh Craig, who began writing an atmospheric soundtrack using a range of analog and digital tools. One of the most fascinating is a color sensor developed by his friend Neil Harbisson, the cyborg artist whose embedded antenna translates color into sound. Josh used his phone camera to replicate that mechanism, extracting the sound of the colors programmed into the video, then processed the audio through a sampler and performed the melodies in the key of the colors.

When the file came back, I reprocessed the video using SoundKeys, a plugin that synchronizes brightness and saturation fluctuations to the musical beats. So what you’re watching, you’re hearing—and what you’re hearing, you’re watching."

Exhibited in 2020 at the inaugural Computer and Digital Art Fair (CADAF), founded by Elena Zavelev. In 2022 it was longlisted for the Lumen Prize.