The exploded infant

... sit[ting] within a huge all-color jewel while this every colored jewel spoke the music of one’s soul ...
—Mary Hallock-Greenewalt on the experience of playing the colour organ

I don’t have synaesthesia, or at least I don’t think I do, but it has been in my life since the beginning ... before the external beginning, even ... since the womb. My mother was a synaesthete. She associated colour, numbers and sound (particularly pitch), so my joke now is that I couldn’t burp or fart at the dinner table without her telling me it was the number ten, a slightly murky orange and a B flat. We could always test the latter at the piano but the other two seemed peculiar to the rest of us and a deep truth only to her.

I didn’t think much of it over the years of my youth; it wasn’t important to me that my birthdays always had colours attached and that the numbers that mum had for notes didn’t match their position in the diatonic scale. I was busy in headphones thrashing away on my cheap drum kit trying to play Def Leppard’s Hysteria album with one hand tied behind my back and committing far more to the aesthetic of self destruction that came with bad hair metal than to the rudiments necessary to actually get any good on the skins. But I remember the music that she composed and that she sang. She became interested in atonal music when I was a child, so I have memories of her rehearsing Schoenberg’s Pierrot lunaire (her score is one of my more treasured relics from her life) and, more importantly in relation to synaesthesia, she loved to sing the Klangfarbenmelodie (sound colour melody) of Anton Webern; melodies constructed from timbre difference. The timbre is often called the ‘colour’ of sound. In the early eighties, she made computer music on mainframes when computers were the size of apartments and the turn around on six seconds of sound was twenty-four hours in the lab. In some of these works she would morph her voice into the sound of a bird attacking a beetle, among other things.

But I digress; this is becoming a eulogy. I guess what I’m saying is that all of these factors contributed to my obsession with joining the senses of sight and hearing together in my work. There were other factors of course. When I was studying composition at university I was in the odd position, eventually, of being able to compose music without really being able to read it. Weird? Maybe. But it’s a language like any other and you can learn the rules of construction and say intelligible things etc., etc. In the end, the linguistic side of music didn’t sit well with me and later in my degree I branched off to write about visual notation, graphic scores, musical gestures represented in abstract visual gestures rather than notated instructions. I guess I was moving toward a situation where the relationship between musical and visual gesture wasn’t causal anymore, but so simultaneous that you couldn’t separate the two. I found this symbiosis through electrical signal, but it has just occurred to me now that what I am talking about is the essence of live performance—that essential link between physical (visual) action and sonic outcome that allows for virtuosity through the constant challenge to the limits of these physical systems.

The way that the sound and light equivalence started for me was with the Cathode Ray Oscilloscope. I was making some quite harsh, angular noise pieces years ago and I happened to have a CRO in my studio. I had heard that you could feed sound into them and ‘see’ the results so I plugged the left channel into the X-axis and the right channel into the Y-axis to see what my noise looked like. For the most part it was pretty uninteresting and unsatisfying … except for one three-second snapshot where the sound and light locked together and I felt like I was looking straight at the geometry of the sound signal. It was a ‘eureka’ moment for me and has defined my audio-visual work since. I studied that three seconds and started to build a library of sounds and techniques that had interesting visual outcomes. The results fascinated me. The more harmonic the spectrum of the sound, the messier the visual result. Pure tones worked beautifully and distortion (overloading the system) was amazing. The important revelation for me was that sound is geometry, not in the ‘Bach-ian’ sense either of geometric patterns composed as pieces of music, but that sound is geometry.

Take the building blocks of electronic music for example. The sine wave. Take time out of a sine wave and fold it on itself and you have a circle; take time out of a square wave and you have a square; a triangle wave … you get the picture. What it means is that when we make sound we are producing complex combinations of these flowing geometries in the form of sound waves. Of course, this wasn’t a general revolution, it wasn’t entirely new at all (people had been working with optical soundtracks, for example, for decades) but it was new to me and it blew my mind and changed the way I worked with sound to this day.

It also led me to the rich history of artists working to forge a connection between sound and light. The ancient Greeks mused on it, and after Newton’s treatise on Opticks there was the sense that light and sound could share properties through wavelength and frequency. One of my favourite examples of an artist attempting to forge a connection is Louis Bertrand Castell’s Ocular Harpsichord (c.1730). It was basically a harpsichord but each key was attached to a system of pulleys that would open a small curtain to reveal a candle shining through coloured glass. Although beautifully simple, in modern OH&S parlance he had created a fire hazard and perhaps it is no surprise that there are no remaining physical examples of the instrument. Other pioneers in the field include Mary Hallock-Greenewalt, whose visual music phonograph (1919) was a record player with accompanying light show, and Thomas Wilfred, whose Clavilux Junior (1930) was definitely a psychedelic pre-cursor to the Xbox. For me, the grandfather of my work is Jules Lissajous, a French mathematician (1822-1880) who, in order to tune his tuning forks, devised an ingenious method for visualizing sound waves. He created a focused beam of light by placing a cover with a pinprick over a candle, then bouncing that light off tiny mirrors attached to the end of his tuning forks. He could see the reflected waves on the wall. He essentially created a crude prototype of a laser projector.

What I produce when I make works where you see and hear the same electrical signal at the same time is a manufactured synaesthetic experience. There is no causality; the two things happen simultaneously so you don’t have time to think about which came first. Sight and sound become the same thing in time and space, like Lissajous’s patterns on the wall. This seems fascinating to non-synaesthetes. Why is that? Margaret Hollis alluded to one possible answer in her essay from the previous Synaesthesia program. I’ll restate it here in my own way. One theory of synaesthesia is that we are all born with it. Imagine a pre-language state (impossible, but try), what Lacan might call the condition of the ‘exploded infant’. All of our senses are one. Vibration in the form of sound, light, smell, even touch, swarm into us as an undifferentiated mass of pure experience. Gradually, through repetition and the establishment of concrete neural pathways, we segregate those sensations and attach perception to them, developing, in the end, our ‘sense perception’. Perhaps works of visual music or synaesthetic artworks draw us back toward that pre-language state, to a neural recklessness where everything is thrown in without deference to the emergent synaptic bureaucracy that parses our senses into organised and functional blocks. People frequently recount to me a sense of euphoria, ecstasy and the feeling of a chemical high after seeing synaesthetic artworks. I’ve certainly noticed that a much broader demographic of punters will sit through (and even enjoy) the kind of noise I generate in my shows when it is accompanied by its direct visual correlate.

Oddly though, one of the most interesting things about synaesthesia is its idiosyncrasy. Each synaesthetic person experiences it in a unique way. It’s an intimate condition, born of the interior and unknowable to others, private. By claiming to manufacture it homogenously in a group of people, am I some kind of cross-modal fascist? Before she died I had the chance to ask my mother more probing questions about her condition. Though it had clearly been a huge aid to her in her musical life, guiding her through fiendishly difficult atonal vocal works, she spoke of it primarily as an affliction, a chorus of often unnecessary and unwanted correlations that she could never switch off. So maybe synaesthesia is something wonderful to behold and to experience from the outside. A fleeting reconnection of now disparate parts of the brain, but would we want it all the time? I’m not so sure.

Audio-visual artist Robin Fox is a repeat performer at MONA FOMA and Dark Mofo. We can’t wait to experience his latest offering at our two-day sound-art spectacular, Synaesthesia+, at Mona (where else?) on August 16 and 17, 2014. Did you get this far without realizing this was a plug?

Synaesthesia, 2012

Photo credit: MONA/Rémi Chauvin

Dark Mofo Winter Feast, 2013

White Beam, Robin Fox

Photo Credit: MONA/Rémi Chauvin