Painting by Numbers
The story behind E&MM's most striking front cover for years. Dan Goldstein talks to Les Arnett, graphic artist extraordinaire, on the creation of a music/cornputer/video masterpiece.
Take a piece of music, sample it, analyse it, add a splash of colour and a touch of artistic licence, and you have E&MM's most striking front cover picture for years. But this is just the beginning.
Not long ago, a standard-issue press release arrived at these offices, accompanied by a rather impressive black and white print of some musical video graphics. The press release has long since disappeared underneath the heaps of waste-paper on the Editorial desk, but I seem to recall it carried the pretentious headline 'Sampling plus Voiceprinting equals A New Art Form', or words to that effect. There was a phone number at the bottom of the missive, so I determined to get in touch.
The man on the other end of the line introduced himself as Les Arnett, a freelance designer who had completed the voiceprint after being commissioned to produce 'a visual representation of music' for a poster promoting a London appearance by the Dallas Symphony Orchestra.
'I often come over to Cambridge to do research at Sinclair', said Arnett, 'so I'll drop by sometime. I'll send you a duplicate transparency of the voiceprint, too, so you can see what it looks like in colour.'
He did, and we were very, very impressed. 'We normally have pics of musicians surrounded by banks of keyboards', I explained as Arnett sipped his first cup of Music Maker coffee. 'But the voiceprint is so stunning we've decided to break with tradition - if you don't mind.'
Arnett minded not at all, and proceeded to show us a second voiceprint, reproduced here, that he'd just put the finishing touches to. He was immediately interested in E&MM and what went into its production, which wasn't surprising, when you consider he spent many years as an Art Editor at several major magazine publishers. When it got to the point where he was ultimately responsible for the artwork of 34 monthly titles, he threw in the towel and set up on his own.
As a partner in a small design consultancy called Van Der Graph, he's undertaken a lot of design work in the commercial hi-tech field. Prior to the voiceprinting experiment, his most notable achievement was managing to interface a modern business computer system with an ancient '50s telex machine using lasers, the object being to illustrate an exhibition stand theme of 'Linking the Unlinkable'. And just to emphasise that theme further, Arnett divided the stand into two BBC TV stage sets, from Fawlty Towers and Blake's Seven respectively.
More recently, the designer has worked on a hi-tech exhibit for the States that involved no fewer than seven moving holograms, and it was a Stateside connection that brought him the DSO voiceprinting job...
'An American company called Electronic Data Systems were sponsoring the Dallas Symphony Orchestra to come over to Britain to take part in the American Festival in May, and it was they who commissioned me to produce some promotional artwork.
'Being a computer-based company that's heavily involved in sponsoring music, they wanted the illustration to be a very graphic visual representation of music. John McNulty, who collaborated with me in this project, is a real computer expert. He has something like 29 patents to his name in the computer field alone, so he was obviously a great help. But it still took us six weeks of research just to find out what we were going to need to make something like this possible. We went to places like the BBC Radiophonic Workshop and other studios, and most people just laughed at us when we told them what we wanted to do; they thought we were round the bend.
'We knew you could get visuals of music in various forms, but none of them were actually representative of the music as it evolved through time. They simply involved people deciding that a certain light should come on at a certain point in the music, so they bore no direct relation to the music at all.
'That was the big problem: getting a representation of music changing through a distance in time. Eventually, we hit upon the process that produced these voiceprints.'
And what, precisely, is that process?
'We applied the principle of Fourier analysis to 300 different wavelengths of different instruments within a piece of music. We then broke that information down and compressed it, fed it into a series of oscilloscopes, and from those into a Digital Equipment VAX computer for final analysis. It's the computer that gives you the scored waveshapes you can see in the finished image, but at that stage, everything is still in black and white. It's when you get to the Quantel system that the "art" bit comes in, because that's the stage at which you introduce colour. You have lots of options: you have to reject certain waveshapes because they don't fit in, and you have to decide where you want to introduce colour and which colours you want to introduce.
'The way I did it was to give each colour its own specific pitch range, starting with blue for the highest frequencies. Where you can see the colours blending together is where more than one instrument is occupying the same frequency range, because obviously, musical instruments overlap in pitch.
'The whole thing took us about six weeks to do, and the latest voiceprint has taken us a similar length of time. John wrote the analysis program himself, and the computing equipment we were using has a very high specification. It's used in industrial applications like stress-testing bridges, that sort of thing. So there were no problems accommodating the musical frequency information. The system could probably accommodate frequencies well beyond what humans can actually hear, in fact.'
But if the computing side of the process posed Arnett and McNulty no technological problems, the fusing together of several technologies (music, computing, video) caused them enough worries to last a year, let alone the duo's six-week deadline.
'That may have been why so many people laughed in our faces at the beginning', reflects Arnett. 'We were trying to blend several technologies that were never really intended for that degree of integration. Each time we pushed the computing side to its limits, the video was pushed further on as well, and so on. That caused us the problems, though it's also what makes the art form unique. What worries me is that some people will look at what we've achieved and say "that's fantastic - but I could do it with a marker pen ". Because you probably could achieve something similar by fudging it, cutting out half the processes that we went through to achieve this. That way you'd end up with an image that wasn't really representative of the music at all, but unless you'd done things properly the way we have, you wouldn't know whether the image that resulted wasn't a true representation of the 1812 Overture, or Bruce Springsteen, or whatever.'
Still, Arnett can be confident that no matter what other artists may do to his process in the years to come, his name will go down in the hi-tech history books as being the first to boldly blend where no man had blended before. And whereas imitators may seek to short-cut their way to artistic glory by copying his technique, Arnett is set to stay well ahead of the pack, simply by refusing to admit that what he's created thus far is the limit of what can be achieved.
'The machinery we've used has a total cost of about £3 million, which gives you some idea of the potential that must exist by using it all in combination. The Quantel system is absolutely mind-boggling. The thing we use most is the Quantel paintbox, and that's quite amazing in its own right, even though it's only a small part of the system. You're given a light pen or an electronic airbrush, which you direct onto the screen using a colour or colours chosen from a palette underneath. And like I say, that's where the artistic interpretation comes in. We do it with the music on headphones, so that the colours reflect the music in some way.
'But I'm sure the current image is really rather crude compared to what could be done. There are two steps we want to take on from this. The first will involve adding another dimension to the waveform, so that instead of it being just an x- and y-axis form, it'll have a depth dimension as well. With the current system, a C played on a guitar looks pretty much the same as a C played on a violin. When we've added depth, we'll be able to represent the difference in harmonic structure between the two, and the waveform will actually come towards you and recede into the distance as that structure changes. That will certainly make things more vibrant.
'The second stage is much more radical. We've been working towards abandoning waveforms altogether, using entirely different visual shapes to represent music. We're hoping someone will finance the development of six more voiceprints. If they do, we'll be using a different way of representing music in each one, so that each one has a different visual form - like a completely digital breakdown, for example.'
Should those half-dozen pieces of artwork be commissioned, it'll also enable Arnett and McNulty to help solve two other current limitations, connected with visuals and music respectively.
'We want to exhibit the six in different parts of the world, but the problem we have to get round is one of making sure the images look as striking as they do on the original monitor screen. You'll probably get a load of stick from your printers over these transparencies, because reproducing them in print is a real problem. It's currently the weakest link in the chain, because as with everything else, the further away you go from the original, the worse the image becomes. What we hope to do when we exhibit these is backlight them on light-tables, so that we don't have to go any further than film in the reproduction. There is one colour scanning house that's currently working on an interface between the digital Quantel data and the scanners, so that the printed image is a direct representation of the monitor one. But that could still be some way off.'
And the musical problem?
'So far we've had to confine ourselves to purely classical pieces. We have looked briefly at other forms of music, but you need the guineas. Taking six weeks off to investigate something costs a lot of money: just sitting in front of the Quantel costs £200 an hour!
'For the first image, we had to confine ourselves to what the DSO were playing in London, so we worked on Mahler's First Symphony. It's described as a dirge and that's pretty accurate. It's interesting in its own way, but musical interest doesn't necessarily translate into visual interest. There are some much livelier pieces that would have produced more striking results.
I think the second piece - which is a representation of a Brahms violin concerto - proves that. We've dispensed with the musical notes and replaced them with a picture of a violinist, which is digitised - and totally changed - from a photograph.
'If and when we do the set of six images, we'll be working on six totally different sorts of music, as well as six different ways of representing them visually. That should confirm our suspicion that different forms of music end up appearing in different visual "families".'
Look forward to seeing them, then.
Feature by Dan Goldstein
mu:zines is the result of thousands of hours of effort, and will require many thousands more going forward to reach our goals of getting all this content online.
If you value this resource, you can support this project - it really helps!
Please note: Our yearly hosting fees are due every March, so monetary donations are especially appreciated to help meet this cost. Thank you for your support!