Animating Film Sound
Catch a glimpse of what goes through the mind of a Newcastle musician commissioned to provide a suitable soundtrack for an animated film whose subject matter revolves around a woman living alone in a high-rise block of flats. John Goodenough relates his experiences.
John Goodenough takes us through the production processes involved in composing and recording a suitable soundtrack for an animated film he completed last year, and provides a rare insight of the decisions he had to make each step of the way.
I first began work on The Box, an animated film, in July 1983 at Amber Films, Newcastle, with the animator Judith Tomlinson. As its title suggests, the film describes the effect of television and the media generally on a woman who lives alone in a high-rise block of flats.
Judy had initially thought in terms of a very simple soundtrack to accompany the film, one which we could work on using the facilities available to us at Amber. That is - recording, transfer (from ¼" tape to 16mm magnetic stock) and sound-to-picture synchronising equipment.
We felt in these circumstances that the soundtrack might be used as a means of focusing the viewer's attention on particularly important events in the film. For example, a door slamming is made to stand forward of the visual action going on around it, by adding sound. It may almost be read as aural 'punctuation' for the visual action.
In talking to Judy and watching the animation which she had completed without sound, it became apparent that the soundtrack for animation need not necessarily be structured in the same way as that of a traditionally-shot live action film, where you are already presented with a certain amount of material which is synchronised, particularly in the case of dialogue. With animation there are more options left open, and often a lack of dialogue means that there is a far greater need for sound to create moods and add 'weight' to the drawing.
Usually the sound is broken down onto several tracks. In our case, six were used. Certain types of sound are placed on the same track so as to make the final sound mix in the dubbing suite easier. For example, usually all 'spot effects' are placed on the same track and all background 'atmosphere' is kept to the same one or two tracks - this is just to make it easier to identify the levels needed.
'Spot effect' tracks, as the name implies, are sounds which need to be critically positioned with the visual action - doors opening, light switches, vehicles starting up and moving away. 'Atmosphere' tracks are usually of background noise and at a low level. There is never absolute silence in the real world, even a very quiet room will have a particular ambience.
However, in animation these levels can, of course, be exaggerated or played down, as the same degree of realism is not always required. This allows for experimentation - cutting in and out of the 'atmos' for instance, to build up tension etc.
I found the most suitable sounds for a particular sequence of pictures, animated or live action, were not necessarily the sounds you would ordinarily expect. Often juxtapositions can work very well eg. Bee (visual) - Aeroplane (sound). Or perhaps even a sound which makes better sense of the visual than the sound which was originally recorded with it.
As an example, I wanted the sound of a bicycle being pedalled away down a quiet street (the visual). We had in the end to turn a bicycle upside down and record the sound of the wheel being set spinning using the pedals, to actually create the necessary sound artificially.
I was being asked in effect to compose a 'score' which included 'real' sounds - traffic, building sites, riots, etc. These sounds in film terminology would ordinarily come under a heading other than 'music'; more like spot effects, atmosphere etc. Yet our ideas for the film implied that I should expand and develop these recorded 'real' sounds and add varied bits of synthesized sound to make as a total, the 'music' for the film.
John Cage wrote about just this 'sound-music' relationship, and it became background material, for what I tackled essentially as an intuitive problem.
For Cage, a rhythmic structure was "as hospitable to non-musical sounds, noises, as it was to those of conventional scales and instruments. For nothing about the structure was determined by the materials which were to occur in it" (eg. street sounds, riots etc.); "it was conceived, in fact, so that it could be as well-expressed by the absence of these materials as by their presence". In short, "- a laying out and filling of empty spaces of time".
Essentially, I was being given a space to create my 'music' within and in that space were a great many ideas (sounds), because the visual scheme (the film) was to an extent fixed. It was therefore more a question of filling in the spaces between these sounds and relating them together. Rhythm was really the vital element in this - together with the idea that the 'visual-sound' relationship should make clear sense to the individuals watching.
I worked at the soundtrack with gusto in two ways:
1. The soundtrack as 'music' - sound.
2. The soundtrack as a necessary part of the film - the film being a source of ideas for sound.
In fact, 2 became the essential discipline for 1.
I was not required to write down a conventional 'score' which could then be performed - the soundtrack was recorded and its production was all that mattered. The soundtrack was the score.
Incidentally, there are similarities between synthesized sound and 'natural' sounds in so far as it is impossible to write down (notate) either of them. Yes, of course you can write down the chords or pitches played on the synthesizer but it is impossible to notate the actual sound produced. There are sampling keyboards which make the pitch organisation of 'natural' sounds a lot easier, yet the sounds produced can still only be recorded on magnetic tape - or given a complex set of figures to define them ie. memorised in a computer.
The work began by timing sections, sequences of action from the film. I then worked with comparatively similar lengths of tape to these referent film sections. This was fairly easy initially and was undertaken in a 4-track (Teac) community studio at Spectro (now Newcastle Media Workshops) in Newcastle.
The first visual idea in the film at this stage was of a woman sat watching and listening to a TV. The majority of the sequence consisted of a shot of the flickering TV on her face. The sound needed only a very limited connection with the visuals on film, since it was coming from her TV - which wasn't directly visible. I simply crammed 45 seconds with lots of disturbing TV sound, news, music from American shows, adverts, soaps etc - often in juxtaposition. This mass of material, I then found, needed layering.
I used the external audio input of a Roland SH09 synthesizer as a way of altering sounds from TV, often quite severely. I found sounds of correspondents giving their stories on the news by telephone very useful. It was possible to filter their sound, use it with music or ordinary speech, and still make sense of it as news. I also played and recorded bits of pastiche news, current affairs, and advert music on my synthesizer. These sounds were all added together in such a way as to create depth. BBC Sound Effects records came in handy, but I found it necessary to add extra sounds to them to produce a more dynamic 'real news' impact.
Having done this first section of film, I became more aware of the necessity of working on the 'music' within the traditional soundtrack arrangements outlined above, because I would have to break the 'music' down to individual spot effects and atmos tracks eventually anyway, in order to be able to fit it accurately with the film.
There are several reasons for this:
1. As I said earlier, I wanted to 'compose' a complete soundtrack, and spot effects would obviously be integral to my general scheme. However, when there were two or more spot effects separated by a sequence of sound, for instance atmosphere, it was impossible for me to set them exactly the right distance apart in the recording studio ie. without the visual action to work from. (One spot is often OK, because the effect - sound to visual - becomes the connecting point for the soundtrack.)
It isn't always necessarily a question of fitting the spot effect exactly to the visual though. There are differing time delays in real sensory perception ie. light travelling far faster than sound. So sound on film does not necessarily match directly with the visual - it's something that can only really be determined with the visuals there in front of you!
Also, delay in spot effects can heighten or lessen tension, can act like pauses in effect, on the visual action. For instance, there is a sequence in The Box where the woman looks through a 'peep hole' in her front door and a young boy is looking back at her - we wanted to position a sound (a loud thump) at this point, and its position was critical in creating a feeling of tension.
2. Although the Steenbeck sound-to-picture synchroniser accurately holds sound and film together (mechanically), it is designed to work with good recorded levels on magnetic film stock. Magnetic stock is exactly the same size as film stock (16mm) and registered to the film on the Steenbeck, or other pic-sync equipment, by sprocket holes. These multiple tracks of magnetic stock are then filtered, equalised and mixed down from say five tracks to a single soundtrack later on in the dubbing suite.
The requirement of individual tracks then is that they be drawn from the best possible (ie. original) copies of the sound involved - rather than from the single track of a multitrack machine, as quality obviously suffers in such an exchange.
However, from a negative point of view, if several separate tracks are involved it is often difficult to establish a real sense of the completed film soundtrack, because the Steenbeck I used (the machine on which most of the editing is done) only takes two audio tracks and film. My sense of the soundtrack therefore had to be ahead of the finished film - like imagining the finished article from an exploded diagram.
The pic-sync machine on which a lot of the more critical track laying was done was manually operated and it proved difficult to wind through the soundtracks at a constant and correct speed. These equipment limitations are not always the case - more sophisticated models are available eg. motorised pic-syncs, four-track Steenbecks etc.
3. There was also a problem with recording levels on sections of multitrack work which I was able to mix together at Newcastle Media Workshops. The reason being that I had to record the originals in 2-track mono, dub them onto a Nagra recorder at Amber Films and then, as described earlier, transfer them to magnetic stock. This Nagra machine was a single (centre) track recorder which meant that to get a good level on it I had to push the record levels on the 2-track (Revox B77) machine very hard indeed.
The logic behind this high recorded level is obvious since the original sounds are transferred three times - once to magnetic stock, and once in the dubbing suite from multiple tracks to a single (stereo or mono) soundtrack. For the result to then be transferred for a third time to the final film print, onto perhaps an optical soundtrack or a narrow magnetic stripe (down the edge of the film), good initial recording levels (to bring the sound well up from the background hiss on the tape) are absolutely essential.
4. We began to find that the original timing of the animation, which had initially worked in visual terms, once married with completed pieces of soundtrack, gave the impression of being rushed. As a consequence of this, certain sections of the animation had to be extended in order to fit in with the timing of the music, and we became aware of how tightly the music needed to be scored, compressing information into shorter pieces while still giving sufficient space, time-wise, for it to develop.
In some ways I feel that I have been a bit negative about my use of the recording studio on this project. Yet, it was really the only way to obtain some impression of what the final soundtrack would be like, and added to this, the equipment there offered more scope for experimentation and allowed the creation of new and different sounds.
For instance, there is a section in the film where the woman gets onto a bus and we see her view of the city (flyovers, tower blocks etc.) through the bus windows. I recorded a solo violin phrase as music for this sequence - when this was placed in context as music it worked, yet was too focused and detracted from the other visual and aural elements at that point - concrete flyovers, the sound of the bus etc. I therefore mixed it together with an electronically synthesized version of itself, the balance between the original and synthesized copy being critical, but it did produce the correct sound - it fitted the context.
Another moment which definitely required work in the studio was a section where we the viewers look down from the 'spy hole' in the woman's front door to her doormat - to see what had come through her letterbox. I used the SH09 synthesizer again here, with an input of corridor atmos (the sound of the corridor outside her door) and as the camera panned down, cut it in and out using the SH09's modulator. The resultant rhythm helped to create the necessary tension.
I found that I was able to use the arpeggiator and (white) noise sound on a Roland Juno 6 synthesizer as atmos (a rhythm in the background), and record varied 'real' sounds of glass being broken and an old cooker being struck with a rusty pipe. These were then mixed (filtered and balanced) carefully together to produce strong percussive sounds or 'chords' - as foreground effects - over the synthesizer atmos.
I also found the arpeggiator and white noise combination useful in that I could add rhythm to street sounds without detracting from the originals - the white noise blending well and not obscuring the original's meaning. We actually produced several sections of film this way which already had elements of rhythm in the visuals and we allowed these to interact with the rhythms of the synthetic sound.
'Rhythm', I found to be frequently the single most important element in the sound/vision relationship. Making animation move is in some senses a rhythmic process in itself, full of careful and intricate timing. Traditionally, animation starts with the music soundtrack already in place, and the drawing is timed to the soundtrack. In this instance, the sounds were created to fit the existing drawings which, although it posed problems, also effected a different way of working and allowed an exploration of the relationship between the visual and the aural.
The relationship between animated film and its soundtrack can be very exciting. For example, it may not necessarily be a question of the visual and sound rhythms fitting exactly, it may be more the point of maximum tension or excitement in the film, where two differing rhythms, the visual and the aural, coincide.
'Mood' is another important factor. You can show the same section of film twice and create absolutely different moods using different music tracks. In animated film moreover, small, quite un-prepossessing visuals like lines or circles, when combined with music or sound effects, can create very strong moods - and can, in fact, suggest things only hinted at by the visuals alone ie. circles (visual) + rain (sound) = rain in puddles.
In animation as in live action, it is obviously the combination of the visual and sound element which produces the finished result. In live action the visual rhythms occur in 'real' time and these are influenced by the camera movement and the way in which the action is edited.
However, because animation (being like a visual shorthand) doesn't have to operate in 'real' time, it can truly complement the soundtrack. This increases enormously the scope for music with animation - allowing far more creative experimentation and integration for the music composer.
Feature by John Goodenough
mu:zines is the result of thousands of hours of effort, and will require many thousands more going forward to reach our goals of getting all this content online.
If you value this resource, you can support this project - it really helps!