Getting into Video (Part 3)
PART 3; David Mellor looks at how the requirements of different types of music for TV and film can dictate the sophistication of your synchronising capabilities, and at how machine synchronisation works in practice.
Moving pictures and music have had a long and successful relationship, one which will no doubt continue for many years to come, and as this relationship continues it will provide creative and financially rewarding employment for thousands of composers, musicians and sound engineers. Music enhances the action taking place on screen, and so modern musicians are carrying on the fine tradition started by the old-time cinema pianists who would play along to silent films.
But let's ignore the history, and look at the situation of a latter day struggling composer as he or she aspires to be immortalised on a film or TV soundtrack, and at the small music studio whose owner envies the high hourly rates charged by video-oriented organisations. How can these people set themselves up to link music and picture? What equipment is available, and how much of it is really necessary?
For a lone composer who works with MIDI on a day-to-day basis, the equipment that is necessary to get into video may not amount to very much. With a reasonable MIDI system - incorporating a sequencer and a good quality stereo tape recorder for mastering - you could start putting music to picture for the princely sum of £0.00 - zero pounds and zero pence. Yes, in certain situations no extra equipment is necessary at all.
A little explanation of the nature of music for TV is necessary here, to outline why some can be recorded so easily, whilst some requires a good deal of expensive equipment. Regardless of instrumentation, there are two distinctly different types of music that appear on the small screen: music for dramas and documentaries, and music for commercials. (I could add to these two categories a third for the various themes, stings and logos that appear throughout the course of a day's broadcasting, but these are more likely to have been recorded entirely separately from the picture, perhaps in the form of library music.) In dramas and documentaries the music is very often atmospheric, setting an aural 'scene' for a sequence of shots. On the other hand, music in commercials is usually much more tightly composed, because of the demands of TV advertising to say a lot in a short space of time.
If the requirement is for suitably atmospheric music that simply lasts for a certain length of time, without any definite link with the on-screen action that would call for tightly timed and composed music, then there is really no pressing need for synchronisation equipment. All you have to do is to write the music to the correct length, record it on your sequencer, then press the sequencer's Start button at the right point as you watch a cassette copy of the production on your home video. If the music fits the required section of picture, you can simply record the music on tape and send it off to the client. Your music will be incorporated into the final soundtrack at a sophisticated audio-for-video facility.
This procedure sounds very simple. It is, and there are only a couple of potential sources of problems. The first is that the time at which the music starts cannot be controlled with total accuracy, due to your error in recognising when to push the Start button, and also the short time that it takes for the sequencer to kick into life; all things considered, this probably adds up to an error of half a second or so. This is quite small, but it means that there must be no precise link between music and picture - a dramatic scene change for example - that needs to be any more accurate than this.
The second constraint is that the tape recorder must run at a constant speed. None do, of course - with the exception of DAT, which has infinitesimally small wow and flutter figures - and over a period of time the music will drift out of sync with the picture. It depends how good your tape recorder is, but you should be able to achieve good results over periods of up to a couple of minutes. Note that you are not really in a position to record the music into the sequencer while you are actually watching the picture. There is too much button-pushing hassle involved and it is better to compose the music separately, checking it against the video from time to time as recording progresses.
The next step up in synchronisation sophistication is to have a timecode-linked sequencer. As you must be aware, virtually every manufacturer with a presence in recording technology makes a timecode-to-MIDI synchroniser of some sort. Usually these are intended for synchronising the sequencer to multitrack tape, and they work by converting timecode into MIDI sync pulses and song position pointers.
When you receive your commission for music for film or TV, if you ask for a VHS cassette copy of the production with timecode (EBU, of course) on the audio track, you should be able to sync your sequencer to this video recording just as you would to a multitrack tape. The only problem here is that the audio track on a VHS cassette is not quite of hi-fi quality, and consequently some sync units may not be able to read the code reliably. It's a case of try-before-you-buy, and I would advise checking that everything works smoothly with your video machine before you try it on your first commission.
The advantage of using timecode to sync up your sequencer is that you don't have to press the Start button yourself whenever you want to check whether the music fits. This makes it easier for you to concentrate on the task of writing the music, while the machines do the work to which they are more suited. With this setup, it is now possible to compose directly into the sequencer while watching the picture, but it is still not possible to deal with anything that requires really precise fitting of music to the screen action.
The key to being able to identify precise 'hit points' on a video - such as a scene change, or someone throwing a punch - and emphasise them suitably is the ability to identify each individual video frame. Timecode recorded on the audio track of the video already does this, but only while the tape is running, and only for the sequencer's benefit. To help you to locate hit points, you need to ask for something else to be put on your work copy of the video - 'burned-in timecode'.
Burned-in timecode does not require any dangerous pyrotechnic materials, and fortunately will not require any extra effort on the composer's behalf other than simply asking for it to be provided on a VHS or U-Matic copy of the production. This is done at the stage of preparing the copy from the video master. The video copying house will feed the code from the master video through a device which resolves the timecode digits as an image, and overlays them on each frame of the video being copied. This results in a work tape in which each frame has the correct timecode values visible in one corner, so that you can see the frame numbers on your TV screen.
Most domestic videos now have good freeze/still frame facilities, and many offer slow motion too. So, suppose you wanted to emphasise musically something that happened very suddenly in the picture, such as a dramatic scene change, you could slow the action down until you found the first frame of the new scene and make a note of its timecode number. Even if your sequencer cannot translate timecode into bars and beats for your information, you can work out when the change in the music has to occur easily enough with a programmable calculator. Playing back the video now, with the sequencer in tow, will place the music exactly in time with the action, to the frame. This is called 'frame accuracy', and there is a big difference between having things spot on and having them even just a couple of frames (1/12th of a second) out.
As you can see from the story so far, it is possible to create music to fit to picture with no more equipment than would typically be available to the common or garden MIDI studio owner. The only potential difficulty is in obtaining a timecode-to-MIDI synchroniser that can read poor quality timecode from a domestic VHS video machine.
There is, of course, more to musical life than MIDI, and unfortunately MIDI sequencers cannot yet cope with recording vocals, guitars and other acoustic devices. These still require a multitrack tape recorder. To achieve frame accurate synchronisation between video and a multitrack tape recorder you need a synchroniser (more fully titled a 'machine synchroniser', to distinguish it from its junior relative, the timecode-to-MIDI synchroniser). The 'looser' atmospheric type of music recording which I described above - as opposed to music which fits in with fast scene changes and frequent hit points - can be recorded adequately on a multitrack without using a synchroniser. In fact, you can even make a frame accurate recording by checking the burned-in timecode visually and using your sequencer as a timecode-linked metronome though this is a little awkward. But to produce high quality work, and to be able to check that your hit points really are as accurate as you think they are, you need to be able to play back the multitrack tape 'in sync' in your own studio before sending your work out.
Unfortunately, this is not as straightforward as it might be: synchronisers are rather more complex devices to install and operate than most recording equipment. Audio interfaces are pretty well standardised - either 0dBu or -10dBu line level, low impedance outputs, high impedance inputs etc - and so is the interface between synthesizers, samplers and sequencers (MIDI to you and me). Machine control interfaces, however, are not standardised yet. They will be eventually, at which point you will probably be able to simply plug in and go, but until then, setting up a synchroniser system is not so straightforward if you want to get good results.
Let's look at what the synchroniser does for its living: a simple chase synchroniser takes a feed of timecode from the master video recorder and another from the multitrack slave. (The video recorder is normally chosen as the master, as such machines tend to be less easy to control than audio recorders.) The synchroniser compares the two timecode values and sends instructions to the slave so that it can find the correct place on its tape, and then start running at the correct speed. The synchroniser keeps comparing the values to ensure that the slave stays in sync. A more sophisticated synchroniser will take account of the transport commands issued by the user to the master machine, and the tach and direction information that the master can provide, to operate more efficiently, as I will explain later.
Reading the timecode and comparing it is simple enough, the difficult part is controlling the slave machine. Look at it this way: there are a small number of manufacturers of synchronising equipment and a rather larger number of manufacturers of multitrack and stereo tape recorders. Few manufacturers - Fostex being a notable exception - make both. Add to this the fact that hardly any two types of machine have identical interfacing requirements, and hardly any two individual machines of the same make and model perform identically, and it becomes clear that the synchroniser is going to have to be extremely flexible.
When you buy a synchroniser, you will actually buy at least two items: the synchroniser itself, and an interface for the particular slave machine that you intend to use. If the synchroniser is more sophisticated than the simple chase variety described above, then you will need an interface for the master machine as well. The interfaces may be as simple as a couple of cables with the appropriate multipin connectors on either end, or they may take the form of small boxes with flying leads that you have to find space to tuck away somewhere. Either way, the synchroniser will also have internal circuitry and software capable of working with various machine types.
The basic setup for chase synchronisation, therefore, consists of the following items: video cassette machine; synchroniser; interface; multitrack. This is shown in Figure 1. Also, to introduce another technical term, this setup would apply if you were working with a code only master. 'Code only master' means that transport commands (start, stop, rewind etc) issued to the video are not passed on by the synchroniser to the multitrack. This will be the case when you use a domestic video, which lacks the necessary interface connection.
This basic system can be run entirely from the controls of the video - even from the infra-red remote control unit if you like. Wherever the video goes, the multitrack will follow, and so will the MIDI sequencer if you hook that in too. The only snag is that this is a slow way of working, due to the time that it takes for the slave to lock up to the master machine. For instance, if you play a three minute section of the video, the audio tape will play along too, without any problems. If you then stop the video, timecode will cease and the synchroniser will stop the slaved multitrack; the two tapes are stopped at the same point. However, since this system relies purely on timecode, the synchroniser will not notice if you start to rewind the video. It will not know that you are going back to the start of the song, and will not find out until the video starts to play again and issues timecode once more. Then the multitrack will start to rewind and will only catch up with the master after a lengthy pause, perhaps the best part of a minute.
It is possible to work this way, if you do most of the recording without the benefit of the picture, and only check your progress every now and then. It is a satisfactory system for the composer/operator working at his or her own pace, but is not so suitable for a commercial studio. A client's patience might wear extremely thin when he realises that he is paying for the time it takes the two machines to lock up.
A better solution is to use a video machine that is designed for synchroniser use. It could still be a reasonably inexpensive VHS model, rather than a more pricey U-Matic, but still effective - what is needed is for the machine to output 'tally' signals when its transport controls are operated. This simply means that there is a rear panel multipin connector which can carry simple analogue signals: for example, the voltage on pin 1 could go high when the Play button is pressed, pin 2 goes high when it is rewinding, and so on. Also, there must be a pulse, known as a 'tach' (short for tachometer) or 'CTL' (control) pulse, that tells the synchroniser the speed at which the tape is winding. Obviously, there must also be an indication of the direction in which the tape is travelling.
From all this information, the synchroniser knows as soon as the video does which of its transport buttons have been pressed, and can relay this information to the multitrack. So, when Fast Forward is selected on the video, the multitrack immediately winds forward also; what's more, the synchroniser knows from the CTL pulse how fast the video is winding. When the video stops rewinding, the synchroniser can bring the multitrack to a halt within a few frames of the video. When Play is selected again, the speed of the multitrack will be adjusted until a precise lock is achieved and synchronised play resumes - lock up is now achieved in a matter of seconds.
With a video recorder designed for synchronisation, a synchroniser with master and slave interfaces, a sync-capable multitrack tape recorder (which most are these days - even some multitrack cassette recorders) and a work video with burned-in timecode, you are ready for some real no-compromises music-to-picture recording. But the system is still limited by the fact that the finished product - your music - will be recorded onto stereo tape and will lose that vital timecode reference.
As I said earlier, if the tape recorder is up to scratch and runs at a constant speed, then it should be possible to re-establish sync when the music is combined with dialogue and sound effects and laid back onto the video master. But why live in doubt, waiting for that awful phone call that says that the tape you sent in isn't up to standard, that it doesn't sync properly for some reason? The solution is available in the form of the centre track timecode stereo recorder.
As you know, a stereo tape machine records two tracks across the width of the tape with a small guard band between them. The guard band was originally specified to avoid crosstalk between the tracks, but as tape head design improved, the need for such a gap diminished.
This is fortunate, because with modern technology a third track can now be squeezed in between the left and right stereo tracks, and this track can be used to record timecode. The timecode on the centre track of a stereo recorder has exactly the same function as the code on track 16 or 24 of a multitrack recorder; so a machine such as the Fostex E2 can be synchronised in exactly the same way as the E16.
So, when the multitrack tape is mixed down to stereo, a regenerated version of the timecode (see last month's article for information on timecode copying) is recorded onto the centre track. Even if the stereo machine is not linked up to the timecode system (ie. is using the multitrack rather than the synchroniser as the source of its regenerated code), at least you know that the tape has the same digits recorded on it that you were working with when you recorded the music, and you can be almost certain that it will synchronise successfully at the final lay-up.
Next month's article will look at the equipment in more detail, concentrating on integrating the Fostex 4030/4035 synchroniser system with the Fostex E2 and E16, and with domestic and JVC professional VHS and U-Matic video recorders.
Feature by David Mellor