Part three of our series on music syncronisation - this month Chris Smith looks at locking your music to video

Well here we are at part 3 of this epic series already. Originally there were only going to be two parts, but that quickly spilled over to three to cover all the multitude of topics related to this ever expanding subject.

During my research for this part I came across a whole mew area of development called broadly 'Video Titling'. Basically it allows you to gen-lock your PC to domestic videos and add graphics, grab picture frames and generally animate your home movies. Sounds great fun. I am in the middle of investigations right now - full report next issue.

That is the marvellous thing about this kind of work - the joy of finding out and understanding. The more you discover, the more you realise how much more there is to discover. Once you stop learning it is time to change jobs.

But learning is a two way street. Don't just read this stuff, get out there and try it out for yourself. If there is anything to do with synchronisation that I have missed out, or that you do not understand, ask me (write to the editorial address). You pay for this magazine - you make the most of it.

MIDI Time Code (MTC)

So far we have talked about synchronisation code in two forms: SMPTE recorded on audio or video tape and and MIDI clocks between MIDI equipments. MTC is a new standard that has been specified to unite these two forms into one by encoding the hrs/mins/secs/frms of the SMPTE signal into a MIDI format and thereby dispense with the conventional MIDI clock/song position pointer format altogether.

New MIDI devices that accept MTC can now, in effect, read raw SMPTE via their MIDI in socket and derive the tempo, SPP and cue points for themselves. This method has numerous advantages:

1) Simplicity. The SMPTE/MTC conversion can be performed without any conversion inaccuracy by a single 'dumb box' with little or no controls. The job of deriving musical tempo and displaying SMPTE time and/or musical bars is passed onto the sequencer itself. Tempo variations and cue points are saved to disc as part of the sequence data.

2) Accuracy. Tempo resolution is now entirely governed by the MIDI devices responding to MTC. The code updates the SMPTE time every 2 frames (regardless of tempo) and allows for cue points to one hundredth of a frame resolution. Lockup times can be in fractions of a second.

3) Universal compatibility. As long as each device converts the MTC into tempo and song position in the same way there should never be system synchronisation problems, no matter what equipment is used or when. So you can change your sequencer/software, drum machines, tape machines, videos, SMPTE/MTC converters etc. at any time during the production with no problems (a dream come true for most engineers). But very few system components are MTC equipped as yet, so they may have to carry on dreaming until the manufactures realise its potential.

4) Advanced features. As well as 'raw SMPTE' the MTC format allows for a whole range of communication messages to be sent between equipments via the MIDI cable. High speed 'thinned out' SMPTE is transmitted for fast forward/rewind of tape machines allowing MIDI equipments to chase to tape, or sequencers to control tape machines, punch in/out points, event starts/stops, cue points, offsets and set up codes can all be enabled, numbered, time-tagged and even named within the protocol. Even the SMPTE user bits (now used to identify tape spools) can be encoded into MTC for display as ASCII characters on remote MIDI devices.

The MTC Format

The MTC specification gets a bit complicated by bit fields and nibbles (it took me ages to work it all out!) but I will try to explain it in simple terms.

The basic problem is that for a sequencer to interpret tempo accurately it must receive MIDI timing messages exactly at the right time. So if MIDI clock or MTC messages are present in a stream of other MIDI data they have to sort of 'barge in'. For example, if some pitch wheel data is being sent over the clock period, that data is split and pushed either side of the MIDI clock or MTC message.

So that these system real time messages (ie. MIDI clocks, MTC, start, stop, continue, active sensing) do not interfere with the data streams that they suddenly appear in the middle of they have to be short (two bytes) and easily identified. The first of these two bytes says 'hey look! I'm a system common message for MIDI clock, or SPP, or MTC - each type of message having its own status byte. This leaves just one byte for the actual data (of which only seven bits can actually be used), plenty of room for a MIDI clock, but not enough to encode a whole SMPTE hour/min/sec/frame time for MTC.

So MTC data is sent in two byte chunks (called Quarter Frame messages) at regular intervals, each identifying a different element of the SMPTE time. Once eight QF messages have been sent the processor in the MTC receiving device is able to assemble and interpret the full SMPTE time, ie. once every 2 frames (15 times per second at 30 fps rate).

For the brainy readers who have kept up with me so far here is some nifty maths that should shake you (if you are lost already skip the next bit).

The MIDI data rate is 31.25 kbits per sec. So one bit lasts for 32 micro secs. Including stop and start bits, one byte (10 bits) lasts for 320 micro secs, and one QF message (2 bytes) lasts for 640 micro secs, okay? Now there are 4 QF messages per frame, which is 120 per second at 30 fps rate, or one every 8.33 millisecs. Quarter frame messages therefore take up a total of 0.0768 (120 X 640 exp-6) of a second every second, or 7.68 per cent of the MIDI bandwidth - a reasonably small amount.

Other parts of MTC such as tape location messages and cue point times are sent as system exclusive data of 10 or 15 byte blocks. These messages are sent far less frequently than the QF messages and even in the busiest of systems should only account for one or two per cent more bandwidth. There should be plenty of room left for sequencing data, even if conventional MIDI clocks and SPPs are sent simultaneously with MTC.

So Where is MTC?

Despite all this techno-praise from your enthusiastic author, I have yet to see a working system with MTC implemented. More manufacturers are beginning to provide the facility now but are slow to point out the benefits to users. It seems that because we already have a system of MIDI clocks and SPP that works well, people are reluctant to move to the new MTC standard, especially one about which precious little is common knowledge.

Cubase, the new Steinberg sequencer is compatible to MTC (although they don't make much of the fact), but Passports MTP (personally my favourite sequencer incidentally) are supporting the standard in America with SMPTE/MTC conversion hardware to match. Other hardware companies (JL Cooper, Adam Smith, Tascam) list MTC is a feature, but say little more about it.

I questioned Will Mowat from C-Lab, who seem to be at the forefront of sequencer technology in all other respects: "We have our own Unitor tape sync system which takes audio SMPTE into the Atari ST directly. We haven't really thought about MTC, what exactly is it anyway?"

After acknowledging that, although it should be, C-Lab is not the only sequencer system that musicians use, or will use in the future. Will did concede that compatibility may be important and that he would talk to Mr. Lengeling about it. Thanks Will.

My opinion is that it will take some time yet, but manufacturers, software writers and users will eventually realise the potential of MTC and sequencers, effects units, audio and video tape machines and countless other devices will all work together in unison, jam-synced, chase-locked and event-controlled by that simple 5 pin din MIDI socket. Imagine a VCR with a MIDI socket you can plug straight into your sequencer - its not that for away... remember where you read about it first.

The Video Equipment

The video production recording world is dominated by the Japanese giant Sony. Their product range is so vast and their marketing so aggressive that many corporate and broadcast video suites consist entirely of Sony gear. Panasonic and JVC hardly get a look in, despite having spent millions on research and development of new products and standards recently.

Sony produced the Betamax VCR (video cassette recorder) system for home use in the late 70's. At that time VCRs were very expensive, but Sony refused to hire out domestic machines, preferring customers to buy. About the same time JVC brought out the VHS (Video Home System) domestic format, and with the backing of American film companies, began to hire out popular films and VHS machines. The Betamax format gradually faded away and Sony concentrated on development of corporate and broadcast machines. Only recently Sony quietly brought out VHS compatible domestic machines and joined the throng of manufacturers, using the worlds most popular video format in the massive domestic VCR market.

The most basic and common Sony machines use the U-matic standard such as the portable BVU-150P pictured here. These machines use 3/4 inch cassette tape as opposed to the 1/2 inch VHS standard.

This was originally developed for ENG (Electronic News Gathering) and can be carried off the shoulder with a camera by a solo operator. There are three types: low band, high band and high band SP (Superior Performance), with increasing degrees of resolution and replay quality. The U-matic standard studio playback and editing machines can all be fitted with time code units and synchronised centrally as described in part 2 of this series.

This gear is not mass produced to the same extent as domestic VHS, so it does not come cheap. A new portable U-matic recorder starts at around £2000, editing recorders and cameras costing from £3000 to £16000 depending on facilities and quality. The video market, like the audio and computer markets, is continually updating and used equipment can be picked up at a fraction of these prices.

JVC and Panasonic have a range of high quality VHS based equipment now but these are still aimed at the semi-pro market with a price tag to match. Sony have two more formats, Betacam and one inch reel to reel video recorders, and have just developed a completely digital video recorder - strictly for professional broadcasting.

As a general guide to prices of video equipment, take the equivalent home/semi-pro/pro audio recording equipment and add at least one nought. So if you want to be involved with video or audio for video production work (and you are not Rupert Murdoch) where do you start? There are a few alternatives.

Hiring Equipment

Practically every type of gear, including cameras, VCR editors, lighting, audio equipment and accessories can be hired from one of the many A/V hire centres throughout the UK. These people can usually point you in the right direction, but it is a good idea to hire the engineer as well to help you set things up. All this can add up so make sure your bills are covered by the value of the final production, and keep to your costed out time scales.

Music/Sound Tracks for Video

More and more institutions are now using video facilities for product promotion, training and presentations. The police, schools, fire service, local government, colleges, hospitals and countless local and national companies are all new growth areas for the medium. Many of these have splashed out on the new video equipment technology, but will have paid little attention to the sound/music track requirements. Small commercial and home recording studios are suitable for economically providing sound tracks to the clients requirements. Given the time coded working VHS cassette copy a fairly competent musician could put together high quality music/sound/effects tracks with basic synth, sampler, sequencer and recording equipment using techniques described last issue.

Video for Music

For short productions without sound effects of two minutes or less, music to picture sync is not usually required. The video engineer overlays an audio recording of specially composed music just as he would library music. On the other hand, a composer may have recorded a highly polished sound track, to create the right atmosphere for an advert for instance, to which a series of video sections are edited together for a visually and audibly coherent effect.

This leads nicely on to music videos which, since Queen's Bohemian Rhapsody back in the 70s have become immensely popular as a promotional medium. About one hundred singles are released every week in the UK, and most of these will have videos, of varying quality and tastes. Just like singles, not many music videos are actually sold. They are shown on TV and in bars and clubs to promote singles, which in turn promote albums, which make money for the record companies. The visual image of a group these days often has more to do with their success than their musical ability (did anybody mention Bros?).

Music promo videos are almost exclusively professional affairs involving artistic direction and loads of high tech gear and engineers, often resulting in stunning visual effects and computer animation. Innovations in the art form have pushed forward the frontiers of presentation and technology in broadcast TV and film.

Videos can cost anything from a few hundred pounds to a few million (eg. MJ's Thriller). The bill is usually footed by the record company, who take a risk on the single's success, and a proportional cut in the profits. So if you want a video making for your latest smash hit single talk to your agent or friendly record company.

The Making of Music Videos

Despite their eventual format, most videos (and TV films) are actually filmed on celluloid. True film cameras generally give a clearer, brighter picture than video cameras, which carries across to the video copies. The group are filmed performing usually three of four times against different backgrounds, miming to their record. SMPTE code from the music track synchronises each shot for reference and off line video copies are taken into the edit suite where the director and engineers sort out the best bits and assemble them together in time with the music.

The successful formula seems to be a succession of bright, colourful panning or movement shots, rarely lasting more than four or five seconds between cuts to match the attention span of the audience. Once everyone is happy with the working copy, computerised editing machines (as described last issue) are used to mix down the actual film shots 'on line' synchronised to the sound track.

Do you remember the Police video 'wrapped around your finger' where Sting and the boys are jumping around in slow motion, but still singing in time with the record? How did they do that? Well they were filmed at normal speed, while miming to the record played at double speed. When the film was slowed down to half speed they were apparently singing in time with the normal speed record, but moving around at half speed. Simple, but effective.

Vertical Interval Time Code (VITC)

Now back to the technical stuff for a minute. VITC is a kind of SMPTE code format used in professional video suites that has a number of advantages over longitudinal time code. To understand what it is we must first look at how the video signal itself is recorded on tape.

Video signals carry far more information than audio signals. Broadcast video has a bandwidth of some 5Mz compared with about 20kHz for professional audio. To record all this information video recorders have to use a much faster write speed, that is the speed the tape moves across the record heads. It does this by wrapping the video tape in a helix around a revolving head drum, recording video frames across the tape at an angle to the direction of tape travel.

LTC is recorded alongside the picture sound on a separate audio track at the edge of the tape. The problem is that you can't read the LTC unless the tape is actually playing, just like you can't hear music when the tape stops. However, you can still see a frame of video because the tape heads are still spinning around the same frame even when the tape is not moving along (ie. freeze frame).

VITC is an hrs/mins/secs/bits digital time code similar to SMPTE but modulated up into the video frequency range and inserted at the end of each video picture frame. In this form the time code can always be read by the video heads as the film is advanced frame by frame for highly accurate cue points and editing.

VITC is inserted by the video heads (sometimes a VCR will have an extra record/play head for this purpose) onto the two lines in the 'field blanking' period of the video frame, which is the bit above and below the visible TV screen next to the Teletext signals. Like teletext, you cannot read it without a VITC reader/demodulator device.

VITC uses 90 bits per frame (10 more than LTC) and it disperses its extra bits among a few functions. Bit 35 is used to identify video fields (every video frame is made up of two interlacing fields usually labelled odd and even) which in effect doubles the editing accuracy resolution. Bits 82 to 89 constitute a Cyclic Redundancy Check for error detection.

But VITC can be a far more complex time code to use in practise. It must be recorded simultaneously with the video signal which presents a problem when editing from source machines to the edit master VCR, since new code must be recorded on the master along with the video. The first segment of video recorded is no problem. When the time comes for the second edit, time code must be recorded on the master in such a way that there will be continuous, uninterrupted code. So the last bit of code from the first edit (say 00.26.06.08) is used as a start time by the VITC generator which inserts new code jam-synced to the incoming video segment signal (starting at 00.26.06.09).

Although jam-sync is used when working with both LTC and VITC, it is an essential feature of a VITC generator. In practise both formats are used and all VITC equipment can simultaneously cope with LTC.

The Control Track

One more important point to note is that many VCRs have a built in 'cue track' counter. This utilises the control track pulses recorded on video tape as a reference for making edits in a 'non-time code' environment. A control track editing system consists of a single playback machine, a record master machine, and an editor. These systems are fast and inexpensive, and when combined with some video special effects devices they can produce quite professional results. Using the control track is not reliable to within a frame though, and has synchronisation limitations relative to SMPTE based systems.

Transport controls for both the playback and record video machines are found on the control track editor. Some kind of readout that corresponds to hours, mins, secs and frames will also be found on both machines. These readouts look confusingly like SMPTE, but they result from counting control track pulses and are not based on reading time codes from the tapes.

Next Time

As I mentioned in the introduction, gen-lock systems for PCs will be the main feature next issue. Staying with PCs, I will also be looking at some of the more popular MIDI sequencing software, and how well they link up to video.

One thing that puts people off high tech gear is high tech jargon and TLAs (Three Letter Abbreviations). So next issue will also include a glossary covering all those useful formats and standards that will help you speak the language.