MIDI Time Code
The marriage between MIDI and SMPTE is now complete, but the burning question remains: how will musicians and engineers benefit from it? Chris Meyer traces the link from its source to its users.
MTC is here to unite the worlds of audio and video composition, and some of its applications hit closer to home than you might expect. What makes it tick?
WE FIRST MENTIONED MIDI Time Code (formerly called "MSMPTE") last summer. An extension to the MIDI 1.0 specification, it was actually adopted by the MMA (MIDI Manufacturers' Association) and the JMSC (Japanese MIDI Standards Committee) at the summer NAMM show in Chicago at about the same time. It was discussed for almost an entire hour, and then Gerry Lester of Adams-Smith (makers of audio and video tape machine synchronisers) proposed a righteously better way of implementing half the specification, and the arduous approval process started all over again.
The reworked specification was finally re-approved in February, 1987. This year's summer NAMM show, however, is where we will start seeing the first applications of MIDI Time Code. With this in mind, now is the best time to discuss exactly what this new area of the MIDI spec is all about, and what some of its potential applications are.
AS HAS BECOME obvious by now, it is not necessary to know all of the bits and bytes of MIDI to be able to use it successfully. However, there are those who are insatiably curious about the details anyway, and it is those people we'll quickly oblige. (For those who don't need to know, we'll try to make this as painless as possible.)
As the full-length title implies, there are two parts to the MIDI Time Code and Cueing Specification - sending timecode over MIDI, and sending a "cue list" via "Set-Up" messages over MIDI.
SMPTE Time Code is the standard language of synchronisation in any video studio, many film studios, in audio studios where synchronisation between tape decks or mixing-desk automation is taking place, and in any cross-breeding between these situations. Given such a weighty task, SMPTE Time Code itself is very simple - all it conveys is what time it is in hours, minutes, seconds, and some fraction of seconds (from 24ths to 30ths - referred to as "frames"), along with eight additional characters for whatever information a user prefers to include (typically, the session date or tape reel number).
There is a SMPTE Time Code message for every frame, consisting of the above information plus a couple of notes concerning colour, frame, parity, and special conditions. There are also 16 bits to signify the end of a frame, making the overall total (in the most common case) 80 bits. All SMPTE devices work from this common "time of day" to stay in sync with each other, and then perform whatever local conversions are needed to get their job done.
In contrast to its simplicity of content, the actual reception and decoding of SMPTE Time Code is not easy. It is either embedded in the video signal (no mean feat to decode), or comes as a series of binary 1s and 0s that occur as often as every 417 microseconds. On top of all of this, Time Code readers have to be both deadly accurate and tolerant of faults (such as tape dropout) to work properly.
This (along with the fact that the demand has surged only recently) is why, until now, SMPTE Time Code readers for music and semi-pro applications have tended to cost around £1000 (and why readers for professional applications still cost a few thousand). Needless to say, adding reliable compatibility with SMPTE Time Code to normal musical instruments has not, in the past, been one of your cheaper propositions.
Conversely, MIDI is already implemented on the majority of musical instruments, along with many semi-pro effects devices. With the hardware cost and the knowledge of how to decode MIDI already in place, it would be nice if receiving SMPTE Time Code became just a software proposition.
THE "TIME CODE" portion of the MIDI Time Code spec is what puts the above idea into practice. It consists of three types of messages: one that transmits just the User Bits (those eight characters of additional information mentioned above - they usually stay the same throughout a reel of tape, or at least throughout a song or video take, and therefore only need to be transmitted once in a while); a Long Form message (a Universal System Exclusive code which, in one message, brings devices online up to the current time where the system is parked); and a series of eight two-byte messages that convey the time while the system is running. Because the first byte of each two-byte message is always a hex F1 (a previously unused System Common message in the MIDI 1.0 spec), these eight messages are usually referred to as the "F1 messages".
These eight F1 messages each carry a piece of the current time - two messages for frames, two for seconds, two for minutes, and two for hours (with one of the hour messages also telling how many frames there are in a second). Four of these messages are sent per frame, equally spaced throughout the frame. Thus, it takes two whole frames to tell what time it is. (Of equal importance as the information inside the messages is when the messages arrive; they mark quarter-frame boundaries.) A happy consequence of this implementation is that very little of MIDI's potential bandwidth - that is, its total capacity for data - is taken up (at the fastest frame rate, 7.7%), while time is updated very often (four times per film frame or twice per video field - roughly every 8-10msec).
Once a receiver reads eight messages and has assembled a valid time, it can keep counting internally and just use the new messages to track fluctuations in tape speed and the like. If need be, the receiver can also interpolate finer times between the messages to keep up accurately and place such things as tempo clocks.
THESE MESSAGES ARE a series of Universal System Exclusive messages that determine when (in SMPTE or MIDI) a particular event is supposed to occur. These "events" might include when an action (such as a sound effect) is supposed to start or stop, when to punch in or out on a particular track on a sequencer or tape deck, or when an event (such as a reverb program-change) is supposed to occur. They also provide the ability to edit this information remotely in a device by deleting events and so on. Each Set-Up message may have additional information encoded within it in the form of a System Exclusive message for the specific device.
Special Set-Up messages include notes to request a cue list from a device, to disable performance of the cue list, to offset the time at which a device is supposed to react (two tape decks that are supposed to synchronise may have different times actually printed on their tapes), and to tell when the whole system is supposed to stop whatever it is doing.
As many as 127 devices may be individually addressed (or "spoken to") and, therefore, set up. And as many as 16,000 different events (tape tracks, sound effects, and so on) may be addressed for each device in the system.
The idea behind the Set-Up messages is to create a cue list which consists of actions that are supposed to occur at specific times. This list is then broken down into a series of Set-Up messages (one per cue, or event) which are transmitted ahead of time to the receiving machines. Then, when these devices see the MIDI Time Code or SMPTE time that matches the time of a cue, they perform that event.
One point worth noting is that there is still room in the specification for over 100 more "general" and 16,000 "special" types of Set-Up messages, so there is quite a bit of room for expansion in the future.
AT THIS EARLY stage, MIDI Time Code is already being confused with music sequencing via MIDI and MIDI clocks. This is a tempting comparison to make. Both use MIDI, and both are about "events" happening in a specific order at specific times provided over MIDI. However, carrying the comparison much beyond this point only obscures the true applications of MTC, and therefore causes confusion.
"Normal" MIDI concerns itself mostly with musical events. A typical MIDI event tends to translate directly to a musical gesture - such as a note-on.
A MIDI Time Code event has a much broader meaning - it can mean starting a whole sequence, telling a tape recorder to punch in a certain track at a certain time, changing a digital reverb's delay parameter, or whatever.
And whereas musical events are timed according to which bar, beat, and clock they fall on, MTC events are usually referred to in terms of SMPTE Time Code - at which frame of a picture a sound effect is supposed to fire, at which time a tape deck punches in and out, and so on. It's best to think of MIDI Time Code as "sequencing" all the other motions that go on in a studio aside from the music itself. Keeping this in mind will, with luck, cut down on the confusion.
Typically, far fewer MTC events occur in a piece of music or scene of film than MIDI musical events, for the simple reason that one rarely changes reverb programs as often as one strikes a snare drum, for example. Despite this, it is good to keep the datastreams of the two systems separate, so as to keep them from competing for precious MIDI bandwidth. This is not too much of a hassle, since the two classes of information tend to be aimed at two different sets of equipment: in the above example, a drum machine as opposed to a digital reverb, though there are crossovers, obviously. Perhaps the best aspect of MIDI Time Code (and its closest analogy to MIDI music sequencing) lies in its ability to bring the co-ordination of many different devices in a studio up on one computer screen.
But we're starting to get ahead of ourselves - let's start talking about specific applications.
THANKS PRIMARILY TO Fostex, more and more home studios are using SMPTE Time Code. As mentioned above, getting our MIDI sequencers, drum machines, and so on to sync up with SMPTE has meant spending about a grand on a SMPTE-to-MIDI clock converter such as the Friend-Chip SRC, Garfield MasterBeat, Roland SBX80, or Fostex 4050. Recently, interface boxes for computers such as the Atari ST (eg. Hybrid Arts' SyncBox and the Steinberg SMP24) and the Apple Macintosh (Southworth's Jam Box) have started to include SMPTE readers at about a half to a third of this cost.
Amazingly, SMPTE-to-MIDI Time Code converters have already appeared for as little as $200 in the US (the JL Cooper PPS1).
I expect we'll soon see sequencers (hardware and software), drum machines and the like starting to incorporate the ability to receive MIDI Time Code directly. This will give them SMPTE capabilities with no additional hardware cost, just a new PROM or system disk. Systems built around these units will only require a single SMPTE-to-MIDI Time Code converter. Considering that the aforementioned SMPTE-compatible computer interfaces are also converting SMPTE to MIDI Time Code for re-transmission, there should be no shortage of such converters.
One of the latest hot topics of conversation these days is semi-pro automated mixing. Currently available devices use MIDI messages to represent fader levels and other such information, and then send it out to be stored along with normal performance data in your system's MIDI sequencer. One such unit - JL Cooper's MAGI - has a built-in automation sequencer that sends and receives MIDI Time Code. There is also software on the horizon (Digidesign's Q-Sheet) that will sequence and provide moving fader representations synchronised to MIDI Time Code for mixers such as the Yamaha DMP7 and the new Twister.
Stepping out of the strictly "musical" realm for a moment, audio-for-video is perhaps the field most eager for MIDI Time Code. Sound effects for film and video have typically all been performed live against the visual image, or triggered from a library of sound effects on cart machines (a variation of the old eight-track players). The creation of these effects is often referred to as "Foley", with the performers being called "Foley walkers" (since one of the most common needs is to recreate the sound of somebody walking).
But nowadays, more and more studios are storing their sound effects on disks for sampling keyboards (the now venerable Emulator II being the most common one) and are using these where the subtleties of a live performance are not needed.
A new "electronic Foley" person lays in sound effects either by playing the keyboard in real time against the video image, or by recording the effects as events in a sequencer which is then synchronised via SMPTE to the video. This last method has been a tedious process in the past, since a lot of calculations are needed to convert tempo, bars, and beats accurately to the video or film frame numbers. More often than not, a sequence is created for each sound effect that has to be triggered, with the sound being on the downbeat and the sequencer's start time being set to that of the desired video frame.
Now, all that should change, with the advent of what I'll call "MIDI Time Code Sequencers". These should allow events to be logged to occur at certain MIDI/SMPTE times instead of bars and beats. The appropriate MIDI events (such as a note-on) are then sent when the appropriate times are received, allowing whole strings of sound effects to be laid in against film or video quickly and with great precision. Some studios are already re-editing dialogue on samplers, and one day, they will also be using these tools to re-sync words to picture.
ALL OF THE above applications are being made reality with just the timecode portion of the MIDI Time Code specification. But life in the MIDI world will move up to a new level when the Set-Up messages start to be applied.
There are a couple of reasons why Set-Up messages are more attractive than the normal, existing MIDI messages for cueing and triggering events. First, some of the applications mentioned, such as automated mixdowns, require a great deal of normal MIDI information to be exchanged. Those who have complained that MIDI is too slow will really start to scream once they try to move 16 faders at once via MIDI. Having the information for those moves sent to or stored in the target device ahead of time, and then merely sending "F1" messages during real time, would make such feats much more feasible.
Second, more details about an event can be encoded into a Set-Up message than can be conveyed using normal MIDI performance messages. For example, the Sequential Studio 440 is capable of firing a sample in response to a MIDI note-on message, as are most samplers. However, the 440 also has the ability to receive and execute a cue list that contains such information as sound number, pitch, panning, velocity, and overall level for each event, as opposed to merely saying "play a middle C at this velocity - whatever that happens to mean to you". Effects devices can have programs downloaded to them along with the time they are supposed to change, as opposed to just responding to MIDI program changes. In general, Set-Up messages (and devices that implement them - both MIDI Time Code sequencers and target units) will allow for even more precision and more portability between different studios than normal "dynamic" MIDI.
Last, and most important, Set-Up messages help to distribute intelligence around a system. A particular device knows better than any other how it's going to react to a given action, and is therefore in the best position to prepare for it. A sampler (or a tape recorder, or even a cart machine) that knows it has a certain amount of delay could internally offset SMPTE times backwards so that it actually fires the sound on time. Samplers with hard disks may even be able to download the appropriate files ahead of time, once given this list of the samples they will be using. A tape deck could internally offset current ramp-up and ramp-down times for its record bias, to make punch-ins and punch-outs tighter. A MIDI sequencer could set up its record buffers ahead of time if it knows that a certain track is going to start recording.
In general, knowing the future makes planning for it a lot easier, and receiving a cue list of Set-Up messages ahead of time gives machines this power.
Where will this all lead? With MIDI Time Code, MIDI seems to be perched on the verge of becoming the recording studio's auxiliary automation bus. For those who can't afford an SSL Total Recall system or own devices that don't respond to it, it'll become possible to automate, at the very least, effects devices (ADA already have a programmable graphic equaliser that responds to MIDI Time Code and Set-Up messages, with promises of more devices to come) and MIDI sequencers, with the possibility of continuing up through tape decks and mixing consoles.
All of this automation power can come from one "black box" that displays, edits, and manages cue lists for each device in a system and tells them all when to do their thing. The hardware cost of adding these capabilities is minimal, and the software cost is far from outrageous, certainly by studio standards. And given the level of sophistication that MIDI music sequencers have already achieved, equally friendly and powerful MIDI Time Code sequencers are certainly not a dream.
"FINE" YOU SAY. "But I've been surviving this long without MIDI Time Code; therefore, I obviously don't need MIDI Time Code to live". Which is true enough. Yet the system does offer some amazing possibilities...
Just as MIDI Time Code is considered an "extension" to the MIDI 1.0 spec, so it also presents extensions to the possibilities of an automated set-up. No one ever needed digital Foley, but it's certainly made laying in sound effects tightly against film or video a lot easier, and made some new tricks possible as well. No one ever needed automated mixdown (let alone total automation of tape decks, effects devices, and sequencers to boot), but again, it's made some tricks a lot more precise, a lot easier, and in some cases, just plain possible.
Your "need" for MIDI Time Code should be viewed in the same light. If you intend to start operating on this new, higher level, then you should start looking for manufacturers who are going to implement and provide this power.
What MIDI Time Code and Cueing will probably become best known for is consummating the impending marriage of the "MI" ("Musical Instrument") and semi-pro audio markets with the high-end pro audio and video fields. Semi-pros will have the power which, until recently, was reserved for those who could afford to see a couple of extra zeros on the end of a price tag, while pros will get to integrate some of the neat toys from the MI and semi-pro market without banging their heads against a wall.
Democracy at work.
(Although Chris Meyer is known as the father of MIDI Time Code, he would like to extend credit to those who also contributed significantly to the specification and its adoption: Evan Brooks of Digidesign, Jim Cooper of JL Cooper Electronics, Anne Graham of After Science, and Gerry Lester of Adams-Smith. For a copy of the MIDI Time Code and Cueing Specification itself, contact the International MIDI Association, (Contact Details). The spec may also be downloaded, along with the text of a conference on the subject, from the Synthesiser and MIDI forum on PAN.)