Paul Overaa continues his series on the technicalities of MIDI. This month - The mysteries of System Exclusive
As far as MIDI is concerned you can forget about music being sounds which you hear. MIDI deals with digital data, streams of numbers and these keep track of everything from the music you play to the knobs you're turning on your synthesizer. Their meaning is defined in the MIDI standard... each number, or group of numbers, will be part of a specific MIDI message, and it's because these messages are being transmitted that other pieces of equipment get to know what's going on as you play.
MIDI messages follow a fixed format: First of all comes a status byte which identifies the type of message. Some messages are in fact only one byte long and in these cases the status byte is the actual message itself. Other messages, often called 'multi-byte messages', include extra data which follow the status byte. We ought to mention in passing that an arrangement known as 'running status' allows some types of messages to use an 'implied' status byte. Running status lets us avoid sending unnecessary additional data down the MIDI lines but, at least for the purposes of explaining MIDI's message structures, it's not something we need to worry about.
So what types of MIDI messages exist? Quite a few to be honest and this month we're going to look at the three classes - system exclusive, real-time and common, which are collectively called 'System Messages'. These messages deal with information which is likely to be of general interest to more than one piece of equipment - they're not channel specific and so they do not include a channel number in the status byte.
System Exclusive Messages
Since MIDI is essentially a 'standardized' interface it may seem strange that this standard provides a facility for allowing 'non-standard', manufacturer specific, information to be transmitted and received between MIDI units. This is however exactly what 'System Exclusive' (SYSTEX) MIDI messages are, and when you see how they operate you'll appreciate why this has been allowed.
SYSTEX messages can be of variable length. To start with there must be some way of knowing when such a message is arriving. This is where the SYSTEX status byte comes in... the first byte of a system exclusive message is the number 240 decimal, so as soon as a piece of equipment reads a byte of data with this value it knows that a SYSTEX message is arriving. What won't be certain is whether the message will make any sense! The second byte of the message helps here because it is a Manufacturer's Identification Number - a numerical code allocated by the MMA (MIDI Manufacturers Association). Each manufacturer designs their own 'data block' and, providing the unit recognizes the identification number, it will proceed to interpret the data that follows according to the formatting rules which the manufacturer has devised. Here's a few of the current identifications which have been assigned...
|Decimal Value ||Maufacturer |
|1 ||Sequential |
|4 ||Moog |
|5 ||Passport Designs |
|14 ||Alesis |
|15 ||Ensoniq |
|20 ||Fairlight |
|23 ||Linn |
|65 ||Roland |
|66 ||Korg |
|67 ||Yamaha |
|71 ||Akai |
If a unit does not recognize the ID number then it simply reads the data, but ignores it. How does it tell when the message is over? Simple, it just waits until it receives a byte of value 247 decimal, which signifies the end of a system exclusive message (this end byte is known as the SYSTEX terminator or EOX byte). At the end of the day a complete SYSTEX message looks like this...
In theory at least, a SYSTEX message can be terminated not just by an EOX byte but by any new non-real-time status byte. So that, in a nutshell, is what a SYSTEX message is! The arrangement allows a manufacturer to bundle up their equipment specific control stuff in a nice tidy packet which can be handled cleanly by any piece of MIDI equipment, even if it doesn't actually understand what the message is about.
From a practical viewpoint there's quite a difference between these SYSTEX messages and the normal run-of-the-mill data stream that occurs during normal playing. The difference is this... when you are playing keyboards or playing back sequenced music the data stream will have gaps, quiet spaces, in it. In general MIDI messages are short (1-3 bytes) and even the odd gap of a few hundredths of a second is quite sufficient for the computer chips that are built into MIDI equipment to analyse and keep up with incoming data. SYSTEX messages are much larger, often between 5000 and 10000 bytes, and they do not contain any of the gaps found with normal playing. This means that the unit has to work hard when receiving this type of data - which in turn means that the risk of potential transmission/reception problems is greater.
SYSTEX message reception can cause problems with some types of equipment and many sequencers include facilities for allowing artificial gaps to be inserted into SYSTEX data thus easing the burden on the receiving equipment.
It is incidentally, exactly this type of 'density' problem, i.e. MIDI messages getting too closely packed, that produces the effect called 'MIDI clogging' - it's just a case of having too much data going down the MIDI line too quickly. The solutions, which may include filtering out unwanted information, or cutting down on pitchbend and controller use etc., all have a common aim - to reduce the amount of message traffic that is flowing down the MIDI communications lines.
Real Time Messages
These messages deal with system timing, start, stop, continue, active sensing and system-reset facilities and have the following numerical values..
|Message Name ||Decimal Value |
|Timing Clock ||248 |
|Start ||250 |
|Stop ||252 |
|Continue ||251 |
|Active Sensing ||254 |
|System Reset ||255 |
Most of these commands are time critical and need to be inserted into the MIDI data stream when needed - irrespective of whatever else is occurring... MIDI clock bytes for instance must, in order to keep accurate time, be inserted into the data stream at exactly the right times.
Because of this, these real-time single byte messages can be inserted anywhere in the MIDI data stream, even between the bytes of other multi-byte messages (that's one of the reasons that these are all single byte messages - it makes the associated programming easier!).
MIDI clock bytes are very important because they govern the overall system timing, i.e. they provide the anchor point which allows the time relationships between note data (and other MIDI events) to be specified. MIDI clock bytes are sent at a rate of 24 clocks per quarter note and it is the presence of this reference data which lets all instruments synchronize to a common 'system time'. Needless to say you must only ever have one piece of MIDI equipment generating clock data otherwise things get more-than horribly confused.
Whilst we are talking about timing we might as well mention the debate about whether 24 clocks per quarter note is a sufficient resolution. Certainly, for a lot of professional uses the answer is NO, a higher resolution clock would have been better - but... the decision to use the current resolution was based on sound economics. If a higher resolution clock had been adopted then more demands would have been placed on the computers present inside MIDI equipment.
A super-speed clock would have had many programming implications and at the end of the day it would have been you, the user, who would have footed the bill. MIDI is affordable mainly because those who formulated the spec made a very realistic appraisal of the potential intended MIDI market - we could easily have ended up with a super-high-speed specification using parallel data transfer (needing connecting leads costing maybe £20 a go) and which required far more computer processing power. Granted we would have had a great spec, but 99% of us would still be looking at the equipment in the shop windows rather than being able to go out and buy it.
Let's also be a bit fair when comparing the MIDI clock resolution to other timecode arrangements and remember that MIDI communications doesn't just have to handle timing data - it must deal with note data, controller data, real time messages and lot more besides... all at the same time!
Start, Stop and Continue messages can be sent from master keyboards, sequencers, drum units etc. When for example you select the internal clock on an RX-8
drum machine and start playing a pattern, several things happen... Firstly a MIDI start message (decimal 250) is transmitted, this lets all other connected units know what has happened.
Secondly the drum notes (and timing data) contained in the pattern are transmitted. The note and timing data continues to be transmitted until such time as you press the stop button, and at this point the machine stops sending data and transmits a stop message (decimal 252), again to let connected units know what has happened. If you then hit the 'continue' button, the RX8 firstly transmits a continue message (decimal 251) - and then resumes playing (and transmitting its data) from the point where it originally left off.
Active sensing is a great idea although many products don't seem to implement it. It's purpose is to enable a piece of equipment to tell whether there has been a break in the communications lines - faulty leads, units being accidentally unplugged etc. The use of active sensing is optional and if you're units never receive active sensing data then they'll pretend it doesn't exist. Once a unit does receive active sensing information it will then expect to receive either some sort of MIDI data (or at the minimum an active sensing message) at least every 300 milliseconds.
If this data doesn't arrive the unit will simply turn off all notes which are sounding and return to a passive, i.e. just switched on, state. You occasionally read that active sensing, although essentially a good idea, can contribute to MIDI clogging problems. This simply isn't true.... properly implemented active sensing only needs real active sensing bytes to be sent when there is no other data on the MIDI lines. During MIDI-load conditions, i.e. when MIDI data is being transmitted, there's no requirement for active sensing bytes to be transmitted at all!
System Reset messages do exactly what you might expect they instruct all of the connected units to assume a just-switched-on state. These messages can be useful, especially if the power up states of various units are themselves programmable.
Song Position Pointer: Units which use an internal register for counting MIDI beats are able to use these messages to let you start playing a song from a place other than the beginning. The messages are three bytes long and take this sort of form...
|Status Byte ||Low Data byte ||High data byte |
|decimal 242 ||0-127 ||0-127 |
Identifies a song or sequence to be played. It's a two-byte message and as always the actual data byte can be any value from 0 to 127 decimal.
|Status Byte ||Data byte |
|decimal 243 ||0-127 |
This single byte message (decimal 246) asks all units within the system to 'tune' themselves. It's original purpose was to ask analogue synthesizers to tune their oscillators but, when acted upon, this command only guaranteed each synthesizer would be 'internally' in-tune... it did not ensure that a collection of synthesizers would all be in tune with each other!
Well that's the story as far as System messages go. Next month we'll have a look at the messages which are concerned with notes, voices and sounds, i.e. MIDI's channel messages.