Home -> Magazines -> Issues -> Articles in this issue -> View
Everything You Ever Wanted To Know About System Exclusive (Part 1) | |
(But Were Too Afraid To Ask!)Article from Sound On Sound, April 1989 |
Martin Russ kicks off our new series by recapping on some MIDI basics before starting to demystify the secrets of MIDI's System Exclusive commands.
Soap opera script writers are notorious for deliberately writing loose ends into their plots - it gives them a way out if something unexpected happens, like one of the cast leaving! Writers aren't the only ones who do this. You might be surprised to learn that the designers of the MIDI standard also left some loose ends on purpose. The idea is that by leaving in a few loopholes you allow the evolution of the standard into unexpected areas - and with the high rate of development in musical technology, they were right to expect the unexpected!
In order to find the loose ends in MIDI, we need to know something about MIDI itself.
MIDI has two basic types of message: Channel messages are used to convey information in the 16 permitted channels, and consists of messages describing Note Ons and Offs, Pressure, Pitch Bend, Program Changes and Controllers - all related to actual live performance data. In contrast, System messages are not associated with a channel, and so can be received by any piece of MIDI equipment connected to the network. This global information deals with the overall timing and control aspects of MIDI: the Timing Clock and MIDI Time Code messages; Start, Stop and Continue commands; as well as Active Sensing and Tune Request messages.
MIDI information is transmitted in the form of messages - each message normally consists of a status byte (1 byte = 8 bits), usually followed by one or more data bytes. The status bytes are indicated by setting the most significant bit (MSB) of the byte to 1 (the most significant bit of the byte is set to 0 for all data bytes) - this means that the basic interchange level of MIDI is really only 7 bits. The status bytes for Channel messages are split into two nibbles (4 bits): the high nibble indicates the type of status message, whilst the low nibble indicates the MIDI channel for which the data that follows is intended.
There are seven fundamental sorts of Channel message and these are assigned high nibble values from $8 to $E in hexadecimal notation. (Remember: the $ dollar symbol is just there to remind you that the hexadecimal numbering system is being used, as opposed to decimal). The $F high nibble value is reserved for the System messages.
System messages use the same two nibble split, but in this case the high nibble is always $F and the low nibble is used to indicate the type of message - there are thus 16 possibilities and they are split into three parts. $F0 indicates a special purpose message called 'System Exclusive', and the details of this will form the bulk of the rest of this series.
$F1 to $F7 are the System Common messages, used for controlling and enhancing other functions. The $F2 Song Position and $F3 Song Select messages are used to control drum machines and sequencers, for example. MIDI Time Code (MTC) sends its timing information as $F1 messages every quarter frame and additional information by using $F0 System Exclusive messages - all for film and video synchronisation purposes.
Analogue synthesizer users might employ the $F6 Tune Request message to activate built-in oscillator tuning routines in their instrument before playing. $F7 is used to indicate the end of a System Exclusive message, of which we shall see more later!
The remaining System status bytes, from $F8 to $FF, are called the Real-Time messages and they deal with the synchronisation of the MIDI system. $F8 is the MIDI Timing Clock, a complete message in one byte - sent at the rate of 24 clocks per quarter note. $FA, $FB and $FC are the drum machine controls - providing remote Start, Stop and Continue commands. Active Sensing uses the single byte $FE message to indicate if a remote keyboard or MIDI guitar is still active (ie. still transmitting MIDI data), and can be used to mute expander modules if the connection fails.
Finally, $FF is the System Reset message — a panic button designed for use only in cases of dire emergency, since it restores the equipment to its power-up state. In reality, most manufacturers seem to tactfully ignore this 'kill or cure' message, with the exception of some European companies obviously eager to prepare for 1992.
When I was a kid I used to read stories of the days when maps of Africa had large areas marked 'unexplored'. The 'undefined' parts of the MIDI map have the same fascination - but where are they? All the Channel messages seem complete, so perhaps a deeper look is needed - the status bytes may all be used, but within the data there are more possibilities. Simple data messages which convey just values will not offer any room for expansion, so the Note Ons and Offs, Pressure, Program Change and Pitch Wheel messages are not worth exploring. This leaves just the Control Change message, and it is here that we find our first 'undefined' region!
MIDI Controllers are defined by the first data byte following the $B nibble of the status byte. The 128 possible Controllers were originally organised into three basic types: Continuous Controllers (modulation wheels, pedals etc), Switches, and Mode messages to control note reception conditions. However, there has been some reshuffling and MIDI Specification 1.0 has complicated this situation. Continuous Controllers are now divided into 14-bit (Controllers 0 to 63) and 7-bit types (64 to 95). Switches are now assigned as a special case of a 7-bit Continuous Controller. Not all Controller numbers are assigned to specific devices like pedals, wheels, etc. In fact, most aren't. Those that are seem to be driven by historical precedent, although the MIDI ruling bodies are gradually imposing some order on things. I don't really count the Unassigned Controllers as areas ripe for exploitation, since consistency is what is needed here - you want a volume pedal to control volume, not LFO rate.
Mode messages, which you might have expected to find in a message all their own, are instead hidden in a quiet backwater - Controllers 122 to 127. The Mode messages control the operation of the MIDI peripheral which is receiving the data: Mono On/Off, Poly On/Off, Omni On/Off determine the way it responds to incoming channelised information such as note or pitch wheel data. Local Control On/Off (122) sets the link between the keyboard of a synthesizer-type device and its internal sound generators - if it is Off then you can play the keyboard and use the MIDI Out, but the only way to produce sound is by feeding note information into the MIDI In.
The final Mode message is All Notes Off (123) - a sort of catch-all designed for when you halt playback on a sequencer whilst some notes are still playing, although this message often seems to be used to indicate when all the keys on a keyboard have been released. This has the unfortunate effect of annoying people who use expanders for their source of sounds, since every time they release the keys on their master keyboard, the expander stops playing, even with the sustain pedal held down! All Notes Off used in this way is not illegal as far as the MIDI Specification goes, nor does it seem to be completely legal either, but the problems it can cause serve a very useful purpose in keeping MIDI consultants like me in work!
The astute amongst you will have noticed that I omitted Controllers 96 to 121. This group is split into two sections - the first covering 96 to 101 is the Registered and Non-Registered Parameters section, where you can assign up to 256 parameters to MIDI Controller messages without using up valuable 14- or 7-bit Controller numbers. In fact, you can use just Controllers 06 and 38 to control the MSB (most significant bit) and LSB (least significant bit) of all these extra parameters. The remaining Controllers, from 102 to 121, are officially undefined. (Except for Controller 121, which is used for the Reset All Controllers message - a sort of All Notes Off command but for Controllers instead of notes. This is not documented in my copy of MIDI Specification 1.0).
If you think the story so far is complicated, then you would be right. MIDI is still evolving, and the Controllers are definitely one area where developments are having a direct effect on the assignment of the commands. Because of this, treat what I have said so far as an outline - my information could be out of date by the time you read this. Even so, this brief look through the Controllers should have shown that the MIDI standard is capable of being improved and extended within its own framework.
The System messages are more restrained in their response to change. Outwardly, only one addition has been made: MIDI Time Code Quarter Frame messages are now sent as $F1 messages.
$F4 and $F5 remain as undefined System Common messages, and $F9 and $FD are undefined System Real-Time messages. All the big changes and additions are hidden deeper inside just one message - the System Exclusive $F0 message.
System Exclusive is rather like the blank pages at the end of a book marked 'Notes' - it is a place where the user makes his own mark on the book instead of just reading it. System Exclusive is a means of removing some of the fixed standardised nature of MIDI and letting individual manufacturers and designers use it for their own purposes. The bare minimum of restraint is imposed by the MIDI Specification - everything else is left to the designer of the MIDI device. This is the loophole which really counts in MIDI. It is the key to all sorts of expansion and improvement - turning MIDI from just a means of sending notes from one instrument to another into a powerful tool for hi-tech music. The remainder of this series will concentrate on a detailed look at some of the possible applications of the System Exclusive message.
It takes a brave person to start a series on System Exclusive and then talk about everything except that! As limited recompense, I offer an extension to the mix of theory, practical examples and printouts from software used in the 'Practically FM' series [SOS May 1988 to October 1988]. This time the software is an integral part of the articles.
Over the course of this new series, each episode will present a piece of software on 3.5" disk (for the Atari ST initially: users of other machines will have to persuade me to convert the programs!) which illustrates and expands upon the ideas in the text. These are available from SOS for £7 per disk. At the end of the series, you will have a valuable and varied collection of software tools for use with MIDI, especially in the context of System Exclusive.
For anyone who prefers not to use computers, I will also try to show ways in which hardware devices can be used to achieve similar results. Last month's article about the Lexicon MIDI Remote Controller (MRC) is an excellent example of the way that a specialised piece of equipment can be used to control MIDI without needing to look or behave like a computer.
A good entrance is important in show business, and so this first article has two pieces of software to accompany it - one old, the other brand new. MIDI Channel messages can be viewed with the MIDI Channel Scope program (SOS Software Disk 06) whilst the System Real-Time and some useful testing messages can be sent with the MIDI Utility program.
Alternatively, many master keyboards and some sequencers allow you to decode Channel and System messages, while the Lexicon MRC allows many similar functions to the MIDI Utility program, as well as much more control over MIDI Controllers. (See 'Alternative Ways' box for further details.)
If you currently use MIDI purely for sending Note On/Offs and Program Changes to your synths, sequencers, and signal processors, then the rest of this series will come as a shock. You will be interacting with MIDI instead of just using it, so now is as good a time as any to see and learn what is happening...
Use the MIDI Channel Scope or another monitoring device to observe what is happening on your MIDI network. If you aren't actually playing something then you should see very little action. However, some drum machines and sequencers send out MIDI Timing Clocks all the time, so you might see these, and some early synthesizers send out Controller data even when the controller isn't doing anything. The Korg Poly 800 is notorious for continuously sending out messages telling you that its Pitch joystick and Modulation joystick are both in the centre position! Active Sensing can also appear in a 'quiet' system if your master keyboard transmits it. MIDI Time Code signals tend to be restricted to MIDI systems which aren't running ordinary MIDI Clock based synchronisation.
Once you play an instrument, you should see activity on the appropriate MIDI channels - Note Ons and Offs, Program Changes, etc. Mixed in with these you will still see the Clocks and Active Sensing messages as before. Aftertouch and other Continuous Controllers are interesting to try, because they send large amounts of data - usually a message for each change in value. (You really do get 128 messages one after another if you move a 7-bit Controller from one end of its travel to the other extreme.) Some synthesizers can also be confusing: early models of the Yamaha DX7 Mk1 (serial numbers 1001-24880 and 25125-26005) and KX1 (serial numbers 1001-1105) implemented Aftertouch as a Controller instead of as the now standard $Dn message (where 'n' is the MIDI channel).
Program Change messages may also be accompanied by lots of additional data: some synthesizers send a dump of patch information when you select a different program - the MIDI Channel Scope will not show this (but the MIDI System Scope program, available later on in the series, will!). Note messages may also appear confusing if you are expecting to see both Note Ons and Note Offs - the convention has quickly become one of using only Note On messages, with Note Offs instead being indicated by a Note On with a velocity of zero. This format allows you to send long sets of note number and velocity data bytes without needing to send any status bytes, and is known as 'Running Status'. Some 'home grown' and public domain software can be confused by this short cut. There is a test in the MIDI Utility for checking this - if you hear sustaining notes then your synthesizer or expander does not like Running Status, and if your drum machine does not start playing then it doesn't either!
Notice that the Controllers are channelised, ie. the data is sent over a specified MIDI channel. Remember this when you find that a Pitch Wheel is only bending one instrument. The all-important Channel number is part of the status byte. It is the low nibble, eg. the 'n' in $En (a Pitch Bend status byte).
One of the most demanding Controllers is Poly Pressure ($An), as seen on the affordable Ensoniq SQ80 and EPS keyboards. This generates messages for each note which is being pressed down - so it is probably a good thing that the human hand is restricted in the speed at which it can play notes and then apply controlled pressure to them!
This first episode has been an introduction to the aspects of MIDI which form the background to the activity using System Exclusive that will follow in subsequent articles. A knowledge of what is happening, and the ability to interpret the often obscure response of real instruments, will be of great value over the coming few months. I will supply some theory, practical examples, hints and some useful software tools, but you will need to supply some effort too in order to use them effectively!
The MIDI Channel Scope and MIDI Utility programs referred to in this article are available on 3.5" disks for the Atari ST only at present.
£7 per disk.
SOS Software, P0 Box 30, St. Ives, Cambridgeshire PE17 4XQ. (0480) 61244.
- Version 0.2 (at time of writing)
- MUTIL02.PRG
- MUTIL02.RSC
(The program works in both medium and high resolution - but the display is best in high resolution mode.)
Ever wanted to be able to send some MIDI messages from your Atari ST directly to your MIDI equipment? You need wait no longer! The MIDI Utility program lets you send any of a number of test routines and other MIDI data, including Channel, Real-Time and System messages.
USING IT
1. Choose a MIDI channel to send the channelised messages on - click in one of the boxes at the top of the screen with the mouse. (System and Real-Time messages are not sent on channels, of course.)
2. Click on the button showing the particular message you want to transmit.
3. Use the System Real-Time Start/Continue/Stop buttons to control your sequencers and drum machines, if you wish.
4. Zeroing the pitch bend whilst using the pitch bend wheel on a synth can produce some interesting effects.
5. Be careful with the Local Control On/Off button - you can stop your synth making sounds under the control of the music keyboard!
6. The Notes and Program Change buttons access a special control mode, where you use the mouse to send notes or program change data. Just move the mouse, note the settings shown in the text box, and press the left mouse button to send that value. To stop and return to the rest of the program, just press the right mouse button.
7. The Running Status test inserts MIDI Clock and Start/Stop/Continue realtime commands in between the Notes. If you get any hanging notes or your drum machine/sequencer does not start, then you will have discovered that your equipment does something wrong with running status!
Version 1.1 (at time of writing)
CHNSCP11 PRG (high resolution mono)
CHNSC11C.PRG (medium resolution)
A useful utility for the quick and easy investigation and display of MIDI Channel information.
The easy to read graphic display shows Notes on a full range MIDI keyboard icon with a range indication showing the highest and lowest note numbers received (very useful for seeing what note numbers a master keyboard will transmit), as well as Velocity, Aftertouch, Release Velocity, Channel Pressure and Pitch Bend indicators - shown as vertical bargraphs. Program Changes and individual MIDI Controllers are shown (decoded, where appropriate) in text form alongside. In addition, the channel number of the displayed values is shown in 16 boxes, which highlight to show the active MIDI channel.
The screen display shows the state of the incoming MIDI messages as a series of snapshots at approximately half-second intervals, and clearly shows most events (although analysing sequencer files or playing fast into it will mean some missed note indications). The response is limited by the speed with which the display can be updated, and thus it is best suited to exploratory tests and investigations.
You can run the medium resolution version of the program in high or medium resolution - the screen display is just half the height in high resolution. The high resolution program will only run in high resolution. Both versions are provided on the one disk to keep the program size as small as possible.
You should put this program at the end of a MIDI network - since it takes time to process the events. Thus there should only be an In to your Atari, and nothing connected to the MIDI Out - but try it if you wish!
Overall, the program is an indispensable tool for the reviewer faced with an unknown MIDI implementation, and a useful utility for any MIDI user.
1st Byte STATUS + Channel Number | 2nd Byte DATA | 3rd Byte DATA | ||||
---|---|---|---|---|---|---|
HEX | BINARY | DECIMAL | BIN/HEX/DEC | BINARY | ||
CHANNEL VOICE MESSAGES | ||||||
8n | 1000 nnnn | 128+n | Note Off | 0kkk kkkk Note Number (Middle C=60) | 0vvv vvvv Release Velocity (default=64) | |
9n | 1001 nnnn | 144+n | Note On | 0kkk kkkk Note Number | 0vvv vvvv Velocity (off=0) | |
An | 1010 nnnn | 160+n | Poly Pressure | 0kkk kkkk Note Number | 0vvv vvvv Value | |
Bn | 1011 nnnn | 176+n | Control Change | 00-31 Continuous Controller MSB | 0vvv vvvv Value 14 bits | |
32-63 Continuous Controller LSB | 0vvv vvvv Value | |||||
64-95 Continuous Controller | 0vvv vvvv Value 7 bits | |||||
Parameters | 96 Data Increment | |||||
97 Data Increment | ||||||
06 Data Entry MSB | 0vvv vvvv Value 14 bits | |||||
38 Data Entry LSB | 0vvv vvvv Value | |||||
98 Non-Registered LSB | 0vvv vvvv Value 14 bits | |||||
99 Non-Registered MSB | 0vvv vvvv Value | |||||
100 Registered LSB | 0vvv vvvv Value 14 bits | |||||
101 Registered MSB | 0vvv vvvv Value | |||||
Controllers | 102-120 (Undefined) | |||||
121 Reset All Controllers | ||||||
Mode Messages | 122 Local Control | 0vvv vvvv Value (0=Off, 127=On) | ||||
123 All Notes Off | 0000 0000 | |||||
124 Omni Off/All Notes Off | 0000 0000 | |||||
125 Omni On/All Notes Off | 0000 0000 | |||||
126 Mono On | 0mmm mmmm (m=channels addressed) | |||||
127 Poly On | ||||||
Cn | 1100 nnnn | 192+n | Program Change | 0ppp pppp Program Number | ||
Dn | 1101 nnnn | 2084n | Channel Pressure | 0vvv vvvv Value | ||
En | 1110 nnnn | 224+n | Pitch Wheel | 0vvv vvvv LSB | 0vvv vvvv MSB | |
SYSTEM MESSAGES | ||||||
F0 | 1111 0000 | 240 | System Exc Start | 0iii iiii Manufacturer ID | ||
F1 | 1111 0001 | 241 | MIDI Time Code | 0nnn dddd Quarter-Frame Messages | ||
F2 | 1111 0010 | 242 | Song Position | 0vvv vvvv LSB | 0vvv vvvv MSB | |
F3 | 1111 0011 | 243 | Song Select | 0sss ssss Song Number | SYSTEM COMMON MESSAGES | |
F4 | 1111 0100 | 244 | (Undefined) | |||
F5 | 1111 0101 | 245 | (Undefined) | |||
F6 | 1111 D110 | 246 | Tune Request | |||
F7 | 1111 0111 | 247 | End Of Exclusive | |||
F8 | 1111 1000 | 248 | Timing Clock | 24 clocks per quarter note | ||
F9 | 1111 1001 | 249 | (Undefined) | SYSTEM REAL-TIME MESSAGES | ||
FA | 1111 1010 | 250 | Start | |||
FB | 1111 1011 | 251 | Continue | |||
FC | 1111 1100 | 252 | Stop | |||
FD | 1111 1101 | 253 | (Undefined) | |||
FE | 1111 1110 | 254 | Active Sensing | Sent at least every 200ms if active | ||
FF | 1111 1111 | 255 | System Reset |
Read the next part in this series:
All parts in this series:
Part 1 (Viewing) | Part 2 | Part 3 | Part 4 | Part 5 | Part 6 | Part 7 | Part 8 | Part 9 | Part 10 | Part 11 | Part 12
![]() Taking Control - Using MIDI Continuous Controllers |
![]() MIDI Evaluated - info available |
Talking MIDI (Part 1) |
MIDI Muting - Sound Workshop |
MIDI Basics - First Steps In Multi-timbrality |
MIDI Theory and Practice |
![]() MIDI Matters - Back To Basics (Part 1) |
Good Enough For The Pro? - Thoughts on MIDI's Next Decade |
Climb Aboard The MIDI Bus (Part 1) |
Technically Speaking |
Adventures In MIDILand (Part 1) |
Practically MIDI (Part 1) |
Browse by Topic:
Topic:
Series:
Part 1 (Viewing) | Part 2 | Part 3 | Part 4 | Part 5 | Part 6 | Part 7 | Part 8 | Part 9 | Part 10 | Part 11 | Part 12
Feature by Martin Russ
Previous article in this issue:
Next article in this issue:
mu:zines is the result of thousands of hours of effort, and will require many thousands more going forward to reach our goals of getting all this content online.
If you value this resource, you can support this project - it really helps!
New issues that have been donated or scanned for us this month.
All donations and support are gratefully appreciated - thank you.
Do you have any of these magazine issues?
If so, and you can donate, lend or scan them to help complete our archive, please get in touch via the Contribute page - thanks!
1st Byte STATUS + Channel Number | 2nd Byte DATA | 3rd Byte DATA | ||||
---|---|---|---|---|---|---|
HEX | BINARY | DECIMAL | BIN/HEX/DEC | BINARY | ||
CHANNEL VOICE MESSAGES | ||||||
8n | 1000 nnnn | 128+n | Note Off | 0kkk kkkk Note Number (Middle C=60) | 0vvv vvvv Release Velocity (default=64) | |
9n | 1001 nnnn | 144+n | Note On | 0kkk kkkk Note Number | 0vvv vvvv Velocity (off=0) | |
An | 1010 nnnn | 160+n | Poly Pressure | 0kkk kkkk Note Number | 0vvv vvvv Value | |
Bn | 1011 nnnn | 176+n | Control Change | 00-31 Continuous Controller MSB | 0vvv vvvv Value 14 bits | |
32-63 Continuous Controller LSB | 0vvv vvvv Value | |||||
64-95 Continuous Controller | 0vvv vvvv Value 7 bits | |||||
Parameters | 96 Data Increment | |||||
97 Data Increment | ||||||
06 Data Entry MSB | 0vvv vvvv Value 14 bits | |||||
38 Data Entry LSB | 0vvv vvvv Value | |||||
98 Non-Registered LSB | 0vvv vvvv Value 14 bits | |||||
99 Non-Registered MSB | 0vvv vvvv Value | |||||
100 Registered LSB | 0vvv vvvv Value 14 bits | |||||
101 Registered MSB | 0vvv vvvv Value | |||||
Controllers | 102-120 (Undefined) | |||||
121 Reset All Controllers | ||||||
Mode Messages | 122 Local Control | 0vvv vvvv Value (0=Off, 127=On) | ||||
123 All Notes Off | 0000 0000 | |||||
124 Omni Off/All Notes Off | 0000 0000 | |||||
125 Omni On/All Notes Off | 0000 0000 | |||||
126 Mono On | 0mmm mmmm (m=channels addressed) | |||||
127 Poly On | ||||||
Cn | 1100 nnnn | 192+n | Program Change | 0ppp pppp Program Number | ||
Dn | 1101 nnnn | 2084n | Channel Pressure | 0vvv vvvv Value | ||
En | 1110 nnnn | 224+n | Pitch Wheel | 0vvv vvvv LSB | 0vvv vvvv MSB | |
SYSTEM MESSAGES | ||||||
F0 | 1111 0000 | 240 | System Exc Start | 0iii iiii Manufacturer ID | ||
F1 | 1111 0001 | 241 | MIDI Time Code | 0nnn dddd Quarter-Frame Messages | ||
F2 | 1111 0010 | 242 | Song Position | 0vvv vvvv LSB | 0vvv vvvv MSB | |
F3 | 1111 0011 | 243 | Song Select | 0sss ssss Song Number | SYSTEM COMMON MESSAGES | |
F4 | 1111 0100 | 244 | (Undefined) | |||
F5 | 1111 0101 | 245 | (Undefined) | |||
F6 | 1111 D110 | 246 | Tune Request | |||
F7 | 1111 0111 | 247 | End Of Exclusive | |||
F8 | 1111 1000 | 248 | Timing Clock | 24 clocks per quarter note | ||
F9 | 1111 1001 | 249 | (Undefined) | SYSTEM REAL-TIME MESSAGES | ||
FA | 1111 1010 | 250 | Start | |||
FB | 1111 1011 | 251 | Continue | |||
FC | 1111 1100 | 252 | Stop | |||
FD | 1111 1101 | 253 | (Undefined) | |||
FE | 1111 1110 | 254 | Active Sensing | Sent at least every 200ms if active | ||
FF | 1111 1111 | 255 | System Reset |
Read the next part in this series:
All parts in this series:
Part 1 (Viewing) | Part 2 | Part 3 | Part 4 | Part 5 | Part 6 | Part 7 | Part 8 | Part 9 | Part 10 | Part 11 | Part 12
![]() Taking Control - Using MIDI Continuous Controllers |
![]() MIDI Evaluated - info available |
Talking MIDI (Part 1) |
MIDI Muting - Sound Workshop |
MIDI Basics - First Steps In Multi-timbrality |
MIDI Theory and Practice |
![]() MIDI Matters - Back To Basics (Part 1) |
Good Enough For The Pro? - Thoughts on MIDI's Next Decade |
Climb Aboard The MIDI Bus (Part 1) |
Technically Speaking |
Adventures In MIDILand (Part 1) |
Practically MIDI (Part 1) |
Browse by Topic: