System Exclusive messages promised to make MIDI a flexible and powerful communications standard; instead it seems to have caused MIDI havoc. Vic Lennard takes an Exclusive look.
System Exclusive: powerful data dumping and loading system or industry communications fiasco? What can be achieved with and what are the limitations of SysEx? Text by Vic Lennard.
CIRCA 1975, MOST synths were monophonic and could only be patched together by jack to jack cables carrying pitch information - called CV, or control voltage, which tended to be a one-volt change for a pitch shift of one octave - and note on/off - or gate - information. When polyphonic synths arrived on the scene the problem became even worse... In 1981 the Universal Synthesiser Interface appeared, courtesy of Sequential Circuits. This was incorporated on the Prophet 10, and replaced CV and Gate connections with a digital serial interface. By the end of '82 the US and Japanese manufacturers had collaborated to set up a standard called MIDI - the Musical Instrument Digital Interface, which most of us now take for granted.
MIDI SPEC 1.0 was quite specific in most areas. The essential hardware was outlined, and all information was to consist of ten bits - a start bit, eight data bits and a stop bit. Travelling at a speed of 31.25 Kbits per sec, these ten bits take 320 microseconds to transmit.
These bits are collectively called a byte and the highest value bit (128 or 80H) is referred to as the Most Significant Bit (MSB). Two different types of byte were specified: Status, which identifies the kind of message and has the MSB set to 1, and Data, which carries the content of the message and has the MSB reset to 0. Consequently, all MIDI messages consist of a Status byte followed by 0, 1 or 2 data bytes unless they are related to System Exclusive (SysEx) messages. The specification is equally rigid on Channel and System messages, but with respect to System Exclusive, the original document, even after revision, called for the following basic format:
|Status byte:||F0||:SysEx Start|
|Data bytes:||0-7F||:Manufacturer's ID|
|F7||:End of Exclusive (EOX)|
System exclusive messages allow you to transmit data to another like synth or to a third party machine such as a computer, and to receive data from a similar range of devices. Data may be in the form of individual parameter changes such as memory protect on/off, but tends to be that of voice data - single voice or bulk.
Due to these vague outlines, many different "standards" exist. In fact there are practically as many of these as manufacturers. Some synths can initiate bulk dump by pressing buttons on the front panel (Roland D50, Yamaha DX/TX7, FB01) while others have to be asked to supply their data by the receiving system. Some synths will allow SysEx information to merrily sail past the memory protection (Casio CZ101, which uses a hardware memory protect), and others will only permit data to be sent to a cartridge (Roland JX10). Then there is the point about whether data is sent in a continuous stream or broken down into blocks, and whether the entire internal memory is sent or split up into areas.
Another bone of contention is that of handshaking. In handshaking mode, acknowledgement commands are sent back every time a block of data is received. This means that an arithmetic code at the end of the data block (called the checksum) has been checked and that the data in the block is correct. MIDI In-to-Out and Out-to-In cables have to be used, which invariably necessitates the use of a MIDI junction box and a tangle of leads. In no-handshake, or one-way mode, data is transferred without acknowledgement of receipt allowing MIDI connections to be simpler but also permitting mistakes in the data to go unnoticed.
Let's consider an example of a current MIDI implementation: the Roland D110, multitimbral LA synthesis module. It uses the following format for data transfer:
|16||:Model ID for the D110|
|data||:Data in blocks of 256 bytes.|
|RQ1||11||Request data 1 way|
|DT1||12||Data set 1 way|
|WSD||40||Want to send data|
|EOD||45||End of data|
|1.||Computer||FO 41 00 16 41 04 02 EC 00 01 76 97 F7|
|2.||D110||FO 41 00 16 42 04 02 EC dd...dd cs F7|
|3.||Computer||FO 41 00 16 43 F7|
|4.||D110||FO 41 00 16 42 04 03 EC dd...dd cs F7|
|5.||Computer||FO 41 00 16 43 F7|
|6.||D110||FO 41 00 16 45 F7|
|7.||Computer||FO 41 00 16 43 F7|
First of all the bad news. Unless you have a computer such as an Atari ST, your ability to dump and load data will be limited. For instance, a Roland MC500 will be able to receive and send data as long as it is initiated from the front panel of the relevant synth in no-handshaking mode - sequencers of this ilk cannot send out request and acknowledgement commands.
As for the Atari ST, SysEx handling is starting to appear within the package of the sequencer. C-Lab's Creator will allow you to record data and edit it but, as with the MC500 problem, this is limited in scope. Steinberg's Pro24 III now has a single dump command line with 17 bytes available incorporated in the Dump Utility option, but very little can be realistically achieved with so few bytes - certainly nothing as involved as editing. Dr Ts have a piece of software called Injector which sets itself up as a desktop accessory - an oddity when you consider that their sequencers are completely non-GEM orientated. Again, it only has the ability to work with ten preset synths and although others will be included in updates, it is hardly a creative SysEx tool.
This, then, brings me to the only professional software system exclusive editor on the market - Hybrid Arts' GenPatch (which I reviewed in MT, April '88).
GenPatch has 11 commands and will not undertake certain tasks, for instance checking checksums - it assumes that they are correct and treats them as bytes of data. The above D110 dump would be treated in the following manner:
|1.||Send sentence||F0 41 00 16 41 04 02 EC 00 01 76 97 F7|
|2.||Loop 2 times|
|3.||Receive sentence||F0 41 00 16 42|
|4.||Receive data||261 bytes (includes address, checksum and F7)|
|5.||Send sentence||F0 41 00 16 43 F7|
|6.||End of loop|
|7.||Receive sentence||F0 41 00 16 45 F7|
|8.||Send sentence||F0 41 00 16 43 F7|
WITH THE ADVENT of the relatively cheap 12-bit sampler came the need for some degree of uniformity with regard to the dumping of sounds. Sequential initiated a format in their Prophet 2000 which was subsequently copied by Akai in their S900 - in fact, samples can be transferred directly between these two machines without the need for computer intervention.
The MIDI Sample Dump Standard was adopted in January 1986 after a series of proposals had been submitted to the Japanese MIDI Standards Committee (JMSC) and the MIDI Manufacturers Association (MMA) in the USA. The basic idea is similar to that of a handshaking in a synthesiser - a request is sent, a header returns carrying relevant data, namely the bit format sample period, sample lengths and sustain loop points. The waveform then follows interspersed by acknowledgements. In greater detail this would read:
|7E||:ID, Universal non-real time|
|03||:Sub ID, Request|
|ss ss||:Sample No.|
|7E||:ID:Universal non-real time|
|ss ss||:Sample No.|
|ee||:Format: No.of bits|
|ff ff ff||:Sample Period = 1/sample frequency|
|gg gg gg||:Sample Length|
|hh hh hh||:Sustain Loop Start|
|ii ii ii||:Sustain Loop End|
|7E||:ID: Universal non-real time|
|02||:Sub ID: Data Packet|
|dd..dd||:120 bytes of data|
You've probably been thinking it; I'll say it - what a minefield. Although there are only two basic techniques of data transferral - namely handshaking and oneway - so many sub-techniques exist as to make life very awkward indeed. Certainly, those of you with dedicated sequencers up to and including the Yamaha QX3 and Roland MC500 are going to find this process nigh-on impossible unless you can dump data one-way from the front panel of your synth. The list of possibilities includes: Yamaha: DX7, TX7, DX21, DX27, DX100, FB01, TX81Z (single voice only) and TX802. Roland: D50, D550, D10, D20, D110.
Most other synths require handshaking. Check your user's manuals, we keep telling you they're useful, don't we? With a large synth setup it's worth investing in an Atari ST and relevant software just to dump and load data, given the cost of data saving media.
MIDI System Exclusive Book
MIDI Resource Book
MIDI Implementation Book All by Steve De Furia and Joe Scacciaserro.
Published in the UK by International Music Publishing.
Feature by Vic Lennard
mu:zines is the result of thousands of hours of effort, and will require many thousands more going forward to reach our goals of getting all this content online.
If you value this resource, you can support this project - it really helps!
Please note: Our yearly hosting fees are due every March, so monetary donations are especially appreciated to help meet this cost. Thank you for your support!