System Exclusive: powerful data dumping and loading system or industry communications fiasco? What can be achieved with and what are the limitations of SysEx? Text by Vic Lennard.

CIRCA 1975, MOST synths were monophonic and could only be patched together by jack to jack cables carrying pitch information - called CV, or control voltage, which tended to be a one-volt change for a pitch shift of one octave - and note on/off - or gate - information. When polyphonic synths arrived on the scene the problem became even worse... In 1981 the Universal Synthesiser Interface appeared, courtesy of Sequential Circuits. This was incorporated on the Prophet 10, and replaced CV and Gate connections with a digital serial interface. By the end of '82 the US and Japanese manufacturers had collaborated to set up a standard called MIDI - the Musical Instrument Digital Interface, which most of us now take for granted.

Conventions

MIDI SPEC 1.0 was quite specific in most areas. The essential hardware was outlined, and all information was to consist of ten bits - a start bit, eight data bits and a stop bit. Travelling at a speed of 31.25 Kbits per sec, these ten bits take 320 microseconds to transmit.

These bits are collectively called a byte and the highest value bit (128 or 80H) is referred to as the Most Significant Bit (MSB). Two different types of byte were specified: Status, which identifies the kind of message and has the MSB set to 1, and Data, which carries the content of the message and has the MSB reset to 0. Consequently, all MIDI messages consist of a Status byte followed by 0, 1 or 2 data bytes unless they are related to System Exclusive (SysEx) messages. The specification is equally rigid on Channel and System messages, but with respect to System Exclusive, the original document, even after revision, called for the following basic format:

Status byte:	F0	:SysEx Start
Data bytes:	0-7F	:Manufacturer's ID
	0-7F	:Data
	.
	.
	0-7F
	F7	:End of Exclusive (EOX)

The identification codes for manufacturers were set up and include 41H (Roland), 43H (Yamaha), 44H (Casio) 47H (Akai) and so on. The basic format still gives carte blanche to manufacturers to continue on their merry way and leave the non-technical musician completely lost and dependant on RAM cartridges/cards for data storage. These usually cost, and leave the technical musician working his way through the MIDI implementation pages at the back of the synth manual - if such pages exist and if they happen to be correct.

Practical SysEx

System exclusive messages allow you to transmit data to another like synth or to a third party machine such as a computer, and to receive data from a similar range of devices. Data may be in the form of individual parameter changes such as memory protect on/off, but tends to be that of voice data - single voice or bulk.

Due to these vague outlines, many different "standards" exist. In fact there are practically as many of these as manufacturers. Some synths can initiate bulk dump by pressing buttons on the front panel (Roland D50, Yamaha DX/TX7, FB01) while others have to be asked to supply their data by the receiving system. Some synths will allow SysEx information to merrily sail past the memory protection (Casio CZ101, which uses a hardware memory protect), and others will only permit data to be sent to a cartridge (Roland JX10). Then there is the point about whether data is sent in a continuous stream or broken down into blocks, and whether the entire internal memory is sent or split up into areas.

Another bone of contention is that of handshaking. In handshaking mode, acknowledgement commands are sent back every time a block of data is received. This means that an arithmetic code at the end of the data block (called the checksum) has been checked and that the data in the block is correct. MIDI In-to-Out and Out-to-In cables have to be used, which invariably necessitates the use of a MIDI junction box and a tangle of leads. In no-handshake, or one-way mode, data is transferred without acknowledgement of receipt allowing MIDI connections to be simpler but also permitting mistakes in the data to go unnoticed.

Let's consider an example of a current MIDI implementation: the Roland D110, multitimbral LA synthesis module. It uses the following format for data transfer:

FO	:SysEx
41	:Roland ID
dd	:Device ID
16	:Model ID for the D110
cc	:Command ID.
data	:Data in blocks of 256 bytes.
.
.
data
F7	:EOX

The device ID allows for more than one D110 to be used and can be set on the relevant modules. Out of interest, the format is the same for the MT32. The command ID's are as follows:

CODE	ID	NATURE
	(HEX)
RQ1	11	Request data 1 way
DT1	12	Data set 1 way
WSD	40	Want to send data
RQD	41	Request data
DAT	42	Data set
ACK	43	Acknowledgement
EOD	45	End of data
ERR	4E	Communication error
RJC	4F	Rejection

A typical situation might be the need to save the voice which has just been edited, such as the one in the Tone temporary area, which, for instance, is the 3rd part of the 8 that the D110 can simultaneously play. The following conversation takes place, in handshaking mode, between the D110 and a computer.

1.	Computer	FO 41 00 16 41 04 02 EC 00 01 76 97 F7
2.	D110	FO 41 00 16 42 04 02 EC dd...dd cs F7
3.	Computer	FO 41 00 16 43 F7
4.	D110	FO 41 00 16 42 04 03 EC dd...dd cs F7
5.	Computer	FO 41 00 16 43 F7
6.	D110	FO 41 00 16 45 F7
7.	Computer	FO 41 00 16 43 F7

Line 1 requests data, from the D110 with device ID 00, starting at address 0402EC and of length 000176 bytes (Hex). The hexadecimal checksum is 97 which, when added to the 6 previous bytes, provides for the least significant 7 bits to sum to zero. The D110 sends across the first 256 bytes and the computer acknowledges them, although it may not have actually checked the checksum. The D110 then sends across the remaining 118 bytes to a start address 256 bytes further on than before. "Thanks Y'all", says the computer, "That's all folks", signs off the D110, and a final "Be seein' ya" comes from the computer.

The addresses to which data can be sent, and the data lengths are obtained from the D110 manual. Even so, I can understand if you are totally perplexed, or at least confused. You're not alone. The main problem arises from the wording of MIDI specification 1.0: "Only the basic shell (for SysEx messages) is defined. Specific formats are designed and published by the manufacturer". Great. So having sorted out a way to transfer specific data from a D110, different problems will arise with each different synth. Also there are a variety of differing procedures within any particular synth. The D110 can dump and load the temporary areas for Timbres, Tones and the Rhythm setup, the memory area for the 128 Timbres, 64 Patches and 64 user Tones as well as the system area which houses the reverb characteristics and the partial reserve for each part.

Computers and Software

First of all the bad news. Unless you have a computer such as an Atari ST, your ability to dump and load data will be limited. For instance, a Roland MC500 will be able to receive and send data as long as it is initiated from the front panel of the relevant synth in no-handshaking mode - sequencers of this ilk cannot send out request and acknowledgement commands.

As for the Atari ST, SysEx handling is starting to appear within the package of the sequencer. C-Lab's Creator will allow you to record data and edit it but, as with the MC500 problem, this is limited in scope. Steinberg's Pro24 III now has a single dump command line with 17 bytes available incorporated in the Dump Utility option, but very little can be realistically achieved with so few bytes - certainly nothing as involved as editing. Dr Ts have a piece of software called Injector which sets itself up as a desktop accessory - an oddity when you consider that their sequencers are completely non-GEM orientated. Again, it only has the ability to work with ten preset synths and although others will be included in updates, it is hardly a creative SysEx tool.

This, then, brings me to the only professional software system exclusive editor on the market - Hybrid Arts' GenPatch (which I reviewed in MT, April '88).

GenPatch has 11 commands and will not undertake certain tasks, for instance checking checksums - it assumes that they are correct and treats them as bytes of data. The above D110 dump would be treated in the following manner:

1.	Send sentence	F0 41 00 16 41 04 02 EC 00 01 76 97 F7
2.	Loop 2 times
3.	Receive sentence	F0 41 00 16 42
4.	Receive data	261 bytes (includes address, checksum and F7)
5.	Send sentence	F0 41 00 16 43 F7
6.	End of loop
7.	Receive sentence	F0 41 00 16 45 F7
8.	Send sentence	F0 41 00 16 43 F7

As before, the address and data sizes for line one are obtained from the manual and everything else is far easier. GenPatch stores the data in the correct order and writes a header to send it back to the correct place when needed. Configurations are available for practically any synth and updates to these cost the price of a disk and postage.

Samplers

WITH THE ADVENT of the relatively cheap 12-bit sampler came the need for some degree of uniformity with regard to the dumping of sounds. Sequential initiated a format in their Prophet 2000 which was subsequently copied by Akai in their S900 - in fact, samples can be transferred directly between these two machines without the need for computer intervention.

The MIDI Sample Dump Standard was adopted in January 1986 after a series of proposals had been submitted to the Japanese MIDI Standards Committee (JMSC) and the MIDI Manufacturers Association (MMA) in the USA. The basic idea is similar to that of a handshaking in a synthesiser - a request is sent, a header returns carrying relevant data, namely the bit format sample period, sample lengths and sustain loop points. The waveform then follows interspersed by acknowledgements. In greater detail this would read:

REQUEST
FO	:SysEx
7E	:ID, Universal non-real time
cc	:MIDI Channel
03	:Sub ID, Request
ss ss	:Sample No.
F7	:EOX

HEADER
f0	:Sysex
7E	:ID:Universal non-real time
cc	:Midi Channel
01	:Sub ID:Header
ss ss	:Sample No.
ee	:Format: No.of bits
ff ff ff	:Sample Period = 1/sample frequency
gg gg gg	:Sample Length
hh hh hh	:Sustain Loop Start
ii ii ii	:Sustain Loop End
jj	:Loop Type
F7	:EOX

DATA PACKET
FO	:SysEx
7E	:ID: Universal non-real time
cc	:MIDI Channel
02	:Sub ID: Data Packet
kk	:Packet No.
dd..dd	:120 bytes of data
cs	:Checksum
F7	:EOX

Should any errors occur such as the checksums not tallying then a NAK (negative acknowledgement) would be sent, requesting that the last packet of data be re-sent.

Note that no programming details can be transmitted from one sampler to a different model using the Sample Dump Standard. There is a piece of software called GenWave/12 from Drumware (reviewed in MT, July '88) which allows interaction between all 12-bit samplers - check it out.

Conceivably, it should be possible to take a sample from an 8-bit sampler such as a Mirage or Emulator II, write a new header onto it and transmit it to a 12-bit sampler. Oh, and as far as I know, Fairlight Series IIIs are very secretive and do not have Sample Dump implemented. Finally slapped wrists time for Casio. Having signed an agreement to adhere to the Sample Dump Standard, out came the FZ1 which adheres to... Absolutely nothing. Who said the spirit of rock 'n' roll was dead? Instead there is a 118-page book entitled "Casio Digital Sampling Keyboard FZ1 Data Structures" for anyone who fancies a challenge.

The Bottom Line

You've probably been thinking it; I'll say it - what a minefield. Although there are only two basic techniques of data transferral - namely handshaking and oneway - so many sub-techniques exist as to make life very awkward indeed. Certainly, those of you with dedicated sequencers up to and including the Yamaha QX3 and Roland MC500 are going to find this process nigh-on impossible unless you can dump data one-way from the front panel of your synth. The list of possibilities includes: Yamaha: DX7, TX7, DX21, DX27, DX100, FB01, TX81Z (single voice only) and TX802. Roland: D50, D550, D10, D20, D110.

Most other synths require handshaking. Check your user's manuals, we keep telling you they're useful, don't we? With a large synth setup it's worth investing in an Atari ST and relevant software just to dump and load data, given the cost of data saving media.

Bibliography:
MIDI System Exclusive Book
MIDI Resource Book
MIDI Implementation Book All by Steve De Furia and Joe Scacciaserro.

Published in the UK by International Music Publishing.