System Exclusive Dumps
The mysteries of SysEx are revealed. Martin Russ discovers that there may be more to that disk drive in your latest acquisition than you thought...
Have you noticed that a strange disease seems to be afflicting the latest batch of keyboards and sequencers to be released? Many of those fitted with disk drives have a new feature: they can act as MIDI Data Recorders (MDRs). This new 'feature' seems to be most noticeable in those items with a 3½" disk drive, but the symptoms can also be observed in other sizes, as well as in computers! Always on the look-out for the unusual and interesting, I collected together some of the afflicted items and placed them and myself in isolation for as long as we could stand each other!
At first nothing happened. I stared at the keyboards and they gave me their toothy grins back. The cursor blinked on the computer display. Time passed... As my mind wandered I began to review what I knew about System Exclusive Dumps. I knew that they involved instrument-specific information, usually things like bulk dumps of the internal sounds, or sequencer files...
To discover what the possibilities of using System Exclusive data could be, you need to know what sort of things the equipment manufacturers use it for. There are at least four major types of information that I could identify. (The following list starts with the obvious and moves to the obscure. There may be other uses which I have not considered - please let me know! I have also included a few thoughts about what you could do with some of the examples.)
These are perhaps the first thing most people think of when you mention MIDI Data Recorders. There are usually two types of information dump specific to sounds - single and bulk. Single dumps are usually of short duration and contain the information for just one sound. Bulk dumps usually contain the information for all of the internal memory of the instrument.
For instruments with only a small number of internal memory locations available for storing your own sounds, the ability to use a MDR could be invaluable. There is no need to buy multiple RAM cartridges if you can exploit MDRs in your set-up, and so you could save a considerable amount of money. MDRs also make maintaining a library of sounds much easier, especially with a computer-based MDR, where you could integrate the disk catalogues into a database.
Most instruments enable you to save a small number of complete set-ups for performance, but there rarely seem to be enough memories for this. MDRs enable you to use all the available memory of your instrument for just a single song, loading in the next set of memories for the next song. This leaves your instrument wide open to a thorough exploitation of its performance capabilities, because you do not have to worry about keeping a 'safe' set of normal playable memories always in the machine.
Most drum machines have limited memory capacity and often only provide external RAM cartridge or cassette/tape storage of additional data. Rumour has it that the Human League used to split their live sets into sections according to when they needed to reload their Roland MC4 sequencers! Nowadays, MIDI Data Recorders (MDRs) enable an entire set to be stored on a single floppy disk - much faster and infinitely better than a cassette. The random access you get with a floppy disk means that you are in control of the running order, whereas with a cassette you usually find it easiest and safest to load the files in sequential order.
The cheaper sequencers without disk storage usually provide cassette storage (tape dump) as their only means of saving sequences. MDRs mean that you can store all of the sequence data on disk providing at least one of your instruments has a disk drive. When you can save musical sequences reliably and quickly, your attitude to using sequences may change! You can then use the sequencer much more as a musical notepad - jotting down ideas for riffs, chord progressions, syncopations etc, as you find them.
Most new sampling instruments support the Sample Dump Standard (SDS), which enables you to at least attempt to transfer a sample between two different samplers. It uses handshaking protocols to transfer packets of data between the two machines with a good assurance of receiving correct data. (There are also some remarkably inventive non-standard formats as well, but these are manufacturer specific.)
Using the SDS is likely to be a slower process than using the internal floppy drive in your sampler (and much slower than an internal hard disk) because of the limited speed of MIDI - even when 'turbocharged'! So what would you use MIDI Data Recorders with samples for? Perhaps for archival purposes - storing the same sample file in two different places is also a good way of guarding against disaster (what happens to all your hard work if you have a fire, flood, or a tree falls through your ceiling in a high wind?). MDRs are also useful for getting sample data into computers and then transferring the sound samples to other people/studios using phone lines and modems.
Most samplers have an additional option of a hard disk drive. This is just a larger, rigid version of a floppy disk, sealed permanently inside the drive unit, but offering 20 or more times the data storage capacity of a single floppy disk. With large amounts of data capacity like this available, you can store whole libraries of sound and sequencer dumps for any other equipment you have, and still leave room for the much larger sample files.
These are lists of incoming program change numbers and corresponding internal patch changes. They enable you to set the incoming program change messages to call up the patch changes you want. So, instead of program change 1 always selecting memory 1 on your synth, you could re-map it to memory 14. This removes the need to store your sounds in the memories in a predetermined order. Most instruments have only very limited memory for storing this sort of table, so if you can store the maps in a MDR then you can be rather more enthusiastic about setting them up in the first place.
Since all the above are just stored as data, it is perfectly feasible to incorporate several together one after the other. These composite files can make coping with the many different sets of MIDI data a lot easier. The System Exclusive messages include an ID byte in the data which is specific to each manufacturer, and so you can send all the MIDI data to every device on your MIDI network and only those pieces of equipment which recognise the correct ID byte will listen in, other manufacturers' products will ignore that data. Thus, in a pause between songs, say, you could reload your sequencer and change the sounds in your sampler and synths, just by sending a single composite MIDI data file through the MIDI network. This would not only make things easier live, but such a dump could be assembled in the studio whilst recording, making it easier to recall the exact instrumentation, rhythms, samples etc for a particular song.
You may be wondering at this point about how to network your MIDI equipment - not to worry, you have probably got the majority of it set up already. Assuming that you have a 'star network' (see SOS Sept 86) where all your MIDI equipment is controlled from a master keyboard or a sequencer/computer via a MIDI Thru box, then all you need to do is connect the Out of the MIDI Data Recorder to the Thru box's Input selector, and have another selector to choose which piece of equipment you want to receive MIDI data from. Figure 1 makes this a lot clearer.
I mentioned above that you could send sound samples using modems and phone lines. In fact, you could send any MIDI data using exactly the same equipment. Sequences used by musicians working on albums have been transferred between 'home' and 'professional' studios using the Performing Arts Network, PAN (see SOS Nov 87). It would even be possible to send the contents of a disk to all the venues of a tour and do a whole series of performances without ever leaving home, but I suspect that someone has probably already beaten me to this idea!!
Storing MIDI data can be accomplished on most instruments fitted with a floppy disk drive or, alternatively, on a computer like the well supported Atari ST. There are advantages to using computers for storing MIDI data, so let's look at what they offer and then contrast this with the typical capabilities of an instrument-based disk drive with MDR features.
Programs which configure a computer to store and retrieve MIDI data are usually called Generic Patch Librarians - the Hybrid Arts GENPATCH ST for the Atari ST is a typical example of the style; it enables you to create requests as well as just acting dumb and recording data dumps. Handshaking is also catered for. This means that you can send dumps to and fro between the computer and the instruments without needing to press any front panel buttons (assuming that the instruments support Dump Requests).
Instrument-based disk drives which can store data dumps tend to be less sophisticated, especially those found in keyboard instruments. Often no control is provided to allow you to write your own request/response handshake routines... This means that you can only 'talk' to instruments on which you can initiate a data dump - the original DX7,for instance, has no 'officially documented' dump request, but it is possible to simulate the front panel button presses by sending MIDI commands. Most modern equipment allows you to initiate some of the possible dumps, but these possibilities are extended by using dump requests via MIDI. Non-computer based data filing systems tend to have advantages in live use - there's no computer on stage to go wrong! It seems that most professional musical equipment is designed to still perform correctly with noisy mains and lots of interference, whereas many personal computers seem to be very susceptible to crashing at the wrong moment.
Computer-based MDR systems also tend to allow you to view and edit any MIDI data, often providing some means of selectively filtering the data you see. This means that you have more flexibility over what you do with the data. Although the latest generation sequencers based on 'clean computer' designs are rapidly catching up. In general, the computer-based systems seem to provide the most facilities, followed by some of the better sequencers. Samplers and keyboard systems seem to have the least facilities. There are also some dedicated MIDI Data Filers, designed specifically for saving System Exclusive data, although these seem to suffer from a 'minimum buttons and display' design philosophy.
MIDI Data Recorders are versatile, effective and useful - every MIDI musician should seriously think about ways to exploit their power. The possibilities for using them are wide and cover both live performance and studio work, as well as the home studio. As the amount of MIDI-based equipment you use increases, you will find more and more uses for MDRs, and their use will make life a lot easier.
In the next few years, as the integration and exploitation of MIDI becomes more important, more and more pieces of equipment will be fitted with MDR facilities. I would not be surprised to see the emergence of comprehensive hard disk-based MIDI network management systems - overgrown versions of what we are starting to see appear now as MDRs. If you start using MIDI System Exclusive Dumps now, then both you and your MIDI network will be prepared and familiar with the terminology for when the big systems arrive.
Feature by Martin Russ
mu:zines is the result of thousands of hours of effort, and will require many thousands more going forward to reach our goals of getting all this content online.
If you value this resource, you can support this project - it really helps!