Magazine Archive

Home -> Magazines -> Issues -> Articles in this issue -> View

System Exclusive Dumps

The mysteries of SysEx are revealed. Martin Russ discovers that there may be more to that disk drive in your latest acquisition than you thought...



Have you noticed that a strange disease seems to be afflicting the latest batch of keyboards and sequencers to be released? Many of those fitted with disk drives have a new feature: they can act as MIDI Data Recorders (MDRs). This new 'feature' seems to be most noticeable in those items with a 3½" disk drive, but the symptoms can also be observed in other sizes, as well as in computers! Always on the look-out for the unusual and interesting, I collected together some of the afflicted items and placed them and myself in isolation for as long as we could stand each other!

At first nothing happened. I stared at the keyboards and they gave me their toothy grins back. The cursor blinked on the computer display. Time passed... As my mind wandered I began to review what I knew about System Exclusive Dumps. I knew that they involved instrument-specific information, usually things like bulk dumps of the internal sounds, or sequencer files...

THE POSSIBILITIES



To discover what the possibilities of using System Exclusive data could be, you need to know what sort of things the equipment manufacturers use it for. There are at least four major types of information that I could identify. (The following list starts with the obvious and moves to the obscure. There may be other uses which I have not considered - please let me know! I have also included a few thoughts about what you could do with some of the examples.)

Voice/Patch/Sound Specification Dumps.

These are perhaps the first thing most people think of when you mention MIDI Data Recorders. There are usually two types of information dump specific to sounds - single and bulk. Single dumps are usually of short duration and contain the information for just one sound. Bulk dumps usually contain the information for all of the internal memory of the instrument.

For instruments with only a small number of internal memory locations available for storing your own sounds, the ability to use a MDR could be invaluable. There is no need to buy multiple RAM cartridges if you can exploit MDRs in your set-up, and so you could save a considerable amount of money. MDRs also make maintaining a library of sounds much easier, especially with a computer-based MDR, where you could integrate the disk catalogues into a database.

Performance/Set-up/Configuration Dumps.

Most instruments enable you to save a small number of complete set-ups for performance, but there rarely seem to be enough memories for this. MDRs enable you to use all the available memory of your instrument for just a single song, loading in the next set of memories for the next song. This leaves your instrument wide open to a thorough exploitation of its performance capabilities, because you do not have to worry about keeping a 'safe' set of normal playable memories always in the machine.

Rhythm Patterns.

Most drum machines have limited memory capacity and often only provide external RAM cartridge or cassette/tape storage of additional data. Rumour has it that the Human League used to split their live sets into sections according to when they needed to reload their Roland MC4 sequencers! Nowadays, MIDI Data Recorders (MDRs) enable an entire set to be stored on a single floppy disk - much faster and infinitely better than a cassette. The random access you get with a floppy disk means that you are in control of the running order, whereas with a cassette you usually find it easiest and safest to load the files in sequential order.

Sequence Files.

The cheaper sequencers without disk storage usually provide cassette storage (tape dump) as their only means of saving sequences. MDRs mean that you can store all of the sequence data on disk providing at least one of your instruments has a disk drive. When you can save musical sequences reliably and quickly, your attitude to using sequences may change! You can then use the sequencer much more as a musical notepad - jotting down ideas for riffs, chord progressions, syncopations etc, as you find them.

Samples.

Most new sampling instruments support the Sample Dump Standard (SDS), which enables you to at least attempt to transfer a sample between two different samplers. It uses handshaking protocols to transfer packets of data between the two machines with a good assurance of receiving correct data. (There are also some remarkably inventive non-standard formats as well, but these are manufacturer specific.)

Using the SDS is likely to be a slower process than using the internal floppy drive in your sampler (and much slower than an internal hard disk) because of the limited speed of MIDI - even when 'turbocharged'! So what would you use MIDI Data Recorders with samples for? Perhaps for archival purposes - storing the same sample file in two different places is also a good way of guarding against disaster (what happens to all your hard work if you have a fire, flood, or a tree falls through your ceiling in a high wind?). MDRs are also useful for getting sample data into computers and then transferring the sound samples to other people/studios using phone lines and modems.

Most samplers have an additional option of a hard disk drive. This is just a larger, rigid version of a floppy disk, sealed permanently inside the drive unit, but offering 20 or more times the data storage capacity of a single floppy disk. With large amounts of data capacity like this available, you can store whole libraries of sound and sequencer dumps for any other equipment you have, and still leave room for the much larger sample files.

Mapping Tables.

These are lists of incoming program change numbers and corresponding internal patch changes. They enable you to set the incoming program change messages to call up the patch changes you want. So, instead of program change 1 always selecting memory 1 on your synth, you could re-map it to memory 14. This removes the need to store your sounds in the memories in a predetermined order. Most instruments have only very limited memory for storing this sort of table, so if you can store the maps in a MDR then you can be rather more enthusiastic about setting them up in the first place.

Since all the above are just stored as data, it is perfectly feasible to incorporate several together one after the other. These composite files can make coping with the many different sets of MIDI data a lot easier. The System Exclusive messages include an ID byte in the data which is specific to each manufacturer, and so you can send all the MIDI data to every device on your MIDI network and only those pieces of equipment which recognise the correct ID byte will listen in, other manufacturers' products will ignore that data. Thus, in a pause between songs, say, you could reload your sequencer and change the sounds in your sampler and synths, just by sending a single composite MIDI data file through the MIDI network. This would not only make things easier live, but such a dump could be assembled in the studio whilst recording, making it easier to recall the exact instrumentation, rhythms, samples etc for a particular song.

You may be wondering at this point about how to network your MIDI equipment - not to worry, you have probably got the majority of it set up already. Assuming that you have a 'star network' (see SOS Sept 86) where all your MIDI equipment is controlled from a master keyboard or a sequencer/computer via a MIDI Thru box, then all you need to do is connect the Out of the MIDI Data Recorder to the Thru box's Input selector, and have another selector to choose which piece of equipment you want to receive MIDI data from. Figure 1 makes this a lot clearer.

How to connect a MIDI Data Recorder (MDR) in a MIDI network.


I mentioned above that you could send sound samples using modems and phone lines. In fact, you could send any MIDI data using exactly the same equipment. Sequences used by musicians working on albums have been transferred between 'home' and 'professional' studios using the Performing Arts Network, PAN (see SOS Nov 87). It would even be possible to send the contents of a disk to all the venues of a tour and do a whole series of performances without ever leaving home, but I suspect that someone has probably already beaten me to this idea!!


INSTRUMENT OR COMPUTER?



Storing MIDI data can be accomplished on most instruments fitted with a floppy disk drive or, alternatively, on a computer like the well supported Atari ST. There are advantages to using computers for storing MIDI data, so let's look at what they offer and then contrast this with the typical capabilities of an instrument-based disk drive with MDR features.

Programs which configure a computer to store and retrieve MIDI data are usually called Generic Patch Librarians - the Hybrid Arts GENPATCH ST for the Atari ST is a typical example of the style; it enables you to create requests as well as just acting dumb and recording data dumps. Handshaking is also catered for. This means that you can send dumps to and fro between the computer and the instruments without needing to press any front panel buttons (assuming that the instruments support Dump Requests).

Instrument-based disk drives which can store data dumps tend to be less sophisticated, especially those found in keyboard instruments. Often no control is provided to allow you to write your own request/response handshake routines... This means that you can only 'talk' to instruments on which you can initiate a data dump - the original DX7,for instance, has no 'officially documented' dump request, but it is possible to simulate the front panel button presses by sending MIDI commands. Most modern equipment allows you to initiate some of the possible dumps, but these possibilities are extended by using dump requests via MIDI. Non-computer based data filing systems tend to have advantages in live use - there's no computer on stage to go wrong! It seems that most professional musical equipment is designed to still perform correctly with noisy mains and lots of interference, whereas many personal computers seem to be very susceptible to crashing at the wrong moment.

Computer-based MDR systems also tend to allow you to view and edit any MIDI data, often providing some means of selectively filtering the data you see. This means that you have more flexibility over what you do with the data. Although the latest generation sequencers based on 'clean computer' designs are rapidly catching up. In general, the computer-based systems seem to provide the most facilities, followed by some of the better sequencers. Samplers and keyboard systems seem to have the least facilities. There are also some dedicated MIDI Data Filers, designed specifically for saving System Exclusive data, although these seem to suffer from a 'minimum buttons and display' design philosophy.

CONCLUSION



MIDI Data Recorders are versatile, effective and useful - every MIDI musician should seriously think about ways to exploit their power. The possibilities for using them are wide and cover both live performance and studio work, as well as the home studio. As the amount of MIDI-based equipment you use increases, you will find more and more uses for MDRs, and their use will make life a lot easier.

In the next few years, as the integration and exploitation of MIDI becomes more important, more and more pieces of equipment will be fitted with MDR facilities. I would not be surprised to see the emergence of comprehensive hard disk-based MIDI network management systems - overgrown versions of what we are starting to see appear now as MDRs. If you start using MIDI System Exclusive Dumps now, then both you and your MIDI network will be prepared and familiar with the terminology for when the big systems arrive.

FURTHER APPLICATIONS

The examples that follow do not necessarily use just System Exclusive data, but since most of the MDR systems under consideration will let us save any MIDI data, then we can use the facility for our own purposes. For example:

EDITS - store edit commands so that you can create performance and/or configuration memories for an instrument which doesn't have one! Store a programmer's 'live' edits so that you can study them later. Edit a voice in between notes. Edit drum patterns/instrument setups between songs.

TIMING CLOCKS - apparently, with a large enough file (probably on hard disk) it is possible to record the timing clocks and patch changes for short sections of a performance (solos, for example) and replay them on demand. The advantage this has over a sequencer is that you can capture the whole of the original performance in real time, whereas most sequencers prefer to record on one MIDI channel, or at most six channels (for guitar synths etc). I have even heard of people recording MIDI Time Code and then editing it, storing it and replaying it to change playback tempos etc.


Disk or RAM?

There are two main methods of storing the MIDI data inside a MIDI Data Recorder: Disk or RAM.

Disk Storage.
Traditionally the province of the computer industry, floppy disks have now invaded the studio but are subject to handling sensitivity, wear-out phenomena, as well as catastrophic media failure.

Floppies can store anything from 360K to 2 or 4 Mbytes. Hard Disks (also known as Winchester Disks) have a much larger storage capacity - 20 Mbytes or more - but they need to be regularly backed up onto disks or tape in case of data corruption or head crashes. Disks in general tend to be susceptible to mechanical failure - that is, life 'on the road' can seriously affect their performance. Floppy disks also have a limitation in being rather slow to get data on and off - it can take several minutes to load a complete high density disk. Hard disks are better in this respect but can suffer from the problem that they are never large enough to contain all the files that you then accumulate.

RAM Storage.
Few computers use RAM (random access memory) for longterm storage. Typically, only a few bytes of RAM are battery-backed in order to store set-up information and user preferences, but disks are usually the major data storage method. Battery-backed RAM is more often found in synthesizers, although the costs of providing enough back-up power often restrict the amount of RAM. Such protected RAM is called non-volatile - since normally the data disappears when you powerdown! Zyklus are one of the few manufacturers who store their data exclusively in RAM, and it is double battery protected - that means you need failures in both back-up systems before you lose any data. RAM has the advantage of being solid state and therefore very resistant to mechanical failure - there are no moving parts to go wrong. It also offers very fast access - no waiting for the disk to be read. The disadvantage of non-volatile RAM is that it is expensive to provide.


SYSTEM EXCLUSIVE

MIDI System Exclusive dumps or MIDI System Exclusive data can be stored on floppy disks using a MIDI Data Recorder - now commonly abbreviated to MDR. These dumps are very different from the more mundane and familiar MIDI information. Normally MIDI cables are kept busy transferring note-ons and note-offs between instruments or computers, and often supply the master clock information which keeps everything else in time. Most of the messages are short, usually only a few bytes long. The format for the Channel messages looks something like this:

<Status Byte>
  <Data Byte>
  <Data Byte>

The number of Data bytes which follow the Status byte is determined by the particular message. A Program Change message has only one following byte, whereas most other messages have two bytes following - for a Note-On message you get a Pitch Number followed by the Velocity data. Each of these messages can be followed by another similar message, although if Running Status is active, then the Status bytes are left out, giving a compressed form like this:

<Status Byte>
  <Data Byte>
  <Data Byte> end of 1st message
  <Data Byte>
  <Data Byte> end of 2nd message
  <Data Byte>
  <Data Byte> end of 3rd message
  ... etc

The shortest messages of all are the System Real Time messages, which control drum machines and sequencers, providing the basic timing clocks and start/stop/continue commands - these are all just single Status bytes:

<Status Byte>

The above applies universally to all MIDI products, regardless of manufacturer - this is the basic level of compatibility.

In contrast, System Exclusive data usually consists of information specific to a particular manufacturer and often to just that product or series of products. The format is specified but the meaning of the data is left to the individual manufacturer. The specified format looks like this:

<Sys Exc Byte>
  <ID Byte>
  <Data Byte>
  ... any number of Data bytes may follow...
<EOX>

This time the messages are in the form of a packet of bytes. The System Exclusive byte tells us it is the start of the packet, and only System Exclusive information is allowed until the end of the packet is reached - indicated by the End Of Exclusive (EOX) byte. The data contained within the packet can be of any length and can contain any information the manufacturer wishes.


Some dedicated MIDI Data Recorders

(*storage or disk format shown in brackets when known)
  • J L Cooper MIDI-DISK
  • Oberheim Prommer (RAM/EPROM)
  • Yamaha MDF-1 (Quick Disk)
  • Zyklus MPS (RAM/Cartridge)

Some sequencers with MDR facilities

  • Korg SQD-1 (Quick Disk)
  • National Logic/Indus MIDI-DJ
  • Roland MC500
  • Yamaha QX1
  • Yamaha QX3 (MS-DOS variant)
  • Yamaha QX5 (cassette or MDR 1 Quick Disk)

Some instruments with MDR facilities

  • E-mu Emax/Emax HD
  • Ensoniq SQ-80 (3½" disk)
  • Yamaha DX7IIFD (MS-DOS variant)
  • Some Yamaha Electones (E-Seq)

Some computer-based MDR systems

  • Mimetics SX-Connect for Commodore Amiga
  • Passport Master Tracks Pro for Apple Macintosh (Apple)
  • Yamaha CX5 MDR program (E-Seq/MSX-DOS)
For Atari ST (all MS-DOS variant):
  • C-Lab Creator
  • Hybrid Arts GENPATCH ST
  • Michtron Superconductor
  • Steinberg Pro-24



Previous Article in this issue

Practically MIDI

Next article in this issue

Edits


Sound On Sound - Copyright: SOS Publications Ltd.
The contents of this magazine are re-published here with the kind permission of SOS Publications Ltd.

 

Sound On Sound - Mar 1988

Topic:

MIDI


Feature by Martin Russ

Previous article in this issue:

> Practically MIDI

Next article in this issue:

> Edits


Help Support The Things You Love

mu:zines is the result of thousands of hours of effort, and will require many thousands more going forward to reach our goals of getting all this content online.

If you value this resource, you can support this project - it really helps!

Donations for January 2021
Issues donated this month: 0

New issues that have been donated or scanned for us this month.

Funds donated this month: £66.00

All donations and support are gratefully appreciated - thank you.

Please Contribute to mu:zines by supplying magazines, scanning or donating funds. Thanks!

Monetary donations go towards site running costs, and the occasional coffee for me if there's anything left over!
muzines_logo_02

Small Print

Terms of usePrivacy