MIDI Data Portability
Thinking of changing drum machines, adding a new synth to your MIDI setup, or buying a new sequencer? Paul Andreas Overaa outlines some sensible tips to help you remove the need to edit your sequences every time you change your equipment.
Most MIDI users have yet to come across serious MIDI data portability problems, but this does not mean that such problems do not exist. Whether it be tomorrow, next week or next year, you will at some time add to (or change) certain parts of your existing MIDI setup. New expanders, extra drum machines and so forth can certainly create a few snags... and when you finally upgrade your computer or sequencer, the fun really starts!
What can you do to avoid the pitfalls? One of the most effective ways is to learn a bit about the areas which cause the difficulties - and then do a bit of advance planning so that as many of the trouble spots as possible can be avoided!
With the early generation of MIDI drum machines, each drum sound was allocated a fixed note number by the manufacturer. Originally, many people hoped that a common note assignment standard would be adopted, so that Yamaha's bass drum notes would be the same as Roland's, which would be the same as those on an Alesis machine, etc. There are several reasons why this didn't happen, and the end result was that virtually all professional sequencers started to incorporate drum note remapping facilities to ease the burden of translating drum parts from one set of drum/note number assignments to another. (See 'Drum Note Assignments' table.)
Whilst the idea of remapping drum parts is fine when you only have a few songs and sequences to modify, nowadays a growing number of MIDI musicians have song libraries containing hundreds of songs, each of which contain many sequences. Because of the time it takes, having to sit down to change every song in such a library is an absolute nightmare.
The present generation of drum machines, such as Yamaha's RX8 and Alesis's HR16, have opted for programmable drum/note number assignments. This type of machine can be set up so that the new drum/note number correspondences match the existing drum note relationships of your stored sequence data. Reconfiguring a drum unit like the Yamaha RX8 takes less than five minutes, and this completely avoids having to edit your existing song data.
So, if this problem has now been solved, why am I bothering to mention it? Two reasons really: Firstly, there's still a large number of drum machines with fixed drum/note number assignments (such as Yamaha's RX21 and RX21L) available on the secondhand market. These can be a cheap way of adding extra percussion facilities to your MIDI setup, but the main danger is that some of their drum note assignments might clash with other assignments you've already adopted. So, before you opt for additional drum units like these, check the MIDI Implementation chart to see that the extra unit's drum note values do not clash with anything you're already using.
The second reason I mentioned the drum machine problem is to drive home a very important point. When only half a dozen songs are involved, going through each sequence and making various edit changes to drum notes is no real hardship. After a few years of arranging and collecting MIDI sequence data however, your 'sequence library' is likely to have grown to the point where manual edit operations are no longer feasible - unless you have a lot of time on your hands.
If you add a new synth to your setup, or change expanders, your existing MIDI sequences will need some tweaking in order to play as intended. The first thing that needs to be done is to identify or create voices which correspond to the voices available on your previous instruments. The second thing is to make sure that these voices can be used when needed. Voice selection is usually done via MIDI 'program change' commands, and these will be embedded in your sequences. For example, if program change number 23 had selected a cello voice on your old Roland synth, then you need to be able to make any new unit select an equivalent voice when it receives a program change 23 command - otherwise, once again, you will be faced with editing much of your existing song library data!
The important facility to look out for on any new instrument is the existence of a user-definable 'program change table'. This lets you assign any voice to any program change number. In short, you create or find the required voices on your new unit, and then define the table so that program numbers already embedded in your existing sequences select equivalent voices in your new expander or synthesizer. By the way, don't forget to keep records of the program change/voice assignments you have in use at the moment, because it's a dead cert that one day you'll need them.
Now for some real potential nasties: MIDI users are going System Exclusive (SysEx) mad, but are not being made sufficiently aware of the potential pitfalls of overusing SysEx messages in their sequences.
These days a lot of MIDI expanders offer very comprehensive SysEx control facilities - including front panel control, bank swapping, voice selection, and individual voice dump/receive options. With expander units like Yamaha's TX81Z, which has quite a sophisticated set of SysEx control functions, it's possible to transmit and set up new voices, modify the program change table etc (so that an alternative set of voices are made available), just prior to the actual start of the music sequence - during the count-in time, say. Such facilities are brilliant... until the time comes when you change expanders and realise that all your clever SysEx stuff is going to have to be completely re-written. System Exclusive messages, you see, are specific to each instrument.
My advice? Certainly experiment and learn about System Exclusive messages because it's important, but as far as possible try to minimise the use of such messages in your sequences.
MIDI's switch and continuous controller messages fall into two groups as far as data portability problems are concerned - those with standardised uses, such as mod wheel (controller 1), pan (controller 10), and volume (controller 7); and those which have more specialised, equipment specific, uses. Problems in this area can be minimised: firstly, by sticking to MIDI controllers which seem to adhere to a de facto standard usage category (see 'Controller' table); secondly, by opting for instruments which offer user-assignable controller functions.
Although, up to now, these options have only been offered with the more expensive MIDI products, there's a good chance that more and more budget priced MIDI equipment will include assignable controller features as the trend for increased programmability develops. Before you buy new gear, take a good look at the MIDI Implementation sheet in the owner's manual and check that none of the controller messages already present in your sequences will conflict with those used by the new equipment. Most sequencers will let you re-assign controller numbers embedded in your sequence data, but ideally you should aim to avoid placing yourself in a position where it becomes necessary to edit controller numbers in all of your existing sequences.
This is where the fun really starts! Disk sizes and disk formats vary from sequencer to sequencer, as do file formats (the logical structure which a particular sequencer uses to represent and store the MIDI data).
If you change from a dedicated hardware sequencer, such as the Roland MC500 or Alesis MMT8, to a computer/software-based sequencer, like an Atari ST running Steinberg's Pro24, C-Lab's Creator or Dr.T's KCS, then the safest way of transferring your existing sequencer files is to do it in real time via MIDI.
Connect the MIDI Out of the first sequencer to the MIDI In of the new sequencer, then play your existing sequences on the first machine whilst you record them one at a time on the other. Don't forget that the sequencer you are recording on should be set so that it is using MIDI clock messages and not any internal clock, otherwise you'll lose all of your existing bar definitions! This method of direct linking has several advantages: it's simple to do and, even though it takes time, it always works!
If you've already got a computer-based sequencer and are changing over to either a different computer or a different sequencer package, then there are two alternative possibilities. If the file formats of the two sequencers are compatible, it may even be possible (using a serial communications program or self-written software) to transmit the sequencer files from one machine to the other directly. Steinberg's new Amiga 24000 sequencer (the Amiga equivalent of Pro24), when it arrives, will hopefully read Pro24 files unchanged. So if you currently use Pro24 on the Atari ST and are thinking of moving to an Amiga, then this is likely to be a very fast way of transferring your sequencer data files.
Another alternative for computer-based sequencer users is the existence of Standard MIDI Files (SMF). In theory, the SMF is a good idea - define a common file format, then offer sequencer program options or separate utilities which let users translate their files into a format which other sequencer programs can then read. In practice, things have progressed less than smoothly on the SMF front. Since the initial introduction of the Standard MIDI File specification some years ago, there seem to have been various changes made to the specification and, to date, there now appear to be at least three types of 'standard' files available (SMF types 0, 1, and 2).
I've spent a lot of time looking at the contents of these different file arrangements and studying them in relation to the initial standard - there are some very worrying differences, and I can honestly say that I've not come across one person in the UK who has any clear idea of the current state of the standard. If, at the current time, you do decide to use SMF facilities, my advice would be to use them with caution.
Before you buy any new MIDI equipment, check the MIDI Implementation charts and look very carefully for potential areas of incompatibility with the sequence data you've built up during your MIDI songwriting lifetime. Look for flexibility in the areas of program change tables and MIDI controller assignability, and identify controller number conflicts before you commit yourself to a new MIDI unit. Above all, avoid too much dependence on control applications of System Exclusive messages - they are equipment specific and invariably cause serious portability problems when embedded in song data libraries.
Over the past few years I've transferred hundreds of songs across to new sequencers and have used these song arrangements with a wide range of MIDI synthesizers, expanders, drum machines, and effects units. In doing so, the one point that has really been driven home to me is that the key issue in such transfers and equipment changes is to find out how to avoid having to undertake any major editing of your existing sequence data!