Home -> Magazines -> Issues -> Articles in this issue -> View
Everything You Ever Wanted To Know About System Exclusive (Part 8) | |
(But Were Too Afraid To Ask!) |
PART 8: Martin Russ explains the circular nature of parameter editing and data dumps.
In the last installment I described the way that individual bits within bytes could be used to represent different parameters when less than seven bits were needed to represent the value. The Kawai K5 example mentioned that the same parameter table could be used to show the parameter numbers and the contents of the bulk dump. I will expand on this theme this month and present some examples of the sort of bit-formatted bytes which you may find in your own MIDI equipment.
The bit-formatted bytes shown last month looked like this:
0aaa bbbb 0ccd deee
and compress more than one value into a single byte. This makes editing such values difficult because we cannot use a program like SYSEX to change one value in isolation - each byte we alter changes two or more values. The Yamaha FB01 example below discusses some approaches to how this might be attempted, but for the purposes of explaining the theory we will concentrate on the 'why', not the 'how'...
So why do some manufacturers choose to put several parameters into the same byte? The answer probably lies in two separate and different reasons: firstly, by compressing parameters together in this way, the RAM memory needed to store them is reduced and so more sets of parameters can be stored; and secondly, it is much more convenient for the programmers to provide the same method of editing parameters via MIDI as it is from the front panel, so if the parameters are stored in a compressed way, then they are edited in that form. Unfortunately, whereas it is quite easy to manipulate individual bits in a microprocessor by a process of shifting and ANDing, it is not quite so easy via MIDI.
For example, suppose that three parameters, a, b and c, are bit-formatted in the following form:
0abb bccc
To alter the three-bit parameter bbb to a particular value xxx, we need to shift the value to the left by three positions:
original value: | 0000 0xxx |
shifted left: | 0000 xxx0 |
and again: | 000x xx00 |
and finally: | 00xx x000 |
which matches: | 0abb bccc |
byte: | 0abb bccc |
mask: | 0011 1000 |
masked byte: | 0a00 0ccc |
AND with: | 00xx x000 |
to give: | 0axx xccc |
Part 2 of this series [SOS May 1989] looked at the typical format of a System Exclusive dump:
$F0 SysEx
$ii Identification
$ss Sub-Status
[block of data]
$cc checksum
$F7 EOX
So far we have concentrated on the parts of this format surrounding the block of data, and the complete packet of information as a whole as stored by our Movie program, but no mention has been made of the contents of the block of data in a bulk dump from a typical synthesizer, until now.
As with almost all System Exclusive formats, no true standard exists, nor does a standard need to exist! Each manufacturer has developed their own format for the data within their own packets of information, and these are generally freely available on request - the System Exclusive Book from the SOS Bookshop (see panel) contains details of some of the System Exclusive formats, but it does not go into the same depth of detail as I am about to give!
Because MIDI instruments have different numbers of sound or patch memories, and differing complexity and storage requirements, I can only outline the general form of a bulk dump for a hypothetical instrument. The SOS Vaguely Defined Hypothetical Hi-Tech Musical Instrument (or SOS VDHH-TMI, as we will call it) has sounds which are made up out of a number of parts, and has storage for several sounds. The [block of data] consists of:
[ Parameters for part 1 of sound number 1]
[ Parameters for part 2 of sound number 1 ]
[ Parameters for part n of sound number 1 ]
[ Parameters for part 1 of sound number 2 ]
.....
[ Parameters for part n of sound number 2 ]
...and so on for all the remaining sounds
The parameters are sometimes just a listing of the editable parameters, but they can be more complex. Decoding these lists is an extension of the ideas we have been pursuing so far.
We have already been looking at parameter tables, and the listings of dumps are much the same in format, as we have discovered. To place the two forms in context, Figure 1 contains part of the parameter table and Figure 2 the dump list for the Yamaha DX7:
Figure 1. DX7 PARAMETER TABLE
PARAMETER | NO. | DATA |
OPERATOR 6: | ||
Level Scaling Left Curve | 11 | 0-3 |
Level Scaling Right Curve | 12 | 0-3 |
Keyboard Rate Scaling | 13 | 0-7 |
Amplitude Mod Sensitivity | 14 | 0-3 |
Key Velocity Sensitivity | 15 | 0-3 |
Output Level | 16 | 0-99 |
Oscillator Mode | 17 | 0-1 |
Oscillator Coarse Frequency | 18 | 0-31 |
Oscillator Fine Frequency | 19 | 0-99 |
Oscillator Detune | 20 | 0-14 |
PARAMETER | BYTE | FORMAT |
OPERATOR 6: | ||
Level Scaling Left Curve | 11 | 0000 xx00 |
Level Scaling Right Curve | 11 | 0000 00xx |
Oscillator Detune | 12 | 0xxx x000 |
Keyboard Rate Scaling | 12 | 0000 0xxx |
Key Velocity Sensitivity | 13 | 000x xx00 |
Amplitude Mod Sensitivity | 13 | 0000 00xx |
Output Level | 14 | 0xxx xxxx |
Osc. Coarse Frequency | 15 | 0xxx xxx0 |
Osc. Mode | 15 | 0000 000x |
Osc. Fine Frequency | 16 | 0xxx xxxx |
To show that continuous development really does happen here at SOS, this month's program is an update to the SYSEX Utility (you were given a sneak preview of the program in one of last month's screen shots). Version 0.3 of the program enables values to be entered in hex and decimal, as well as converted between the two. Some additional manufacturers have also been added to the buttons. Further developments are in the pipeline!
Continuing the theme of editing, but adding bit-formatting and dump listing, here are a few more examples of System Exclusive editing messages.
The first true workstation from the major exponents of FM synthesis, the V50 is a powerful, flexible and somewhat underrated synthesizer. [A disk of voices and performances programmed by the author will soon be available from SOS Shareware; see yellow pages - Ed.] The supplied MIDI information is of a high quality and should pose few problems. For example, the V50's Program Change Table can be tedious to set up using the front panel controls and so could be a suitable candidate for editing via System Exclusive. The manual gives the required information in a very clear form:
1111 0000 | F0 |
0100 0011 | 43 |
0001 nnnn | nnnn = Device Number |
0ggg gghh | ggggg = 00100 (4), hh= 00 (0) |
0ppp pppp | ppppppp = 1111111 (127) |
0kkk kkkk | kkkkkkk = Program Change Number |
0hhh hhhh | hhhhhhh = high data byte |
0111 1111 | 1111111 = low data byte |
1111 0111 | F7 |
High byte | Low byte | Voice/Perf. | Memory |
---|---|---|---|
0 | 0-99 | Voice | Internal I0-I99 |
1 | 0-99 | Voice | Card C0-C99 |
2 | 0-99 | Voice | Preset P0-P99 |
3 | 0-99 | Perf. | Internal PFI00-PFI99 |
4 | 0-99 | Perf. | Card PFC00-PFC99 |
5 | 0-99 | Perf. | Preset PFP00-PFP99 |
1111 0000 | F0 |
0100 0011 | 43 |
0001 nnnn | nnnn = Device Number |
0ggg gghh | ggggg = 00100 (4), hh= 00 (0) |
0ppppppp | ppppppp = 1110110 (118) |
0sss ssss | sssssss = switch number (Table 1) |
0ddd dddd | ddddddd = switch data 0=Off, 127=On |
11110111 | F7 |
9 | Stop |
10 | Play |
54 | Effect Bypass |
56 | Reset |
The most pressing need for System Exclusive editing comes from instruments which have no front panel editing facility. Only two notable examples of this come immediately to mind, both of which were subsequently (and quickly) replaced (or augmented) by similar products with editing features! The two expanders in question are the Roland MT32 (the D110 is the editable version) and the Yamaha FB01 (the TX81Z is the editable version); neither makes any particular provision for the editing of the sounds with the front panel. Of these two, the Yamaha FB01 has acquired a certain notoriety as being difficult, if not impossible, to edit without using a dedicated software package. Interestingly enough, the latest generation of products designed to form part of a complete computer-based music generation system are based on similar technology to the non-editable versions; so perhaps repackaging does work in the long term after all.
The problems with the FB01 are three-fold: firstly, the manual is too detailed, but unfortunately splits the detail into a large number of tables and lists of parameter blocks etc, spaced out over several pages; secondly, the MIDI messages are both nibbleised, bit-formatted and addressed via offsets to values held in other tables, making the assembly and editing of parameters very difficult; and thirdly, the manual does not seem to give any clear idea of how the various component parts of the System Exclusive messages fit together — for example, there are seven different and incompatible bulk dumps plus an entirely non-standard way of generating notes.
Having waded through the MIDI documentation whilst writing an editor program for the FB01, I have spent a considerable amount of time working out exactly what is going on inside. So what follows is mostly given in the manual, but presented here in a (hopefully) understandable form.
There are three basic sets of parameter in the FB01:
1. Voice Data. This includes the basic information about the FM parameters used to produce the sounds - the parameters are detailed in the Voice Data block. Because the FB01 is multitimbral, sounds can be edited by the MIDI channel they are assigned to, or by the MIDI channel plus the part or instrument number. The editing messages have the value data split into 4-bit nibbles over two bytes.
2. Configuration Data. This includes the majority of the multitimbral assignment and control information - again, edited by channel or channel plus instrument number. In this case, the value data is sent as bytes instead of nibbles. Confusingly, the parameters are held in the Instrument Definition block, not the Configuration Data block.
3. System Data. This covers only a few global parameters: the master output level, the memory protect, etc. This is only Editable by MIDI channel (since it deals with all the instruments, it is not accessible by channel plus instrument number) and uses bytes to represent the values held in the Configuration Data block.
Armed with this road map of the FB01, we can now explore a few messages. The System and Configuration messages seem the best place to start, since they do not use nibbleised data values. Master Output Level seems like a useful starting point, especially since there is no front panel control for overall output level, and also the normal MIDI Volume Controller ($07) is channelised and so affects only the instruments assigned to that channel. The basic System editing message is:
$F0 | Sysex |
$43 | Yamaha ID |
$75 | Sub-status |
$0s | s = system channel number |
$10 | Parameter Group |
$pp | Parameter Number |
$dd | value data |
$F7 | EOX |
$F0 | Sysex |
$43 | Yamaha ID |
$75 | Sub-status |
$0s | s = system channel number |
%0001 1iii | Instrument Number (0-7 = 1-8) |
$pp | Parameter Number (0-$17) |
$dd | value data |
$F7 | EOX |
Param No. | Value Range | Name |
$00 - $0F | (see Instr def block p52) | |
$10 | 0-127 | LFO Speed |
$11 | 0-127 | amd |
$12 | 0-127 | pmd |
$13 | 0-3 | LFO waveform |
$00 | %0000 xxxx | number of notes (0-8) |
$01 | %0000 xxxx | MIDI channel (0-15) |
$02 | %0xxx xxxx | high key limit (0-127) |
$03 | %0xxx xxxx | low key limit (0-127) |
$04 | %0000 0xxx | voice bank (0-6) |
$05 | %0xxx xxxx | voice (0-47) |
$06 | %0000 0xxx | detune (0-127) |
$F0 | Sysex |
$43 | Yamaha ID |
$75 | Sub-status |
$0s | s = system channel number |
%0001 1iii | Instrument Number (0-7 = 1-8) |
$pp | Parameter Number ($40-$7F) |
$0y | low nibble of value data |
$0x | high nibble of value data |
$F7 | EOX |
$0A | %xyyy yyyy | x = LFO sync at note on mode (0=Off, 1=On) y = Pitch Mod Depth (pmd) |
$0B | %0wxy z000 | w = operator enable 3 x = operator enable 2 y = operator enable 1 z = operator enable 0 |
$0C | %lrxx xyyy | (l,r = Stereo output enables - this is not shown in the Yamaha documentation!) x = feedback level y = algorithm |
S0D | %0xxx 00yy | x = Pitch Mod Sensitivity (pms) y = Amplitude Mod Sensitivity (ams) |
$0E | %0xx0 0000 | x = LFO waveform (0=saw, 1=square, 2=triangle, 3=random) |
System Exclusive editing can sometimes provide access to features of an instrument which are either difficult or impossible to obtain from the front panel. More importantly, such editing messages can be embedded into sequencer tracks to provide control over instruments as they are playing, without the need for 'real time' manual changes of parameters. Sometimes, using System Exclusive is easier than pressing a dozen buttons in the right order whilst bent double trying to read the display on an expander at the bottom of the effects rack! In these days of universal and generic editors and MIDI controller software, the ability to be able to exploit System Exclusive data with these tools is becoming increasingly important.
Next month, we return to the subject of bulk dumps, and look at their storage and the calculation of checksums.
FURTHER INFORMATION
The System Exclusive Utility program referred to in this article (plus other useful programs) is available on the 'SysEx ToolKIt 2' disk, price £7.
SOS Software, (Contact Details).
MDR Bulk Dumps | Mar 88 |
How to use MDRs | May 88 |
Storing SPX90 Patches | Oct 88 |
MIDI Automation Systems: | |
Part 1 | Feb 88 |
Part 2 of 2 | Mar 88 |
MIDI Effects: | |
Part 1 | Dec 85 |
Part 2 of 2 | Jan 86 |
Integrating MIDI & Studio | Sep 88 |
MIDI Matters: | |
Song Position Pointers | Feb 87 |
Implementation Charts | Mar 87 |
Timbre Editing | Apr 87 |
Sample Editing | Jun 87 |
MIDI Basics: | |
Part 1 | Aug 87 |
Part 2 | Sep 87 |
Part 3 | Oct 87 |
Part 4 | Dec 87 |
Part 5 of 5 | Feb 88 |
MIDI Networks | Sep 86 |
MIDI Software Guide | Oct 89 |
MIDI Software Roundup | Feb 86 |
MIDI: | |
The Universal Answer? | Nov 85 |
What's Wrong With It? | Apr 86 |
Practically MIDI: | |
Part 1 | Sep 87 |
Part 2 | Oct 87 |
Part 3 | Nov 87 |
Part 4 | Dec 87 |
Part 5 | Jan 88 |
Part 6 | Feb 88 |
Part 7 | Mar 88 |
Part 8 of 8 | Apr 88 |
System Exclusive: | |
Part 1 | Apr 89 |
Part 2 | May 89 |
Part 3 | Jun 89 |
Part 4 | Jly 89 |
Part 5 | Aug 89 |
Part 6 | Sep 89 |
Part 7 | Oct 89 |
Part 8 | Nov 89 |
Talking MIDI: | |
Part 1 | Dec 85 |
Part 2 | Jan 86 |
Part 3 | Feb 86 |
Part 4 | Mar 86 |
Part 5 | May 86 |
Part 6 | Jun 86 |
Part 7 | Jul 86 |
Part 8 of 8 | Aug 86 |
Read the next part in this series:
Everything You Ever Wanted To Know About System Exclusive (Part 9)
Topic:
Series:
Part 1 | Part 2 | Part 3 | Part 4 | Part 5 | Part 6 | Part 7 | Part 8 (Viewing) | Part 9 | Part 10 | Part 11 | Part 12
Feature by Martin Russ
Previous article in this issue:
Next article in this issue:
mu:zines is the result of thousands of hours of effort, and will require many thousands more going forward to reach our goals of getting all this content online.
If you value this resource, you can support this project - it really helps!
New issues that have been donated or scanned for us this month.
All donations and support are gratefully appreciated - thank you.