• Everything You Ever Wanted T...
  • Everything You Ever Wanted T...
  • Everything You Ever Wanted T...

Magazine Archive

Home -> Magazines -> Issues -> Articles in this issue -> View

Everything You Ever Wanted To Know About System Exclusive (Part 7)

(But Were Too Afraid To Ask!)

PART 7: Martin Russ delves deeper into the secrets and applications of MIDI's System Exclusive messages.


You have probably noticed a major difference between the two programs I described in the preceding two parts of this series - Part 5 featured a universal continuous MIDI Controller with the Atari's mouse as the driving device, whilst Part 6 provided a rather tedious method of changing values by typing numbers and then pressing the Return key to send the MIDI message. As most real world parameters are much easier to edit with some sort of continuous control, I have written two simple utility programs which convert a MIDI Controller message and mouse movements into a pseudo general purpose System Exclusive message. The two programs are called SXMOD and SXMOUSE - no prizes for figuring out that SXMOD works with the Modulation Wheel and SXMOUSE converts the mouse position.


THEORY



These programs actually illustrate a very interesting point about System Exclusive editing messages which needs further explanation. When you start using the programs, you should quickly become aware of an interesting difference between them and SYSEX. Unlike SYSEX, the two programs this month can only send a single byte of variable data in response to the controller (Mod Wheel or Mouse). This may not seem so important at first sight, after all, I said previously that System Exclusive messages were of the form:

$F0 Sysex
$ii Identification
[sub-status]
[parameter]
[value]
$F7


and so the [value] section just needs to be replaced by the Mod Wheel or Mouse position, doesn't it? Actually, as I mentioned in Parts 4 and 6, the [value] section can be one or more bytes - typically it is either one or two bytes, usually representing a 7-bit or 14-bit value. Parameters that require more precision tend to be split into additional parameters, like the Coarse and Fine Frequency parameters on an FM synth, for example. For the single and double byte there are several ways that the value can be represented - only one of these was used last month, but this month we will look at them in more detail. Here are some of the possible formats:

• Single byte representing 7-bit value. Here seven bits (or less) are used to represent the value of the parameter over a 0 to 127 range. This is the way that the DX7 example last month represents parameter values. If less than seven bits are used, the available range drops in proportion:

Bits Min Max
7 0 127
6 0 63
5 0 31
4 0 15
3 0 7
2 0 3
1 0 1

This sort of byte format is usually represented in System Exclusive documentation as:

0aaa aaaa for a 7-bit parameter where the most significant bit (MSB) is a zero as expected, and the 'a's show the number of bits used for the parameter. The zero is usually extended downwards as less bits are used, eg:
000b bbbb for a 5-bit parameter

• Single byte representing several low precision values. These are used where the size of the data dump is minimised, since they allow the number of bytes to be kept small by compressing several parameter values into one byte. These bytes usually appear in the documentation in one of two forms:

0ccd deee
or
0cc
   d d
      eee


where cc represents a 2-bit parameter, dd is another 2-bit parameter, and eee is a 3-bit parameter. Sending these as three separate bytes would produce:

0000 00cc
0000 00dd
0000 0eee


which wastes a lot of space and replaces it with zeroes, but would be much easier to control with a utility program like SYSEX, and very hard to control with a program which only allows you to change a single 7-bit value - in this case, it would be altering three separate parameters all at once.

• Two bytes representing a 14-bit (or less) value. A 14-bit number provides 16,384 different values and so has more available precision than most internal parameters commonly encountered in synthesizers! This often means that less than the full 14 bits are used - in some cases only eight bits, in this form:

0000 000f
0000 0000


where the 8-bit byte formed from fggg gggg is expressed in two bytes, wasting the majority of the high order byte. The order of high and low bytes can also be reversed, depending upon the particular manufacturer, so you will need to examine the documentation carefully, since the same eight bits could also be expressed as:

0ggg gggg
0000 000f


if the high byte is transmitted after the low byte. You will often find that an individual manufacturer is consistent within their own products, but do not depend on this!

• Two bytes representing an 8-bit value in a nibbleised form. This is similar to the previous format but leaves the whole of the high order nibble (four bits) as zeroes, resulting in this:

0000 hhhh
0000 iiii


where the eight bits are hhhhiiii (or iiiihhhh depending on the order of transmission). This form is also not very efficient and tends to produce long dump packets. It also obscures ASCII textual information - and so our ASCHEX program will produce a display which shows:

0000 05 03 05 09 05 03 04 05 05 08 etc

when an un-nibbleised version would show:

0000 53 59 53 45 58

which is the ASCII coding for 'sysex' and would be shown on the right-hand side of the screen in the ASCHEX program. (I am still working on altering ASCHEX so that it can show either form.)

The complex way that values are represented in System Exclusive messages makes designing general purpose editing programs difficult, and makes decoding or verifying data by hand from bulk dumps error-prone and tedious. As far as this month's programs are concerned, the lack of nibbleised values means that anything which sends data in the form of nibbles or separate groups of bits will not be controllable over its whole range.

The Korg M1 example given last month only used the low seven bits from a 14-bit representation, by setting the upper bits to zero. This was perfectly acceptable because the M1 only allows 63 values for its Cutoff Frequency parameter - so there was one bit to spare from the available seven bits. A similar scheme can be used for nibbleised values, except that using only the lower four bits restricts the available control range to 0 to 15, out of a possible range of 0 to 127. In these cases, the SYSEX program is a much better way of making edits, although changing the numbers that represent the parameter values will be tedious. Perhaps once you are hooked on editing with System Exclusive, you will consider purchasing one of the many Universal/Generic Editor programs which are now becoming available.

As we discovered in Part 5, this complication with nibbles, bits and bytes has arisen because no-one (except Ensoniq) seems to use Registered or Non-Registered Parameters. If you recall, MIDI Controllers are either 7-bit or 14-bit, and the representation is closely defined in the MIDI Specification 1.0 - this is why the MIDI Controller program (MC03MR) can be so simple, whilst SYSEX V0.3 (the version I am currently working on) is many times more complex.


PRACTICE



All of these examples of editing formats (like 0aaa bbbb using two's complement number representation) are only useful if they can be applied to real world situations. Armed with your new knowledge, try looking at last month's M1 example again and see if the description of Note 13 in the Korg documentation makes any more sense now! You should also be able to see why SYSEX is far superior for editing purposes than this month's two programs - which make editing easier and less tedious, but compromise on flexibility. To point you in the right direction, I will describe a few examples of single byte editing messages suitable for use with SXMOD or SXMOUSE.

Example 1: E-MU EMAX

The powerful Emax sampler has a comprehensive MIDI System Exclusive implementation which includes MIDI Sample Dumps as well as more conventional editing. The documentation is dear and unambiguous - all the System Exclusive editing messages are of this form:

$F0 Sysex
$18 E-mu ID
$02 Emax ID
$nn Exclusive Command
$cc control byte(s)
$vv value of parameter
$F7 End Of Exclusive

Some of the Exclusive Commands are requests for dumps, some execute the equivalent of front panel switches, and others allow editing:

Command Action
$1A Change Voice parameter
$1B Change Preset parameter

The Voice parameters and Preset parameters are given in separate tables, which detail the parameter, the number of bits used and the allowed range of values. Because samplers like the Emax tend to need lots of information about exactly which sound is allocated to which key, and what sort of assignment priority it has, you need to supply the high and low keys to define the Voice (in much the same way as you do when editing from the front panel), as well as the Primary, Secondary or both voices. This makes it difficult to describe a typical editing message because of the possible variations. Instead, I will show how the Voice Parameter table gives us all the information that is needed to use SXMOD or SXMOUSE (or even SYSEX) to edit a parameter. The Emax Voice Parameter table looks like this:

Parameter (Dec) Bits Value Range
31 Filter Cutoff 7 0-120
32 Filter Q 7 0-99
33 Envelope Amount 7 0-100
39 VCF Release 5 0-31
44 Velocity to Q 4 0-15
53 Chorus 1 0-1 (On/Off)


So a message to control the Filter Cutoff would be of the form:

$F0 Sysex
$18 Emu ID
$02 Emax ID
$1A command=change voice parameter
$ll low key number (0-87)
$hh high key number (0-87)
$0l level (0=primary; 1=secondary, 2=both)
$pp voice parameter (see table)
$vv new value of parameter
$F7 End Of Exclusive

The table referred to gives exactly the sort of information that you need in order to decide on the number of bits and the allowable range of values for the parameter. Once you know the format of the message and the number of bits in the value, then all you need to do is plug the values into the message and send it.

Example 2: CASIO CZ101/1000/5000

Casio have a clear and well thought-out way of showing exactly how to send System Exclusive messages: they use a table which shows the bytes in boxes with the order of transmission from left to right. Responses to requests and further commands are shown on subsequent lines, so the whole thing reads in much the same way as a piece of text. For simple. parameter changes like Key Transpose, the table is simple:

Computer -> CZ101/CZ1000/CZ5000
F0 44 00 00 7n 41 d1d2 F7


where n is the current MIDI channel, and the values of d1 and d2 are given by another table:

Key: G A A# B B# C C# D D# E F F#
d1d2 45 44 43 42 41 00 01 02 03 04 05 06

where all numbers are in Hexadecimal. Such a message would be difficult to use with anything other than the SYSEX program, because of the limited range of allowable values. The Casios CZ5000 has other controllable parameters which have more uses:

Computer -> CZ5000
F0 44 00 00 7n 44 d1d2 F7


Here d1d2 can range from 0 to 99 ($00 to $63) to control the Glide or Portamento time.

Computer -> CZ5000
F0 44 00 00 7n 45 d1d2 F7


Here d1d2 can again range from 0 to 99 ($00 to $63) to control the Modulation Depth. Both of these messages are easily controllable by using SXMOD or SXMOUSE.

Example 3: KAWAI K5

This is probably my favourite synth of the moment, and so I have been using my own utilities (and Hollis Research's MIDIman - I am writing a 'profile' for the K5) to explore some of the editing possibilities. The Kawai documentation uses a single chart to show the contents of the dumps and the parameter numbers, and so needs some interpretation. The editing message is more or less as you might expect but it throws up an interesting example of using two bytes to represent an 8-bit parameter value, but combines this with a 2-bit control as well!

$F0 Sysex
$40 Kawai ID
$0n Channel Number
$10 Parameter Send
$00 Synthesizer
$02 K5 or K5m
0000 0aab aa = Source select: 0 = S1, 1 = S2, 2 = Both
b = MSB of parameter number (bit 7)
0ccc cccc ccccccc = parameter number (bits 0-6)
$0h high value nibble
$0l low value nibble
$F7 End of Exclusive

The use of hexadecimal numbers for fixed values and binary numbers for values where the individual bits are significant is quite common - remember that they are just different ways of showing the same 8-bit number. The K5 uses nibbleised values for its parameters, and two's complement for negative numbers - so you will need to change two bytes in the SYSEX program message box for each edit. Here are two excerpts from the Kawai parameter chart:

(DDF)
S381 0ccc cccc 107 S1 Cutoff 0-99
S382 - - S2 -
S383 000m mmmm 111 S1 Cutoff Mod 0-31
S384 - - S2 -
(LFO)
S470 pppp pppp 175 Speed 0-99

The 'S' numbers on the left refer to the number of the byte in the dump (but don't forget that the K5 sends nibbles of data, so each parameter needs two bytes to be transmitted; each number therefore needs to be doubled to give the position in the dump). The next column shows the format of the bits in the reconstructed byte (after putting the low and high nibbles together). The next column shows the parameter numbers - because the Sources which will be affected by the editing are determined by the 'aa' bits in the System Exclusive parameter editing message, only one parameter number is used to edit the Filter Cutoff Frequency of either S1, or S2, or both.

The parameter numbers can range from 0 to 173, and so the full eight bits are needed in order to set the correct parameter number. Notice that the parameter number is split into a 7-bit and 1-bit form, NOT the parameter value! You should always be prepared for the unexpected and unusual in System Exclusive editing.

SUMMARY



These examples begin to show the way that apparently simple System Exclusive messages can require careful consideration before they can be used effectively. As the complexity increases with bit-formatted examples, even more care is needed to be able to create System Exclusive editing messages. Also, we begin to enter a twilight zone where editing messages and bulk dumps converge - almost a return to a previous part of the series!

The next installment will look at some even more complex examples and attempt to make their innermost meanings comprehensible, as well as describing the link with bulk dumps.

FURTHER INFORMATION

The System Exclusive Utility program referred to in this article (plus other useful programs) is available on the 'SysEx Toolkit 2' disk, price £7.

SOS Software, (Contact Details).

POSITIVE & NEGATIVE

One problem with numbers lies in the way that numbers are represented inside computer-based equipment. I have often assumed that the values are between zero and some larger positive value, but many parameters actually vary in a positive and negative direction from the zero value. There are several ways to represent such 'signed' numbers, and these can be expressed in the same types of one or two byte formats described in the main text, giving a huge set of possible ways to represent values of parameters. Two commonly used ways to show positive and negative values are:

• SIGN AND MAGNITUDE. Here the most significant bit is used to show the sign (0 = positive, 1 = negative) whilst the remaining bits show the actual value, ignoring the sign:

0j00 kkkk

Here 'j' shows the sign, whilst 'kkkk' is a 4-bit value, giving a total number range which can be represented of +/-15. In this case, 0 would be 0000 0000, whilst +15 would be 0000 1111 and -15 would be 0100 1111. Notice that the 'kkkk' magnitude bits only give information about the size of the number - and the only difference between -15 and +15 is the 'j' bit.

• TWO's COMPLEMENT. In this representation things are more complicated, but more closely reflect the way that real world microprocessors often work. If you look at an increasing series of binary numbers:

0 0000
1 0001
2 0010
3 0011
4 0100
5 0101
6 0110
7 0111
8 1000
9 1001 etc

but ignore the left-most column, then when the count has reached 7, the three right-most bits are all 1s; and, at the count of 8, all these bits revert to 0s. This suggests that above the 0 value of 0000, we might expect to see 1111 - in other words, that -1 should be represented by an all-ones binary number. If we wrap the series of numbers around on itself we will get:

-8 1000
-7 1001
-6 1010
-5 1011
-4 1100
-3 1101
-2 1110
-1 1111
0 0000
+1 0001
+2 0010
+3 0011
+4 0100
+5 0101
+6 0110
+7 0111

Notice that if we took the most significant bit (MSB) to be the sign bit, then these numbers would be the same as the sign and magnitude representation for positive values, but different for negative values: —7 in 4-bit sign and magnitude is -1 in this case.

We also have more negative values than positive values: -8 as opposed to +7.


Series



Previous Article in this issue

Hollis Research MIDIman

Next article in this issue

Digital Signal Processing


Sound On Sound - Copyright: SOS Publications Ltd.
The contents of this magazine are re-published here with the kind permission of SOS Publications Ltd.

 

Sound On Sound - Oct 1989

Donated by: Mike Gorman, Bird201

Scanned by: Mike Gorman

>

Should be left alone:


You can send us a note about this article, or let us know of a problem - select the type from the menu above.

(Please include your email address if you want to be contacted regarding your note.)

Topic:

MIDI


Series:

System Exclusive

Part 1 | Part 2 | Part 3 | Part 4 | Part 5 | Part 6 | Part 7 | Part 8 | Part 9 | Part 10 | Part 11 | Part 12


Feature by Martin Russ

Previous article in this issue:

> Hollis Research MIDIman

Next article in this issue:

> Digital Signal Processing


Help Support The Things You Love

mu:zines is the result of thousands of hours of effort, and will require many thousands more going forward to reach our goals of getting all this content online.

If you value this resource, you can support this project - it really helps!

Please Contribute to mu:zines by supplying magazines, scanning or donating funds. Thanks!

We currently are running with a balance of £100+, with total outgoings so far of £1,026.00. More details...
muzines_logo_02

Small Print

Terms of usePrivacy