Last month Jay Chapman looked at the numerical representation of musical information and the MIDI hardware which moves such information around for us. We can now start to look at the MIDI protocol which defines the actual messages that can be sent round our MIDI network.

This month we will develop and discuss a simple protocol of our own just to show that there is no black magic involved in MIDI! This illustrative protocol will be for a very simple synthesizer system comprised of one master keyboard controller (which has a pitch wheel) and two MIDI sound expander units. Having developed some ideas for this simple system we should then have gained an insight into the thought processes of the MIDI protocol designers and will hopefully be better able to discuss MIDI in detail. Our example protocol will give us a basis for comparison with MIDI and will also help bring out points worthy of discussion where MIDI protocol develops in a different direction.

The sort of information that must be included in some of the messages that are defined in any MIDI-type protocol can be quite obvious to the musician given a moment or two of thought. For example, if a slave instrument (an expander, say) is to play exactly what is being played at the keyboard on some master instrument (a keyboard controller, say), it is obvious that information about which keys are being pressed must be transmitted (via MIDI), received and acted upon. The assignment of a number for each key is reasonably obvious (see last month's article) and so is the fact that we need to say something about when each note starts and stops.

Since we intend to send other information as well as the key pressed information, we need to include some means of identifying which type of message is which. With two possible slave instruments (the expanders) in our simple MIDI system, we need to be able to say whether a message is intended for one or the other (or both) of the slaves since they will be connected in the daisy-chain fashion described last month and will therefore hear all messages and not just their own.

CREATING OUR OWN PROTOCOL

Assume that our master keyboard and slave expanders can only deal with two octaves of pitch information - from middle C to the B note one octave above (shown as B'). We shall number these notes from 0 to 23 as shown in Table 1. In binary form (you have read last month's article I hope!?) these numbers are represented in 5 bits as the patterns shown in the left hand column of Table 1 - the % symbol indicates binary for clarity's sake.

Not all the possible patterns of the bits have been used since 5 bits can represent (2 to the power 5 =) 32 states. The patterns %11000 to %11111 (decimal 24 to 31) are not used in our example. The minimum number of bits required to represent 2 octaves (24 semitones) is 5, because with one bit less we could only represent (2 to the power 4 =) 16 different states and therefore only 16 semitones.

If we bought another synth that had a two octave plus one note keyboard, running from C to C", we could extend our code by using our first so far unused binary pattern, %11000 (decimal 24), to represent the extra key (C"). However, we could not easily extend our code to deal with a three octave keyboard (C to B") since we would need to represent 36 states, made up of the 12 semitones in each of the three octaves, which would then require a minimum of 6 bits giving (2 to the power 6 =) 64 states. If we settle on 5 bits as being sufficient (no music requiring more than 32 semitones) then we have 3 bits so far unused in the byte. Finding an unused bit to make our original 5 up to 6 (should it become necessary) will be difficult once our protocol has been accepted around the world, so let's hope we get it right!

When a key is pressed on our master keyboard we will assume that the key velocity is sensed and that there are only seven different levels of key velocity discernible which we will number from 1 to 7. We can represent seven states in 3 bits using the binary values %001 to %111. The pattern %000 we will reserve for a special use later on. Our velocity values will effectively represent how hard the key was struck with % 001 meaning very soft, % 111 meaning very hard and %100 (which is in the middle of our number range) meaning medium hard as shown in Table 2. For simplicity, I am assuming that the loudness we hear is directly related to the numbers the keyboard unit gives out and the expander expects to receive.

Knowing that we will be transmitting all our information in byte-sized chunks made up of 8 bits, we can see that both the key number (5 bits) and the velocity with which it was played (3 bits) can fit neatly into one byte - very good for both storage efficiency and reducing the number of bytes to be transmitted!

If we agree to a format where the 3 velocity bits always precede the 5 key number bits (this format definition is another part of our protocol), then the master keyboard must create bytes and the expanders must understand and act upon bytes such as %10001100, %11100000 and %01100011 by dismembering them according to our protocol as follows:

%10001100 = %10001100 = velocity 4 note 12 = medium hard C'

%11100000 = %11100000 = velocity 7 note 0 = very hard C

%01100011 = %01100011 = velocity 3 note 3 = moderately soft D#/Eb

MESSAGE TYPES

This is all fair enough providing the receiving ends of our network know that these are supposed to be key pressed messages - so far we have not defined how to tell one type of message byte from another! So let's do that.

There is obviously no room left in the key pressed bytes to tell anybody anything else so we will send a byte down the line before the key pressed byte to specify what type of byte will follow. In our very simple system we will assume that there are only four different types of message:

Key pressed we have already dealt with and key released is exactly the same as pressed in terms of the format of following (data) bytes. However, the interpretation of the two groups of bits differs in that now the first 3 bits tell us the key release velocity whilst the last 5 bits still define a key number.

We will define pitch bend as follows. If the pitch wheel moves, we send out a message indicating its new position. We never transmit anything if the pitch wheel is stationary otherwise we will be sending out redundant garbage all the time and never get any useful work done! The pitch wheel's position is seen as one of 256 steps evenly spread out over its possible range of movement. If we define increasing pitch to be represented by increasing numeric value, then a sprung centre return-type pitch wheel would be at position 128 when left to its own devices, at position 0 when pulled right back (lowering the pitch) and at position 255 when pushed right forward (raising the pitch).

Program change will have a byte following it indicating the new program number which will be in the range 0 to 255 for convenience. I'll digress on MIDI's use of the words 'program' and 'voice' in a later article!

At this point, we've still got 6 bits unused in our message type byte but two of those can be used up very easily in deciding which of our two MIDI expanders will take note of the current message type. Each of the two bits refers to one of the expanders and each expander will need to know which bit is its bit, perhaps by setting a switch on its front panel. If an expander's bit is a 1 in the message type byte then the expander should respond to the message whereas if the bit is a 0 then the message should be ignored. In other words, the which expander? bits are interpreted as follows:

The first state (%00), where neither expander is supposed to respond, is probably not going to be used too much! Oh, I don't know though - see later...

We will assume that the which expander? bits sit just to the left of the message type bits and that the most significant (left-most) 4 bits of this byte are all zeros. Some example message type bytes might be %00001100, %00000110 and %00001011 which we decypher as meaning:

%00001100 = %11 00 = expander 1 AND expander 2; key pressed

%00000110 = %01 10 = expander 2 ONLY; pitch bend

%00001011 = %10 11 = expander 1 ONLY; program change

We have now reached the point where we can consider some real message passing situations.

USING OUR PROTOCOL

Imagine that we sit down at our master MIDI keyboard, use it to change the program number on both expander synths to the same value, play a chord meant for expander 1 and a bass note for expander 2 (I'm assuming a keyboard split facility!) and then do pitch bend - which affects both expanders - before releasing all the notes held.

Using our protocol, this is what the master keyboard transmits:

(Both expanders have now stopped playing. Note that they are both still well and truly 'pitch bent' since we have not sent them a new position for the pitch wheel, so they will think that it is full forward until further notice.)

So, we have sent off 20 bytes, all conforming to our very own protocol, and the expanders have coped manfully, coughed forth and pitch bent a C minor chord. We will now consider some problems that our protocol has and how it could be improved!

IMPROVING EFFICIENCY

Let's have a look at how efficient our protocol is. We (will often) have a lot of key pressed messages for the same expander (the messages above starting with %00001000 for example) and it would seem a little wasteful to keep sending out a new %00001000 for each key pressed! If we think about it, most of the messages sent over a MIDI keyboard network are going to be involved with keys being pressed and released. If we only sent a message type byte when something other than key movement is involved we would save sending quite a few redundant bytes, wouldn't we?

In the protocol example above, we would still have to send the message type byte for the first key press, or if the expander addressed changes, or if we changed from key presses to key releases. Even so, we would still save sending out four bytes, namely bytes 5, 7, 17 and 19. The idea we are discussing is known in MIDI terminology as running status and is obviously very useful! There is a further twist worth investigating since we would save even more bytes if we didn't have to change between key pressed and key released messages which are going to be by far the most common events our protocol must handle.

Since the velocity upon key release is not used by most current synthesizers, and therefore won't be missed (but if it is needed we just use the full two byte key released message), we just need to know which key is being released and the fact that it is a release rather than a press that has taken place. Now, the key pressed message format already holds the key number so if we could somehow mark certain key pressed messages as actually being key released messages we wouldn't have to change the 'running status' half as often! This is why the velocity bits = %000 was reserved earlier in the article. If a key pressed message comes in with velocity set to %000 it is interpreted as a special key released message!

No further bytes will be saved in our protocol example above by using the special key released message since the key pressed and key released messages are separated by a change of status anyway (for the pitch bend). Imagine that you are playing the same four note chord repeatedly however. Each release and replay of the four notes would have taken 16 bytes (4 key pressed messages followed by 4 key released messages - at 2 bytes each) with our original scheme, 10 bytes with running status or 9 bytes with key pressed coping with both press and release. Eight bars of the same with four 4-note chords per bar would use 512, 320, or 257 bytes respectively. When you come to store this lot in your sequencer the ideas we have just laboriously waded through suddenly become very worthwhile - you can store a lot more song for your money!

PRIDE COMES BEFORE A FALL

Having now patted ourselves on the back for our extreme cunning in saving so many bytes, we now learn the meaning of the phrase 'pride comes before a fall'. Consider the following byte stream:

%00001000 [%0000 10 00] expander 1 ONLY; key pressed

%10001000 [% 100 01000] key number 8 pressed (medium velocity)

%00001000 [% 000 01000] key number 8 released (special key pressed)

          [%0000 1000] expander 1 ONLY; key pressed

% 10001001 [% 100 01001] key number 9 pressed (medium velocity)

I have managed to apply our fledgling protocol to the third byte in two different ways - which one is the right interpretation? If we assume that 'running status' is in force then presumably the first interpretation is correct - or is it? We had better be sure of what we mean since in one case we end up with only note 9 playing and in the other case both notes 8 and 9. Apart from the musical difference this makes, it may well be that note 8 doesn't get turned off anywhere else and therefore drones on forever!

With our pride already in tatters, I'm afraid things are going to get worse before they get better! If we are to use 'running status', we must be able to tell when the status (or message type) has changed. Also, if our system becomes more complex it may well be that we need to send messages consisting of 3 bytes in which case we will need to know if a given byte is a message type byte or the first or second of the two data bytes. Consider the following byte:

%00000111

There is absolutely no way of telling what this byte means just by looking at it! It could be a message type byte [% 0000 01 11 pitch bend] or possibly a special key pressed byte [%000 00111 note 7 Off]. In other words, although it could well represent a status change, which is very important, we cannot be sure... oh dear! Even if we thought we knew where we were in a byte stream we could not be absolutely sure. And if a few bits were corrupted during transmission we would never be able to work out what anything meant again! (In fact, the true MIDI protocol has real trouble if data gets corrupted even though it can recover from getting lost, after a while, in most cases.)

EXPANDER 3?

Our next problem is that when we buy expander number 3 for our MIDI set-up we have no way of sending it messages since we have only included 2 bits (the which expander? bits) in our message type format. We have a number of choices here.

Firstly, we could instruct expander number 3 that if both of the which expander? bits are off, then the message in question is meant for it. We can get away with this because we know that each of the other expanders will ignore a message with their bit turned off. Secondly, we could decide to change the meaning of those two bits completely and read them as the (binary) expander number allowing us to incorporate up to four expanders numbered 0 (%00), 1 (%01), 2 (%10) and 3 (%11). Thirdly, we could use up one of the four spare (most significant) bits in the message type byte to specify expander number 3 explicitly.

We have now shown up possibly the only advantage that our example protocol has over the real MIDI protocol, namely, we can change it to suit ourselves whenever we like. When you have thousands of (paying) users of your equipment, you think VERY carefully before changing anything at all!

WHAT NEXT?

I've now taken you through the creation of a simple (and thus not too useful) protocol that defines communications in a very simple MIDI synthesizer system. The reason for this is that although it would have been very easy to just copy the MIDI protocol, word for word, from the official MIDI Specification, you wouldn't have gained any feel for why MIDI works the way it does and where it's coming from (street-wise music phrase dictionary page 11).

It should be considerably easier, and certainly less confusing, to understand the MIDI protocol having now seen how such a protocol could be constructed. Having already worked with the ideas of multi-byte messages and bit group format specifications in this article, you will be able to concentrate on understanding the usage of the byte streams defined in the MIDI protocol. See you next month.