This month, Martin Russ looks at a real-world problem and its technical solution. It all starts with a letter from a reader...

I have been experimenting with writing my own System Exclusive Dump program for the Atari ST, using HiSoft Basic 2. When I try to receive a dump I can grab all but the last few bytes — and then I get incorrect bytes. Any ideas?
John Garside, Hitchin

HiSoft's excellent Basic 2 is a good choice for any general-purpose programming task, but for MIDI programs there are a couple of not-so-obvious 'tricks-of-the-trade' which can sometimes help enormously.

The traditional way to grab inputs in the BASIC language is to read them with an INPut statement, and then store them in an array. But MIDI's data rate of 32 kbits per second can sometimes be just too fast for Basic 2 to keep up with if you use arrays, and so you get the 'lost bytes' phenomenon you mentioned. So this gives us two problems: speed and lost bytes.

Imagine that you are a computer program. Bytes of MIDI data arrive at your input and you have to store them in the next drawer in a numbered set. By the time you have worked out which is the next drawer, another byte has arrived, and so you put it to one side whilst you put the first byte in the correct drawer. By the time you have picked up the second byte another has arrived, and so you put that on one side. Whilst you are trying to figure out which drawer the second byte goes in, another byte arrives — so there are now two bytes waiting to be stored. As you can see, it only needs to take slightly longer to put the byte in the drawer than the time between bytes arriving, for you to get in a mess, with bytes queueing up waiting to be stored...

BUFFER

So how many bytes can be queued before things go wrong? It depends on storage — you need to put the bytes somewhere whilst they are waiting. This is called a buffer, and in fact, most input and output on a computer is done by using buffers. The keyboard, serial port and MIDI port are obvious examples, but many computer games use a buffer for the screen display — the program writes to one screen whilst displaying the other, and then swaps them. A real world example of a buffer is a bucket with a leak — a continuous stream of water runs out of the leak, even though you might fill the bucket with large amounts of water every few minutes.

In an ST, there is an area of RAM memory set aside to provide a buffer between the MIDI input port and the Basic 2 program. It defaults to 256 bytes long, which means that for short System Exclusive dumps the time that the program takes to store bytes really does not matter, since the buffer never fills up completely. But if your System Exclusive dumps are so long that the buffer does fill up and overflow, then you will lose bytes, and the System Exclusive data that you store in the array will not be right. Since it takes a while for the buffer to fill up (at least 256 bytes) the problem does not affect the start of dumps. The 'last few bytes' that is mentioned indicates that the problem is probably buffering.

Increasing the size of the MIDI buffer involves finding out where the parameters that set the size and location of the buffer are, and then changing them so that they refer to a new larger buffer. It also helps to keep track of the original buffer, so that it can be restored later. The steps are thus:

• Define a new buffer.
• Find out where it is in memory.
• Find the details of the existing buffer and store them.
• Insert details of your new buffer.

FASTER

Buffers may only be part of the solution. If the part of the program that works with the bytes is too slow, you end up needing a buffer that is as large as the largest dump you will ever encounter, which is very wasteful of memory. Speeding up programs often depends on making changes to the detail of what they are doing. For dealing with MIDI dumps, the obvious thing to do is to define an array, and then fill it with incoming bytes, something like this:

DIM midi(100)
FOR a = 0 TO 100
midi(a)=INP(3)
NEXT a

The problem is that, in practice, loops like this don't seem to run very quickly in BASIC 2. A much faster way is to find out where the array is stored, and then put incoming bytes into it — directly! This is how the example input routine described in the panel on the next page works, and it is taken from a real working program. For most purposes, I have found that using direct POKEing to an array is fast enough to make increasing the buffer size unnecessary. The key to developing MIDI programs is to experiment to test and confirm your techniques as you go along, so don't be afraid to try your own ideas out and see what happens.

You could always rewrite the input routine in 68000 machine code to gain a little more speed, but this goes against the idea of buying BASIC in the first place. Why buy an 'easy-to-program' language, and then replace bits of it with something which is significantly harder to learn? From my experience, many hi-tech musicians would like to dabble with MIDI on their computers, but are not eager to learn C or machine code. Modern structured languages like HiSoft's Basic 2 fit the bill nicely — and with a little ingenuity, can often be persuaded to do things which are not immediately obvious: like capturing SysEx dumps.

PRACTICAL

When I started writing MIDI programs for the ST, there was little help available. You certainly didn't find listings of code in magazines! In this context, this month's Atari Notes is more like 'technical news' than the rumours and comment you normally find. Please let me know if this has been useful; and if you have any hints and tips on MIDI/audio on the Falcon, get in touch.