As technology continues its exponential curve into cyberspace, keeping abreast of developments can be a nightmare. Rest easy as Mixed Media takes you by the hand and tiptoes through the tulip field of sound and vision that is Multimedia

MIDI megalomania

Once MIDI technology is coupled to lightshows or video sequences, it can only be a matter of time before the process is reversed, and music created by light and shadow-play.

The process was pioneered by David James, who linked colour to MIDI note information, and images (or sequences of images) to live MIDI information. He's now refined the software to play MIDI instruments in response to body movements, picked up by a standard video camera.

The image is digitised and decoded for spatial position, then output as MIDI channel and note information to any connected MIDI instrument. All previous colour and image sequencing possibilities continue to apply. The pictures are produced by the MIDI-triggering software, and their flowing colours swirl out of the screen on body position command, while the MIDI note is played. The camera-MIDI system is particularly interesting when played as an 'Air Drum Kit'. The player simply lashes out at empty space, and 'crash!' a snare drum is heard, accompanied by a burst of colours erupting onto a previously dark 15ft by 12ft video screen.

David has produced plans for a total sensory immersion club, with sychronised colour, sound, image and dancer feedback. He has also designed simple and effective 3-D sound-positioning hardware.

Dancers will see themselves projected onto the walls of the club (preferably dome-shaped), and will be able to change the angle of the projected horizons/colour/image size using the MIDI-camera link. They will also be able to affect the tempo of the beat, and apparent sound source position. The sound will be truly three-dimensional (see picture), and capable of creating spiralling cones rising to the ceiling on a crescendo, for example. David feels that the sensory immersion venue could attract a lot of custom, and initial predictions indicate good commercial possibilities.

On a another note, David James has just written a prototype software package, that allows extremely complex waveforms to be mathematically synthesised, sequenced and played back on the Atari STE. The same software concept may be applied to any computer.

Originally, the wave package was written for the Amiga as part of a Phd research in Art and Music. David had been looking for geometric symmetries in chord structures, with a view to comparing them to visual colour waveform triplet structures, in an attempt to find a correlation between colour triplet frequencies and audio chord frequencies. Amongst his findings was that the visual spectrum of red to violet, covers almost exactly a doubling of frequency, making violet the octave of red.

The wavefiles are the cumulative addition of separate waveforms, each with its own highly complex amplitude and pitch shift envelopes. Each component waveform may be given a complex amplitude and pitch envelope, the complexity of which is only limited by computer memory. These hand drawn, on-screen envelopes can easily become complex sound waves in their own right. Standard settings allow 65000 envelope coordinates over six seconds of mathematical sample time. Compare this with six coordinates for TVA on Roland synth units.

The end result can be the production of very intriguing sounds. You may wish to examine the wave structures in a sampler software package, as some of the most interesting sounds are also the most beautiful looking, according to David. Other sampled wavefiles can be customised by the wavesynth software, given extra harmonics, frequency-stretched and generally messed around with.

There is no need for worrying about processor speed limitations for playback, and only allowing certain spans of keyboard to be used, because mathematical frequency transposition allows all notes to be played back at the same processor clock speed. Cloned and transposed wavefiles can then be played happily over the entire keyboard, having been digitally frequency-shifted to any scale you like.

Basically, whatever an analogue synth or hybrid synth can do can be mimicked, with the right mathematical approach and correct software. If anyone could be bothered, they could build up a wavefile comprising thousands of 'mathematical oscillators', with associated complex pitch and amplitude envelopes. This wavefile would be only one of hundreds, each of them assigned to a different MIDI keyboard note. Compare this with the number of individual oscillators in a standard synth.

David sees the day when a CD will contain the seed information for generating endless varieties of synthesised wavefiles, which will then be played according to seed algorithms, which will in turn produce synchronised colours and infinitely varying 3-D landscapes. Built into the seed control algorithms will be adaptive response codes, which will take into consideration human reaction to the sounds and images the system is producing.

The programme will evolve to suit the requirements or whims of its human masters. Say you've just seen and heard a pleasing piece of art and music, you could then signal to the system's eyes or ears (camera or mics) that you wish it to extemporise around the last piece. Total computer-generated art and music evolving intelligently with human guidance!

David uses computer-driven note play algorithms, to test the various MIDI pathways between computers and keyboards. 'Blurps & Mangle', 'Woozle Fever' and "Axiom Reshuffle' are some simple examples of computer-generated wave structures being sequenced and played live. Shirley Skeel

David James has a Bsc in Physics/Electronics, an Msc in Bioengineering, and has spent 3 years researching for a Phd in colour-music links.

He is presently seeking serious financial interest from entrepreneurs/businesses interested in the sensory immersion dance club, recording companies/successful artists interested in producing a new kind of sound to supercede the 'thumping bass beat syndrome', and electronics companies willing to develop proper 3-D sound systems for home and club use.

Anyone interested in the Mathematical Synth Software may also contact him on: (Contact Details)

Humanitarian Aid

Without a doubt the best thing to happen to music of late, aside perhaps from the Mr Blobby record, was the invention of MIDI. A cleverly designed and internationally standardised interface which offers us all the chance to make music as horrific as the pneumatic pink person, flawed only by the fact that a lot of its users don't properly understand it, apparently.

Hoping to lighten our darkness are ex-BBC boffins Robert Treen and Paul Simmonds, who together have launched MIDI For Humans; an educational project of epic multimedia proportions.

The first phase (which makes it sound rather like a building development) is the 'Computer Guide to MIDI'. This software package is already available, and contains a full description of MIDI, its implementation charts, a complete manufacturers directory plus a glossary of all that esoteric terminology. The program is distributed on a shareware basis, with certain features enabled only to registered users.

Phase two involves some more ambitious proposals, including a series of video tutorials, CD-i, CD-V and Internet bulletin boards, all of which are being planned as we speak. In fact, input of a constructive nature (or fiscal for that matter) could very well be quite helpful to them. The whole drive behind RTPS is to produce a series of products all aimed at adding a touch of humanity to the computerised mayhem that is MIDI. Let us hope this will make it possible for the technologically literate and illiterate alike the chance to take a bigger plunge into the pool of electronic music.

RTPS Systems can be contacted at: (Contact Details)