The Morph is uniquely suited to building expressive real-time visual instruments, and here’s mine. It uses 4 Morphs and a touchscreen, all mounted in a custom frame. Here’s a video showing the first time I had it all running:
This is an evolution of a previous project (see http://spacepalette.com) that used the Microsoft Kinect to provide 3D mousepads in mid-air. I used TUIO as a protocol between the Kinect and the rest of the software, so all I needed to do after getting the Morph was to create a TUIO server for the Morph, using the C API. I did this, and it’s available in this GitHub repo: https://github.com/nosuchtim/TUIO3D_Sensel .
The touchscreen provides a way to controls the sounds and visuals that you’re playing, as well as providing a preview of the visual output that would normally be displayed on a larger monitor or projected on a screen.
This instrument is both visual and musical, and all three dimensions of input are significant. Visually, you’re fingerpainting - each pad paints a different kind of visual, and the pressure of your fingers controls the size of the graphics. Using pressure in this way is completely natural, expressive, and a lot of fun. Musically, each pad plays notes of a different sound. In some sense it’s like a harp - you’re stroking invisible strings. The horizontal position of your fingers controls the pitch - it’s 2 octaves left-to-right. The vertical position of your fingers controls the quantization of the timing - at the top of each pad you’re playing faster notes, and at the bottom of the pad you’re playing slower notes. The pressure of your fingers is used to control musical aspects such as vibrato, volume, or a filter - depending on which sound is being played.
I use Resolume as a host for visual plugins. I wrote one Freeframe plugin that receives the TUIO and draws moving sprites, and other Freeframe plugins (that come with Resolume) process the visuals in many varied ways. The same Freeframe plugin I wrote also generates MIDI for the musical notes. I use Omnisphere 2 as the soft synth (running inside Plogue Bidule) for all the sounds, except for a few drums done with Battery 4.