I've been playing with a jammer, breath controller, foot board, and tilt controller, trying to figure out both what sort of music they're good for and how to combine them with piano/mandolin when playing live. Here's what each can do:

• Jammer: 98 velocity sensitive keys in a Wicki-Hayden hex layout.
• Breath controller: measures breath pressure
• Foot board: effectively two velocity sensitive keys
• Tilt controller: measures the angle of my head
• Piano: 88 velocity sensitive keys in an awkward layout that I have a lot of practice with.
• Mandolin: string instrument, needs two hands

I think I've talked about all the others before, but the tilt controller is a Yocto 3D V2 on a hat:

These can combine in a lot of ways. For example, I can play mandolin with my hands, drums with my feet (kick / hi-hat), and then use the breath controller to play something jawharp like controlling pitch with head tilt.

Let's go through some of the pieces:

The combined instrument has a sense of the current root note. This can be selected either by pressing a pair of keys on the jammer (root select, chosen note) or by pressing a bass key on the piano. The root note isn't used directly, but is an input to several other things.

The root note can then be modified with the head tilt sensor to choose a scale degree. I've been using, looking down at the my head with my nose towards the top of the screen:

major:
   2 6
  4 1 5

mixolydian / dorian / minor:
   6 7
  4 1 5

The 1 is with my head in a natural resting position, 5 involves tilting right slightly, etc.

The scale degree isn't enough on it's own: you also need to know the mode. There are keys for selecting major (no flats), mixolydian (flat seven), dorian (flat three), and minor (flat six).

This combines to give the active bass note. I typically use this in two ways:

• When playing piano, I just use the root note of my piano playing and turn off tilt.
• When playing mandolin or jammer I use the piano or jammer to select a root note and then modify it with tilt.

One place I use the active bass note is when playing bass with my feet. Instead of playing rocking octaves with my left hand on the piano, I can play them with my feet. Left foot plays the active bass note, right note plays that an octave up. I don't generally combine this with piano playing, but it combines well with mandolin and jammer.

The other place I use the active bass note is with kind of a midi jawharp. The active note controls what note to play, which I have set to come out constantly at low volume. Then the breath controller adds additional volume on top of that. I've typically been pulsing this on the offbeat, but there's probably a lot more I could do with it. This can combine with anything.

In addition to playing bass, the feet can play drums. I have four toggles that control the feet:

• Footbass on/off
• Drum low on/off (Kick drum, left heel)
• Drum high on/off (High-hat, right heel)
• Drum special on/off (Snare/Splash, right toe)

With the Footbass turned off and the three drums turned on I can play all the patterns I normally do on the footboard. With the Footbass turned on I can play rocking octaves, and then by toggling the other drums I can add Kick (four on the floor) and Hi-hat (upbeats). The limitation here is that you need to be playing a pattern that makes sense for both bass and drums.

The last piece of this setup is that there are several instruments I can play with the buttons of the jammer, some of which use the breath controller as well:

• Accordion: breath controller for intensity. Button press velocity is ignored, which makes it easier to play quick things because I don't have to worry about how hard my fingers come down. Polyphonic.

• Accordion radio: same as the accordion, but the buttons act as radio buttons. That is, each button stays on until I hit the next one. Good for very notey tunes. Monophonic.

• Sax / Trombone: breath controller for intensity, velocity for attack. Two different brass synths. Monophonic.

• Keyboard: velocity for attack, breath controller ignored. The jammer acts like a regular keyboard. Polyphonic.

• Lead: velocity ignored, breath ignored, radio buttons. Intended for playing notey tunes where my expressive energy is all going into other things. Monophonic

Ideas for future work.

• Pipe the tilt sensor through to the sax and trombone synths to make them more expressive (control growl etc).
• Have the breath controller affect my regular piano playing, maybe adding distortion at high pressure.
• Learn to play jammer as well as I play piano, so that I no longer need a bulky awkward piano. They're capable of all the same things.
• Learn more tunes solid on the jammer so I can play melody better.
• Rewrite the code so it gracefully handles instruments appearing and disappearing instead of needing to restart several things in order.
• Make a simple version of this that doesn't require nearly so much equipment.
• Find a piano sound I really like and start running the piano through the computer as well.
• Figure out a note selector that is more convenient to use while my hands are busy playing mandolin.
• Get more bits of information out that I can use musically. (Bite pressure? Tongue movement? Shoulder movement?)

The code for all this is on GitHub but more as a reference for other people looking to see how I did this than as code I expect anyone else to use. It's very tied to my particular equipment. If anyone is thinking of exploring in directions like this I'm happy to try and help.

More demos: