Tangible Music Interface With Candy
If there is one thing I truly like, it’s the mix of music and candy :) In this case the combination leads to The Bubblegum Sequencer, a physical step sequencer that lets you create drumloops by arranging colored balls on a tangible surface. It generates MIDI events and can be used as an input device to control audio hardware and software. The Bubblegum Sequencer senses the position of the balls through a video camera mounted underneath the surface. The captured image is processed by a computer vision routine that computes the average color in each hole. The colors are quantized and mapped to notes. For each note, a MIDI event is generated and sent to the operating system’s MIDI bus.
In addition to the original features shown in the video, we’ve implemented a few more:
- Tempo tapping: Tap three or more times on a pressure-sensitive area on the side of the sequencer to set a new tempo for the playback loop.
- Visual feedback: The sequencer now sports a row of running LEDs to indicate the current position. We’ve also experimented with projecting animations of popping bubbles onto the surface to provide direct feedback which beats are currently played.
- Melody mode: Instead of just playing monophonic beats, we developed a mode in which the vertical position of the balls encodes the pitch of the sample played. To increase the range, two balls can be combined, totalling seven different possible notes on the blues scale.
- Voiceover mode: To make the sample mappings less reliable on computer-based UIs, we wrote a Processing application that lets you record voice samples for each color at playback time. You record by holding down the spacebar and speaking into a microphone.
Finally, people can’t claim anymore that electronic music isn’t handmade.
The Bubblegum Sequencer is a project by Hannes Hesse, Andrew McDiarmid and Rosie Han.
It was conceived and created in the course “Theory and Practice of Tangible User Interfaces” at UC Berkeley’s School of Information in the Fall semester 2007.
For more information, visit the project site