Groove Pilot

Groove Pilot is a prototype interactive spatial music mixing “experience”.

As part of my continuing series of explorations around spatial music I threw together this prototype to play with some ideas about:

  • Spatial music
  • Real time mixing
  • Interaction

How to play?

  • You must use headphones or earbuds to experience the spatial music
  • Collect all 14 sound orbs to complete the level.
  • As you collect each sound orb it enables another layer of the downtempo chill music track to play.
  • Each layer of music is located in 3D space so is mixed relative to all the others as you move around, dynamically changing the music mix and experience on the fly.
  • In a sense it’s a generative music piece controlled by the players movement through 3D space.
  • Each sound orb is also visualized in different ways with real time signal processing to drive reactive graphics.
  • Collect each sound orb before you run out of fuel.
  • Each one collected will give you a bonus fuel boost as well.

Try it out: https://groovepilot.netlify.com
Note: This is WebGL and best used in Chrome/Firefox with minimal tabs open.

Next step is implement as a full VR app.

Technical details.

Built in Unity 3D.

Desktop only as Unity WebGL is currently not supported on mobiles.

Fopra – Focused Practice

FOcused PRActice (https://fopra.live) is a timing tool to help you practice music (and other things) more effectively. It’s a session timer that lets you split your precious practice time up into three stages: Warmup, Practice, and Perform. You can read more about how to use it on the about page at https://fopra.live/#/about

I’m interested in developing educational tools to help people learn faster and more effectively so this is another small step in that direction.

Please use it and give me some feedback! Note: Best used with Google Chrome browser

Technical Development

This is also a prototype app to test a number of things I’ve been exploring in the rapid web application and PWA (Progressive Web App) development space. PWA’s are single page applications that can work both online and offline. You can also add the app to your mobile device homescreen just like a “normal” mobile app but without the friction of having to deploy it through an appstore as it’s a pure web application. You can find the “Add to Homescreen” option in your mobile browser settings.

Fopra is built with a vue.js based framework called Quasar (http://quasar-framework.org). Having worked with some other reactive javascript frameworks such as Meteor I am really impressed by the quality of Quasar, considering it comes from a one man operation.

Some other things I was also testing with this app were Gitlab’s CI (continuous integration) processes and SPA hosting via Netifly.com, complete with Https & CDN support.

Spatial Music Visualizer

Inside Music is a Google WebVR Experiment that lets you step inside a song, giving you a closer look at how music is made. The bonus is the music is spatialized as well so you get a completely different audio experience from a normal stereo mix.

Open the Song Visualizer in a new tab: https://sonicviz.gitlab.io/sonicviz-spatial-music/
You can move around using the WASD keys and mouse, just like a regular game controller mode.
Note: Best used with Google Chrome browser with no other tabs open.

“Interaction
Select a song from the menu. The stems of the song will appear in a circle around you, each represented by a sphere. In 360 Mode, tap the spheres to turn them on or off. In VR Mode, you can use your controller to toggle their state. On Google Cardboard, you will have a retical (a small circle in front of you eye) which can be used to turn the stems on and off.”

I thought it would be a good opportunity to pull apart and test it with a couple of my own songs:

There’s huge potential with spatial music to revolutionize music production and delivery, and we’re only just getting started. For some more info on this you can read my blog post on “Immersive Audio and Musical AI“.

There’s a bit of a process to go through, including configuring your development workflow and tools but in the end it’s a pretty cool way of getting inside the music. I also used it as an opportunity to test gitlab CI and page hosting.

Next step will be to extend it with some custom visualizations, refine the asset pipeline workflow. I’ve actually had a similar concept bouncing around to do in Unity3D so I’ll probably do that at some point.

See also: https://www.canvas.co.com/creations/3901

Generative Music System

Algorithmic Music with seeded HMM and Stochastic Noise

A demonstration of a prototype generative music system using a variety of techniques from seeded HMM to stochastic noise.

The prototype has two generative music systems:

  • A generative controller that uses a hidden markov model to generate new compositions from a seed music database
  • A random music generator using a variety of algorithms from a windchime emulator to stochastic noise.

The system is built with Java, and uses an open source synth ZynAddSubFX as the sound source.
It was written in 2006  based on research work I did for my Music Masters degree in 2003, and I’m currently porting parts of it to C#/Unity & HTML5/WebAudio.

In 2007 I produced 2 relaxation music albums each with 4 x 15 minute tracks using this system, mixed with ambient environment nature sounds from another generative system. Currently these are offline but I hope to redistribute them again sometime. Here is a track from Album #1:

Generative music systems are a rich field of exploration, and the methods presented here are well known.
I have extended them a little more with some added features such as:

  • Object database containing seed compositions with metadata
  • More parameters for randomization and variability
  • More experimentation with noise generation algorithms to drive music generation

Potential uses of such as system are varied:

  • Affective computing  – detected user emotions to drive system feedback via music mood matching
  • Art and music therapy
  • Music education

Some screen shots are below, followed by a video that briefly explains both systems.

Seeded HMM Generative Music Generator

Seeded HMM Music Generator

Stochastic Random Generative music generator

Stochastic Random music generator

Check out the video for a more in depth explanation.

You can read more about my music research here: music.sonicviz.com

Pin It on Pinterest