Category: Music and Technology

The Idea….

As a diversion from the industrial IoT projects I’ve been working on, I decided to investigate the wonderful world of wearables and build a system to track violin bow movement.

An attitude and heading reference system is a 3 axis description of an object that transmits the data to a mobile device via bluetooth ble. The data can then be used for generating sound, tracking a person’s movements, falls, etc. I used the Adafruit Flora wearables platform because it seemed well suited for the task at hand which was to gather data on how my violin bow moves about when I play percussively as in Métis fiddle music which I enjoy. I will use the data to build a midi track that produces a percussion accompaniment and will produce a video in the near future to demonstrate.

Skills Needed:

You’ll need some experience at installing Arduino/Flora libraries and some familiarity with sensors and bluetooth. There’s a lot of info out there so you won’t have any difficulty finding it.

Hardware Required:

Adafruit Flora Wearable Board

Adafruit BlueFruit BLE

Adafruit LSM9DS0 Accelerometer, Gyro, Magnetometer

If you don’t purchase their kits, be prepared to acquire some soldering and sewing equipment.

Read about the platform here…

Software

Arduino IDE

Following Flora Libraries: ( follow directions on Adafruit site for installing )

Adafruit Universal Sensor Driver

Adafruit ARHS

Adafruit LSDM9DS0

Bluetooth Apps for receiving data

nRF Toolbox for BLE , Android

Adafruit Bluefruit LE Connect, Android

Adafruit Bluefruit LE Connect, iOS

LightBlue Explorer – Bluetooth Low Energy, iOS

 

There are many more of course on the AppStore and Google Play.

Or you can download the source code for a barebones Android app that I have created that will allow you to customize the way you receive and view the data. You will have to code anything you want to do with the data that isn’t raw bytes.

Down the app source code here !

Alligators and Sensors during Testing

Code Walk Through

After all the components are wired up and communicating we can begin to push some data out to our mobile devices. First we wake up the sensors and bluetooth and repeatedly query the AHRS module that takes the input of the sensors and calculates the Euler angles and returns them to the main loop which bundles them up and transmits to the mobile device. Using one of the above mentioned apps, we can view the data as it arrives. Simple eh ?

The comments in the code provide a good narration of each function.

The source code for this Arduino project can be found on my github site.

Coming Soon: The Performance !!

About the Author

Bob Hale is a mobile and IoT developer and musician who likes to write about it all…

A couple weeks ago I watched a show on PBS that discussed whether mathematics existed in the natural world or if we created mathematics to explain the workings of the universe. Music was used as an example.

I’ve always been fascinated with the mathematical relationships in music. So much so I bought a Commodore Amiga computer and Yamaha DX-7 synthesizer and taught myself C Language programming to create apps that would allow the two computers to communicate. ( Yes, that along ago!) In the world of Data Science we use visualizations to imagine what data relationships might look like, I looked at the the data and said why not listen to it ? This series of articles will explore that process through the wonders of MIDI.

This file assigns the values of the climate data to pitches, not even scratching the surface of what can done with effects and more complex note structures. We’ll leave that for a future article.

To begin, I selected a collection of climate data from the Minnesota Dept. of Natural Resources. The particular sample I selected was from 1/1/2010 to 12/31/2010 and contained date, high temp, low temp and precipitation columns. Below is small sample.

2010-01-01,6,-9,T,T,9,24,24,24,0,0
2010-01-02,1,-15,0.00,0.0,9,24,24,24,0,0
2010-01-03,7,-14,0.00,0.0,9,24,24,24,0,0
2010-01-04,7,-10,0.00,0.0,9,24,24,24,0,0
2010-01-05,10,-9,0.00,0.0,9,24,24,24,0,0
2010-01-06,16,-4,0.00,0.0,9,24,24,24,0,0
2010-01-07,16,4,0.06,2.2,11,24,24,24,0,0
2010-01-08,8,-6,0.00,0.0,11,24,24,24,0,0
2010-01-09,6,-14,0.00,0.0,11,24,24,24,0,0
2010-01-10,15,-6,0.00,0.0,11,24,24,24,0,0
2010-01-11,22,8,T,T,10,24,24,24,0,0
2010-01-12,22,2,T,T,10,24,24,24,0,0

Next I coded a Python script that would allow me to read the data rows and build a MIDI music file as the result. I chose Python because of the quick, cross-platform usage. There are several very robust MIDI libraries available for this task written in Java, .Net and other high level languages. The library I chose is MidiUtil-0.89 mostly because it’s written in Python.

First, we had to adjust some of the data values to account for negative values ( cold days ) as MIDI values are 0-128 and increase the values so the pitches where closer to middle C to avoid extremely low pitches that are barely audible. For expediency I accomplished this with inline code but could also be done in R or in a database.

Here is the entire script with comments #

############################################################################
# A sample program read a data file, generate a midi file
# and write to disk.
############################################################################

#Import the library
from midiutil.MidiFile3 import MIDIFile

import csv

# constant values
channel = 0
channel2 = 1
channel3 = 2
 
track1 = 0
track2 = 1
track3 = 2
time = 0

beats = 540

# indexes to elements of data row
highTemp = 1
lowTemp = 2
precipitation = 3

highTempAdjustment = 20
lowTempAdjustment = 30

# Create the MIDIFile Object with 3 tracks plus names of tracks

MyMIDI = MIDIFile(3)
MyMIDI.addTrackName(track1,time,"Temperature MusicHI")
time = time +1
MyMIDI.addTrackName(track2,time,"Temperature MusicLOW")
time = time +1
MyMIDI.addTrackName(track3,time,"Temperature MusicPrecip")
time = time +1
MyMIDI.addTempo(track1,time, beats)
time = time +1
MyMIDI.addTempo(track2,time, beats)
time = time +1
MyMIDI.addTempo(track3,time, beats)

# set voice (sound) to be played on tracks
# we used General Midi sounds ( see General Midi docs )
time = time +1
MyMIDI.addProgramChange(track1,0, time, 53) # voice 1 = 53
time = time +1
MyMIDI.addProgramChange(track2,1, time, 53) # voice 2 = 53
time = time +1
MyMIDI.addProgramChange(track3,2, time, 119) # cymbal = 119

time = time +1

# open and read each line ( data object ) in file
f = open("climate2010.txt")
for row in csv.reader(f):
 
 # calculate pitch value from temperatures
 pitch1 = int(row[highTemp]) + highTempAdjustment
 pitch2 = int(row[lowTemp]) + lowTempAdjustment
 duration = .5
 volume = 100 
 
 # add initial tracks
 # Add a note. addNote expects the following information:
 MyMIDI.addNote(track1,channel,pitch1,time,duration,volume)
 time = time +1
 MyMIDI.addNote(track2,channel2,pitch2,time,duration,volume)
 time = time + 1
 if row[precipitation] != "0.00": #got some rain today
 pitch3 = 96
 MyMIDI.addNote(track3,channel3,pitch3,time,3,90) 
 #print(row[1])


time = time + 4

# change track 3 to ocean sound for the finale !!

MyMIDI.addProgramChange(track3,2, time, 122) # 122 = Seashore
time = time + 1
MyMIDI.addNote(track3,channel3,40,time,45,100) # let it ring....


# And write it to disk.
binfile = open("climatetemp2010_TempsPrecip.mid", 'wb')
MyMIDI.writeFile(binfile)
binfile.close()

And the resulting file ! Not much to look at – unless you are a real hex-head – just a series of hex values. However with a tool such as Midi Microscope you can view all the events in both hex and carbon-based annotation. A necessity for those in the trade! In an upcoming article I’ll cover some of the tools I use to produce sounds, create MP3 files and edit the raw data.

4D 54 68 64 00 00 00 06 00 01 00 03 03 C0 4D 54
72 6B 00 00 0E 69 00 FF 03 13 54 65 6D 70 65 72
61 74 75 72 65 20 4D 75 73 69 63 48 49 96 40 FF
51 03 01 B2 07 96 40 C0 35 96 40 90 1A 64 83 60
80 1A 64 8B 20 90 15 64 83 60 80 15 64 8B 20 90
1B 64 83 60 80 1B 64 8B 20 90 1B 64 83 60 80 1B
64 8B 20 90 1E 64 83 60 80 1E 64 8B 20 90 24 64
83 60 80 24 64 8B 20 90 24 64 83 60 80 24 64 8B
20 90 1C 64 83 60 80 1C 64 8B 20 90 1A 64 83 60

After this file has been created I played the midi file through a VirtualMidiSynthesizer  so the hex data in the file is ‘voiced’ by the settings embedded in the file. I ran the output through another piece of software which captured and saved the output as a MP3 file. Audacity is another open source product I use for this purpose. There is quite a bit of setup required to use all these tools so I’ll cover them in another post.

The following MP3 file is the result played back through a digital software synthesizer. The voices are the high and low temps chasing each other about getting higher and more frenetic in summer ( the middle of cut ). The cymbal sounds for each day there was precipitation. Interesting to hear the lack of rain during summer and autumn. So sit back and enjoy the weather ! If you listen to this using Windows Media Player, turn on the visualization options for an added treat.

The code and data files used in this project are available on GitHub.

 

 

 

 

 

 

 

Searching for data sets that might work as Sonifications  I discovered wind data for the Minneapolis MN area from the year 2015. I selected the wind direction and velocity fields and did a quick and dirty test which resulted in a blast of too much noise. Since one of my objectives is to let the data sound I was hesitant to edit the data however I came up with an acceptable solution. I used a relational database to load the data and label the direction fields according to their ‘general direction’. The quadrants below illustrate the labeling process.

Now with only four directions I set out to see what would happen. The results sounded to me like a electronic windchime or a Gamelan orchestra.

The data was obtained from  NOAA who collects vast amounts of data on climate and weather.  There is an important distinction between weather and climate that is often used to muddy the waters in discussion of climate change. Generally speaking weather are the events that take place in a climate. I’ll leave it there.

I decided to move the sound files to SoundCloud for ease of distrubution. You can hear my ‘songs’ there if you don’t want to read the stories.

Thanks for listening !  Contact me @ bobhale at CityWorksApps dot com.

 

 

Quite a few years ago,I lived outdoors for a summer at the base of Mt. Evans in Colorado. Each night I slept in the moonlight, feeling a little lost during the new Moon when it wasn’t visible. As our relationship grew over the years, I became aware of other things happening within the Synodic period of the Moon; one of them being the amount rain or snow that fell during the new and full Moon. In my research I found that it is true; there is a relationship between the lunar cycle and precipitation. No surprise really, considering the way the Moon affects the tides and other earthly events. I grabbed a year’s worth of lunar cycle data and matched it by date to precipitation data for the same period. You can get it here US Naval Observatory

Since lunar data is cyclic, one way to visualize it was a sine wave, so I assigned the pitches to match the peaks, valleys and transitions. The first sounds resembled Beethoven’s Moonlight Sonata – so cool !!. Next I added precip data which did indeed cluster about the new and full moon. Some editing of the data was necessary in order to make it play a little less ragged due to the differences in the moon movement and our date system. In electronic music we call this Quantization, the process of making sure the notes hit on the beats. This is especially important because at some point I intend to perform this piece accompanied by yours truly on the electric violin. If not for Quantization, it would be like playing with drummer possessing an irregular sense of time.

The lunar cycle is represented by the lower piano arpeggios and the koto plays the rain ( or snow ). Made a few more adjustments for aesthetics sake. The Naval Observatory also collects data on the movements of the planets and stars. My mind reels with the possibilities of the data. It could be the real Music of the Spheres !

To add inspiration, I listened to The Planets by Gustav Holst as I worked.