Here some example Arduino code for sending and receiving OSC via the cheap ESP8266 serial WiFi module.
Note that the Open Sound Control messages here are very basic - only 4 bytes packed into a single 32bit integer.
upload the code below to an Arduino.
connect ESP8266 TX pin to Arduino pin0.
connect ESP8299 RX to Arduino pin1. It is safest to use a 3V3 lever converter for this line (or at least a voltage divider).
power the ESP8266 (VCC and GND) from an external 3V source. Do not use the Arduino 3V3 pin as it cannot provide the required current. I used an LF33CV voltage regulator to get 3.3V from the 5V supply that also powers the Arduino.
connect ESP8288 RESET pin to Arduino pin4.
and last, connect ESP8266 CH_PD to 3V3
Optional: connect a separate USB-Serial (FTDI) chip to Arduino pins 2 and 3 to use software serial debugging. Start debugging in a terminal with something like screen /dev/tty.usbserial-A4015TKA 115200
The Arduino code sits and waits for an incoming OSC message (/tap). It then replies by sending out a counter-message (/sti).
//f0 150705
//sending and receiving UDP OSC with an ESP8266
//for an Arduino + ESP8266 with firmware 0.9.5.2
#include <SoftwareSerial.h>
#define WLAN_SSID "SSID"
#define WLAN_PASS "PASS"
#define WLAN_ADDR "192.168.1.51" //laptop running sc
#define PORT 1112 //incoming OSC port
String tag = "/tap"; //incoming OSC addy
SoftwareSerial mySerial(2, 3);
uint8_t buf[16];
byte cnt;
byte id, on, hi, lo;
boolean resp;
void setup() {
//--OSC message
buf[0] = 47; // /
buf[1] = 115; // s
buf[2] = 116; // t
buf[3] = 105; // i
buf[4] = 0;
buf[5] = 0;
buf[6] = 0;
buf[7] = 0;
buf[8] = 44; // ,
buf[9] = 105; // i
buf[10] = 0;
buf[11] = 0;
buf[12] = 4; // a high (id)
buf[13] = 3; // a low (on)
buf[14] = 2; // b high (hi)
buf[15] = 0; // b low (lo)
mySerial.begin(115200);
Serial.begin(115200);
Serial.setTimeout(10000);
mySerial.println("");
mySerial.println("starting");
mySerial.print("hard reset...");
digitalWrite(4, 0);
pinMode(4, OUTPUT);
delay(10);
pinMode(4, INPUT);
resp = Serial.find("ready\r\n");
mySerial.println(resp);
mySerial.print("mode1...");
Serial.println("AT+CWMODE=1");
resp = Serial.find("OK\r\n");
mySerial.println(resp);
mySerial.print("connecting...");
do {
Serial.print("AT+CWJAP=\"");
Serial.print(WLAN_SSID);
Serial.print("\",\"");
Serial.print(WLAN_PASS);
Serial.println("\"");
resp = Serial.find("OK\r\n");
mySerial.println(resp);
} while (!resp);
mySerial.print("mux1...");
Serial.println("AT+CIPMUX=1");
resp = Serial.find("OK\r\n");
mySerial.println(resp);
mySerial.print("udp...");
Serial.print("AT+CIPSTART=4,\"UDP\",\"");
Serial.print(WLAN_ADDR);
Serial.print("\",57120,");
Serial.print(PORT);
Serial.println(",0");
resp = Serial.find("OK\r\n");
mySerial.println(resp);
Serial.setTimeout(1000);
}
void loop() {
while (Serial.available()) {
String abc = Serial.readStringUntil('\n');
if (abc.startsWith("+IPD,4,16:" + tag)) {
id = abc[22];
on = abc[23];
hi = abc[24];
lo = abc[25];
mySerial.print("id:");
mySerial.println(id);
mySerial.print("on:");
mySerial.println(on);
mySerial.print("hi:");
mySerial.println(hi);
mySerial.print("lo:");
mySerial.println(lo);
buf[15] = cnt++;
Serial.println("AT+CIPSEND=4,16");
Serial.find(">");
Serial.write(buf, sizeof(buf));
resp = Serial.find("OK\r\n");
mySerial.print("send...");
mySerial.println(resp);
}
}
}
I modified the power supply to only give out 12V (yellow&black cables) and also made it start up automatically by shorting the green cable (PS-ON) to ground (black).
There's no volume control so better take care - the system is very hot.
Here is a quick tutorial on how to install and run Pure Data headless on a Raspberry Pi. The instructions assume you want to start with a new clean Raspbian system image and do it all from scratch.
The instructions also assume you have a Raspberry model B, a USB soundcard like Terratec's Aureon Dual USB and an ethernet cable with internet access (just connect the ethernet cable between your RPi and your home router).
What you get after following the below tutorial is an SD card with a Pd patch that automatically starts when the RPi is booted.
Preparations
Put the Raspbian image onto a +4GB SD card (it is easily done with balenaEtcher).
On newer versions of Rasbian, activate SSH by creating an empty file called 'ssh' directly on the SD card
Insert the SD card+ethernet+usbsoundcard and power up the RPi
Open the terminal application on your laptop and type...
ssh pi@raspberrypi.local #password is 'raspberry'
See notes below if fail.
Setup
When successfully logged in run this on the RPi...
sudo raspi-config
and do the following system configurations...
Select expand filesystem (only needed on Wheezy and older versions of Raspbian)
Change user password
Optionally lower the GPU memory under advanced / memory split
Select finish and reboot
Log in again from laptop with your new password...
ssh pi@raspberrypi.local
sudo apt-get update #check for new updates
sudo apt-get upgrade #update any old packages
sudo apt-get dist-upgrade #update the distribution
Test sound
lsusb #should list the USB soundcard
aplay -l #should also list the soundcard
sudo speaker-test -t sine -c 2 -Ddefault:CARD=Device #should sound if headphones connected. Stop with ctrl+c
Note: this assumes that your USB soundcard name is Device - check what the aplay command posts and edit the CARD= in the line above if needed.
Install Pd
Download and install Pure Data + required packages with...
sudo apt-get install puredata
Test Pd patches
Copy the following two example Pd patches (or download the attachments below) and save them on your laptop (here assume on the desktop). To copy Pd patches just paste the cryptic text into a plain text editor and save with .pd file extension.
This is also how you can transfer more Pd patches later on.
Run Pure Data
ssh pi@raspberrypi.local #log in from laptop again
pd -stderr -nogui -verbose -audiodev 4 testsines.pd #stop with ctrl+c
pd -stderr -nogui -verbose -audiodev 4 testmic.pd #stop with ctrl+c
Note: if no sound test with different audiodev - 4 is usually the USB soundcard.
You will also need to connect headphones or speakers for the first example to work. And some kind of audio input (e.g. electret mic or line-in from an MP3 player) for the second example (testmic) patch to work.
Autostart
Here is how to start Pd and run a patch at boot...
nano autostart.sh #creates a new file. Copy the two lines below into this new file. (save and exit with ctrl+o, return, ctrl+x)
#!/bin/bash
pd -nogui -audiodev 4 /home/pi/testsines.pd
chmod +x autostart.sh #make the autostart.sh file executable
crontab -e #and add at the end... (again save and exit with ctrl+o, return, ctrl+x)
@reboot /bin/bash /home/pi/autostart.sh
After rebooting (with the sudo reboot command) the sine tones patch should have started automatically.
Stopping
ssh pi@raspberrypi.local #log in from laptop once more
sudo pkill pd #stop Pd
sudo halt -p #turn off the RPi safely
Notes
If you cannot log in and you get ssh: Could not resolve hostname raspberrypi.local, you might need to replace raspberrypi.local with the exact IP address of the RPi (e.g. ssh pi@192.168.1.51). The exact address will vary and can be found in your router setup.
Note: if you get WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! then run the command ssh-keygen -R raspberrypi.local to reset the SSH key.
When ready with everything and you have the correct Pd patch autostarting you can (on the older RPi models with a full-sized SD card) physically lock the SD card. This will put it in no-write mode and possibly prolong its life (especially if you cut the power without properly turning off the system with sudo halt
If you experience audio dropouts you might try the suggestions here. Most important force USB1.1 and set CPU governor to performance mode.
If you get ALSA output error Device or resource busy when trying to start Pd, then delay the ;pd dsp 1 message in your Pd patch with about 100 milliseconds.
To remove the autostart just delete the file autostart.sh and go into cron again and remove the last line you added with crontab -e
Updates:
160109: also works great on a Raspberry Pi 2 with 2015-11-21-raspbian-jessie.img
180102: updated for 2017-11-29-raspbian-stretch.img and 2017-11-29-raspbian-stretch-lite.img
Just cleaned up an example for SuperCollider and Arduino that I found on my computer. It is demonstrating the SCFirmata class by Eirik Arthur Blekesaune.
//how to read pin A0 with SCFirmata...
//for Arduino1.0.6 and SC3.6.6
//first in Arduino IDE:
// * select File / Examples / Firmata / StandardFirmata
// * upload this example to an Arduino
//then in SC install the SCFirmata classes
// * download zip file https://github.com/blacksound/SCFirmata
// * extract files and put them in your SC application support directory
// * recompile SC
SerialPort.devices
d= SerialPort.devices[0]; // or d= "/dev/tty.usbserial-A1001NeZ" - edit number (or string) to match your Arduino
f= FirmataDevice(d);//if it works it should post 'Protocol version: 2.3' after a few seconds
s.boot
Ndef(\snd, {|freq= 400, amp= 0.5| SinOsc.ar([freq, freq+4].lag(0.08), 0, amp.lag(0.08)).tanh}).play;
f.reportAnalogPin(0, true) //start reading A0
f.analogPinAction= ({|num, val| [num, val].postln; Ndef(\snd).set(\freq, val.linexp(0, 1023, 400, 800))})//control freq
f.analogPinAction= ({|num, val| [num, val].postln; Ndef(\snd).set(\amp, val.linexp(0, 1023, 0.001, 1))})//control amp instead
Ndef(\snd).stop
f.reportAnalogPin(0, false) //stop reading A0
f.end
f.close
Compared to generating a serial bitstream in audio, analysing and extract serial data from audio is much harder. The SuperCollider code below does it, but the program has limitations and is quite sensitive for noise.
The code takes a string, chops it up into groups of six 8bit bytes and generates a serial audio bitstream from that. Another part listens to this sound and tries to decode it. If it finds six full bytes it sends the result back to sclang via OSC where it is printed.
To test the example connect an audio cable directly from your computer's output to its input (preferably via a small mixer), or change the audioSerial SynthDef to use an internal audio bus. I can also imagine it could function with a mic next to the speakers - but I didn't test this.
If it only prints gibberish try with a different threshold setting, different volume on your computer or use a lower baud rate.
(
s.waitForBoot{
var baudrate= 9600;
SynthDef(\serialAudio, {|out= 0, amp= -0.5| //for sending out serial via audio
var data= Control.names([\data]).kr(Array.fill(60, 0)); //max 6 bytes
var src= Duty.ar(1/baudrate, 0, Dseq(data), 2);
OffsetOut.ar(out, src*amp);
}).add;
SynthDef(\audioSerial, {|in= 0, thresh= 0.05| //for receiving serial via audio
var raw= 0-SoundIn.ar(in); //here change to In.ar if trying internal audio bus
var src= raw>thresh;
var reading= DelayC.ar(Trig1.ar(src, 1/baudrate*9), 1/baudrate/2, 1/baudrate/2);
var osc= Phasor.ar(reading, baudrate/SampleRate.ir);
var clock= (osc-Delay1.ar(osc))<0+Impulse.ar(0);
var index= PulseCount.ar(clock, reading);
var stopTrig= index>7;
var data= Latch.ar(src, index>=#[7, 6, 5, 4, 3, 2, 1]);
var byte= (1-data).sum{|x, i| 2**(6-i)*x};
SendReply.ar(stopTrig, '/data', byte);
DC.ar(0);
}).add;
OSCFunc({|msg| msg[3].asInteger.asAscii.post}, '/data');
s.sync;
Synth(\audioSerial);
};
)
(
var str= "hello supercollider!";
var baudrate= 9600;
fork{
1.wait;
str.ascii.clump(6).do{|bytes|
var data= bytes.collect{|x| [1]++(1-x.asBinaryDigits.reverse)++[0]}.flat;
s.bind{
Synth(\serialAudio, [\data, data]);
};
(1/baudrate*60+0.005).wait;
};
};
)
One can use this technique to communicate with another computer via audio. To communicate with a microcontroller (e.g. an Arduino), one needs additional electronics (amplification, rectification). Here's schematic for a bi-directional circuit for talking to a 5V Arduino.
This audio-to-serial technique was used to get input from RFID, touch and bend sensors in our Reflect installation i.e. SuperCollider is running on an iPod Touch and receives all sensor data via audio from an ATmega168 microcontroller.
Another way (compared to FSK in my previous blog entry) of sending data via audio is to directly generate the serial bit stream using SuperCollider.
To test and learn about these things I first wrote and uploaded a very simple program to an Arduino board. The program just transmitted the bytes 128, 10, 20, 30, 40 and 50.
Then I connected the Arduino serial TX pin (pin1) to the audio line-in of my laptop (via a 1K + 10K voltage divider) and recorded the sound of the serial transmission.
I then analysed the sound by hand and wrote a little program in SuperCollider that could generate similar waveforms.
s.boot;
o= {|chr| [1]++(1-chr.asBinaryDigits.reverse)++[0]};
(
SynthDef(\serialAudio, {|amp= -0.5| //for sending out serial via audio
var data= Control.names([\data]).kr(Array.fill(60, 0)); //max 6 bytes
var src= Duty.ar(1/9600, 0, Dseq(data), 2); //baudrate
OffsetOut.ar(1, src*amp);
}).add;
)
Synth(\serialAudio, [\data, [128, 10, 20, 30, 40, 50].collect{|c| o.value(c)}.flat, \amp, -0.5]);
This screenshot show the signal recorded from the Arduino in the first channel, and the SuperCollider generated one in the second.
After all this, I could reverse the process, generate any serial data and send it back to the Arduino RX pin (pin0). A small amplifier circuit in between helped to get a more stable communication going.
This serial-to-audio technique was used to control the 24 LEDs (6 PWM channels) in our Reflect installation i.e. SuperCollider is running on an iPod Touch and sends out serial audio to an ATmega168 microcontroller.
Here is another example that can fade a single led by sending serial commands over audio. Includes schematics for an amplifier circuit plus SuperCollider and Pure Data example code.
For a few projects in the past, I had to communicate data bidirectionally via sound. It involved, for example, hooking up a microcontroller to an iPod Touch (running SuperCollider) so that the device could read sensors and/or control LEDs. One successful method was to do Frequency-shift keying. See /f0blog/soft-modem/ for another blog entry on that.
I've published FSK Pure Data code for that before, and also some SuperCollider code for generating/encoding FSK, but never the decoding part I think.
So below is some old code I've cleaned up a bit. It just demonstrates sending audio via internal SuperCollider busses at a very low baud rate. For good baud rate and min/max frequency settings see code.google.com/p/arms22/source/browse/trunk/SoftModem/SoftModem.h.
(
s.latency= 0.05;
s.waitForBoot{
SynthDef(\redFSKdecode, {|in= 20, thresh= 0.1, baudrate= 126, lo= 882, hi= 1764|
var sig= InFeedback.ar(in, 1);
var sigActive= Amplitude.ar(sig, 0.01, 0.01)>thresh;
var fre= 1/Timer.ar(sig*sigActive);
var trg= Schmidt.ar(fre, lo+(baudrate*0.5), hi-(baudrate*0.5)); //0= lo, 1= hi
var trgLo= trg<0.5;
var trgHi= 1-trgLo;
var trgCarr= Sweep.ar(trgHi, 1)>(1/baudrate*16); //when found more than 16bits high in a row
var trgData= SetResetFF.ar(trgLo*trgCarr, trgCarr);
var imp= Phasor.ar(trgData, baudrate/SampleRate.ir);
var writeTrg= (imp-Delay1.ar(imp))<0+Impulse.ar(0);
var writePos= PulseCount.ar(writeTrg, trgData);
var writeVal= Median.ar(31, trg);
var ok, done= 1-trgData;
var buf= LocalBuf(13).clear;
Demand.ar(writeTrg, 0, Dbufwr(writeVal, buf, writePos, 0));
ok= done*(1-Demand.ar(done, 0, Dbufrd(buf, 1, 0)))*Demand.ar(done, 0, Dbufrd(buf, 11, 0));
SendReply.ar(ok, '/data', Demand.ar(ok, 0, Dbufrd(buf, (2..9), 0)));
DC.ar(0);
}).add;
SynthDef(\redFSKencode, {|out= 0, amp= 0.6, minFreq= 4900, maxFreq= 7350, invBaudrate= 0.008|
var data= Control.names([\data]).ir(Array.fill(8, 0));
var env= EnvGen.ar(Env(#[1, 1, 0], [invBaudrate*(16+11), 0]), doneAction:2);
var parity= data.sum+1;
var freq= Duty.ar(Dseq([invBaudrate*16]++invBaudrate.dup(11)), 0, Dseq(#[1, 0]++data++[parity%2, 1]));
var src= SinOsc.ar(freq*(maxFreq-minFreq)+minFreq, 0, amp);
OffsetOut.ar(out, src*env);
}).add;
};
)
//test sending a simple counter
(
var baudrate= 126;
var lo= 882;
var hi= 1764;
OSCFunc({|msg|
var byte;
//("received:"+msg[3..]).postln;
byte= msg[3..].sum{|x, i| 2**i*x};
("received:"+byte).postln;
}, '/data');
Routine.run({
Synth(\redFSKdecode, [\in, 20]); //bus 20
inf.do{|i|
var byte= (i%256).asInteger;
var data= byte.asBinaryDigits.reverse;
s.bind{
("sending :"+byte).postln;
Synth(\redFSKencode, [\minFreq, lo, \maxFreq, hi, \data, data, \out, 20, \invBaudrate, 1/baudrate]); //bus 20
Synth(\redFSKencode, [\minFreq, lo, \maxFreq, hi, \data, data, \out, 0, \invBaudrate, 1/baudrate]); //monitor
};
(1/baudrate*(11+16)).wait;
};
})
)
//test a string of text
(
var baudrate= 126;
var lo= 882;
var hi= 1764;
OSCFunc({|msg|
var byte;
//("received:"+msg[3..]).postln;
byte= msg[3..].sum{|x, i| 2**i*x};
("received:"+byte.asInteger.asAscii).postln;
}, '/data');
Routine.run({
Synth(\redFSKdecode, [\in, 20]); //bus 20
"hello supercollider!".do{|x|
var byte= x.ascii.clip(0, 255);
var data= byte.asBinaryDigits.reverse;
s.bind{
//("sending :"+x+byte+data).postln;
Synth(\redFSKencode, [\minFreq, lo, \maxFreq, hi, \data, data, \out, 20, \invBaudrate, 1/baudrate]); //bus 20
Synth(\redFSKencode, [\minFreq, lo, \maxFreq, hi, \data, data, \out, 0, \invBaudrate, 1/baudrate]); //monitor
};
(1/baudrate*(11+16)).wait;
};
})
)
This FSK technique was used in many projects to control LEDs. e.g. most of the 3rd gen projects on RHYME are running SuperCollider on an iPod Touch with the 6 channels led brightness data being sent via FSK on one of the audio output channels.
Below some code for reading and parsing data from the Android app AndroSensor.
With this free app, you can record a lot of different sensors in your Android phone (like GPS, accelerometer, mic, battery, light etc) and save it to a comma-separated file (CSV). Later one can copy over the data file from the Android phone to a computer and read it in SuperCollider. the CSV files are normally stored on the SD card in a folder called AndroSensor.
By default, the app only saves and display data at slow update rates, so go into the AndroSensor's settings and change update interval to very fast and recording interval to for example 0.05 seconds (the fastest).
// https://play.google.com/store/apps/details?id=com.fivasim.androsensor
//--read the data - edit the path to your CSV file
(
var path= "~/Desktop/Sensor_record_20141018_115750_AndroSensor.csv"; //edit here
var data= CSVFileReader.read(path.standardizePath, delimiter:$;, startRow:1);
var data2= data.flop;
var dict= ();
data2.do{|x|
var key= x[0];
var val= x.copyRange(1, x.size-1);
while({key.last==Char.space or:{key.last==$:}}, {
key= key.copyRange(0, key.size-2);
});
if(val[0].any{|x| #[$:, $/].includes(x)}.not, {
val= val.asFloat; //make single numbers into floats
//}, { //else do nothing - keep date and satellites as strings
});
dict.put(key.asSymbol, val);
};
~dict= dict; //handle
)
//--list all stored keys...
~dict.keys.do{|x| x.postln}
//--access data stored at keys...
~dict['ACCELEROMETER X (m/s²)']
~dict['ACCELEROMETER Y (m/s²)']
~dict['ACCELEROMETER Z (m/s²)']
~dict['GRAVITY X (m/s²)']
~dict['GRAVITY Y (m/s²)']
~dict['GRAVITY Z (m/s²)']
~dict['GYROSCOPE X (rad/s)']
~dict['GYROSCOPE Y (rad/s)']
~dict['GYROSCOPE Z (rad/s)']
~dict['LIGHT (lux)']
~dict['MAGNETIC FIELD X (μT)']
~dict['MAGNETIC FIELD Y (μT)']
~dict['MAGNETIC FIELD Z (μT)']
~dict['ORIENTATION X (°)']
~dict['ORIENTATION Y (°)']
~dict['ORIENTATION Z (°)']
~dict['PROXIMITY (i)']
~dict['SOUND LEVEL (dB)']
~dict['LOCATION Latitude']
~dict['LOCATION Longitude']
~dict['LOCATION Altitude ( m)']
~dict['LOCATION Speed ( Kmh)']
~dict['LOCATION Accuracy ( m)']
~dict['Satellites in range'] //note as strings
~dict['Temperature (F)']
~dict['Level (%)']
~dict['Voltage (Volt)']
~dict['Time since start in ms'] //timestamps in milliseconds
~dict['YYYY-MO-DD HH-MI-SS_SSS'] //absolute timestamps - note as strings
//--plot the xyz accelerometer data...
(
[
~dict['ACCELEROMETER X (m/s²)'],
~dict['ACCELEROMETER Y (m/s²)'],
~dict['ACCELEROMETER Z (m/s²)']
].plot
)
//--plot the xyz orientation data...
(
[
~dict['ORIENTATION X (°)'],
~dict['ORIENTATION Y (°)'],
~dict['ORIENTATION Z (°)']
].plot
)
//--plot a lot of data (but not all)...
(
[
'ACCELEROMETER X (m/s²)',
'ACCELEROMETER Y (m/s²)',
'ACCELEROMETER Z (m/s²)',
'GRAVITY X (m/s²)',
'GRAVITY Y (m/s²)',
'GRAVITY Z (m/s²)',
'GYROSCOPE X (rad/s)',
'GYROSCOPE Y (rad/s)',
'GYROSCOPE Z (rad/s)',
'LIGHT (lux)',
'MAGNETIC FIELD X (μT)',
'MAGNETIC FIELD Y (μT)',
'MAGNETIC FIELD Z (μT)',
'ORIENTATION X (°)',
'ORIENTATION Y (°)',
'ORIENTATION Z (°)',
'SOUND LEVEL (dB)'
].collect{|x| ~dict[x]}.plot;
)
//--sound example mapping accelerometer to simple sound
(
s.waitForBoot{
var a= {|freq= 500, amp= 0, pan= 0| Pan2.ar(Pulse.ar(freq.lag(0.25), 0.2, amp.lag(0.25)), pan.lag(0.25))}.play;
s.sync;
Routine.run({
var prevTime= 0;
~dict['Time since start in ms'].do{|x, i|
a.set(
\freq, ~dict['ACCELEROMETER Y (m/s²)'][i].linexp(-20, 20, 100, 1000),
\amp, ~dict['ACCELEROMETER Z (m/s²)'][i].linlin(-20, 20, 0, 1),
\pan, ~dict['ACCELEROMETER X (m/s²)'][i].linlin(-20, 20, -1, 1)
);
(x-prevTime*0.001).wait;
prevTime= x;
};
"done".postln;
a.release;
});
};
)
Attached below is a demo CSV file of me first keeping the phone still, and then shaking it a bit + turning it around. The data recording is only about eight seconds long.
If you plot for example the 3D accelerometer data it will look like this...
