//practical sound analysis - fredrik olofsson

//organized by lullcec 28-30oct 2011, hangar, barcelona

//--session #01

//--installation/configuration

//--install supercollider

http://supercollider.sourceforge.net/downloads/

//--install sc3-plugins

http://sourceforge.net/projects/sc3-plugins/files/

osx binaries3.4.4 sep2011

put the folder where sc can find it and recompile...

Platform.userExtensionDir //install extras for this user only

Platform.systemExtensionDir //install extras for all users

osx: "~/Library/Application Support/SuperCollider/Extensions"

win: "C:\Documents and Settings\USERNAME\SuperCollider\Extensions"

linux: "/usr/share/SuperCollider/Extensions"

if discrepancy problem (duplicated classes) remove the offending classes (e.g. BEQSuite, Scale)

//--test sound

s.boot

{PinkNoise.ar(0.5)}.play

//panic stop all sound keyboard shortcuts

//cmd+. - for osx

//alt+. - for win

//c-c, c-s - for emacs

s.meter //level meters (or press 'l' when server window in focus

{DelayN.ar(SoundIn.ar, 1, 1)}.play //mic input with 1sec delay

{DelayN.ar(SoundIn.ar, 1, 1)}.play(outbus:1) //trick to avoid feedback a bit - play sound in right speaker only (mic is located on the left side on most mac laptops)

/*

Server.default= s= Server.internal; //mac osx only - need to use internal server for scope to work

*/

s.boot; //(or press 'space' with server window in focus)

s.scope; //(or press 's' with server window in focus)

{SinOsc.ar([400, 404])}.play //scope shortcuts: s, S, k, j, l, +, -, *, _

{LFSaw.ar(200)}.plot //10ms the default (0.01)

{LFSaw.ar(200)}.plot(0.2) //200ms

{LFSaw.ar(200)}.plot(1) //1sec

//careful when plotting more then 1sec.  might crash

{LFSaw.ar([200, 300, 400, 500])}.plot(0.1, minval: -2, maxval: 2)

{LFSaw.ar(200)}.plot2 //nicer

{LFSaw.ar(1).poll}.play //poll is also handy for debugging

{LFSaw.ar(SinOsc.ar(0.2).poll)}.play //put in a poll anywhere

{SinOsc.ar(0.2).poll(label: "sin")}.play //adding labels - useful when many poll in the same patch

{SinOsc.ar(0.2).poll(Impulse.ar(1), label: "trig")}.play //custom trigger - poll once a second

//optional:

install some quarks via Quarks.gui

Spectrogram //mac osx only

//--introduction/overview/examples

//--motivation

why analyze sound?  some applications...

interactive installations (call and response, dialogue, transformation, statistics)

composition (instrument and live electronics, spectral orchestration, agent based systems)

music transcription (scores)

speech recognition (computer answering machine)

data transfer (modem)

physics (car metal tests)

automatic feedback prevention systems (arena)

etc.

//--human hearing

see equal loudness curves (Fletcher-Munson curves)

the ear is extra sensitive between 1 and 4kHz

frequency range: 20Hz to 20kHz (10kHz for old people (about 1 octave less))

amplitude range: 120dB (1 trillion in amplitude)

frequency separation: about 0.3% at 3kHz, about 3% at 100Hz

amplitude separation: about 1dB (12% change in amplitude)

direction separation: about 3 degrees

AmpComp //see helpfiles

AmpCompA

//compare:

{var freq= MouseX.kr(100, 9000); SinOsc.ar(freq, 0, 0.1)}.play;

{var freq= MouseX.kr(100, 9000); SinOsc.ar(freq, 0, 0.1*AmpComp.kr(freq, 100))}.play;

//--common techniques

amplitude tracking

pitch tracking

timbre analysis

envelope following (not only amp)

beat/tempo tracking

event detection

segmentation (vertical)

separation (horizontal)

feature extraction (database)

voice recognition

speech recognition

music information retrieval (pattern matching, mpeg-7, metadata, genre categorizing)

//--literature

Albert S. Bregman - "Auditory scene analysis: the perceptual organization of sound"

Tristan Jehan - "Creating Music by Listening" (PhD Thesis)

Nick Collins - "Towards Autonomous Agents for Live Computer Music: Realtime Machine Listening and Interactive Music Systems" (PhD Thesis)

Dan Stowell - "Making music through real-time voice timbre analysis: machine learning and timbral control" (PhD Thesis)

//--examples

score following http://www.youtube.com/watch?v=q55Okme1vTc

sCrAmBlEd?HaCkZ! http://www.youtube.com/watch?v=eRlhKaxcKpA#t=92s

//--simple amplitude and pitch tracking warmup exercise

Amplitude

Pitch

{Amplitude.ar(SoundIn.ar).poll}.play

//trick to avoid feedback on mac laptops where the mic is placed near the left speaker.  so we play in right speaker only

{Amplitude.ar(SoundIn.ar).poll}.play(outbus:1)

{SinOsc.ar(440, 0, Amplitude.ar(SoundIn.ar))}.play(outbus:1)

//another trick to avoid feedback - delay 1sec

{SinOsc.ar(440, 0, DelayN.ar(Amplitude.ar(SoundIn.ar), 1, 1))}.play(outbus:1)

//using mouse x/y to set amplitude tracker sensitivity

{SinOsc.ar(440, 0, DelayN.ar(Amplitude.ar(SoundIn.ar, MouseX.kr(0, 1), MouseY.kr(0, 1)), 1, 1))}.play(outbus:1)

//strange and fun effect - the louder the more reverb

//(also how to organize code using variables)

(

{

var src= SoundIn.ar;

var amp= Amplitude.ar(src, 0.05, 0.1);

FreeVerb.ar(src, (amp*2).clip(0, 1));

}.play(outbus:1)

)

//Pitch outputs both frequencies and flag. here we use DC.ar(0) to output silence so we don't listen to the output of Pitch

{Pitch.kr(SoundIn.ar).poll; DC.ar(0)}.play

{SinOsc.ar(Pitch.kr(SoundIn.ar)[0], 0, 0.2)}.play

{SinOsc.ar(DelayN.kr(Pitch.kr(SoundIn.ar)[0], 1, 1), 0, 0.2)}.play(outbus:1)

(

{

var pct= Pitch.kr(SoundIn.ar);

var src= SinOsc.ar(pct[0], 0, pct[1].lag(0.01)*0.2);

DelayN.ar(src, 1, 1);

}.play(outbus:1);

)

//see PracticalSoundAnalysis01examples.html

//--more examples and possible applications

//--examples

klipp av - audiovisal feedback examples (klippav-canned1.mov, klippav-capdemo31.mov)

HarmonicsVoice.html //in folder supercollider/examples/demonstrations

//--pre and post processing

filtering, normalizing, mixing sounds before analysis (e.g. BPF, LeakDC)

treating and storing analyzed data (e.g. lag, round, >, lincurve, curvelin)

//--using analyze to drive synthesis parameters

mapping parameters

hijack the structure of other music

obviously parameter feedback is fun

replacing similar sounds

//some simple examples

(

{

var src= SoundIn.ar;

var pch= Pitch.kr(src);

var amp= Amplitude.ar(src);

LFSaw.ar(pch[0], 0, amp);

}.play(outbus:1);

)

//the louder the more panned to the right

(

{

var src= SoundIn.ar;

var pch= Pitch.kr(src);

var amp= Amplitude.ar(src);

Pan2.ar(LFSaw.ar(pch[0], 0, amp), amp.linlin(0, 1, -1, 1));

}.play

)

//amp.linlin(0, 1, -1, 1)

//amp*2-1 //same thing as linlin

//swapping loud/soft amp - talking loud will result in soft oscillator, talking soft will result in a loud oscillator

(

{

var src= SoundIn.ar;

var pch= Pitch.kr(src);

var amp= Amplitude.ar(src);

Pan2.ar(LFSaw.ar(pch[0], 0, 1-amp*0.2), 0);

}.play

)

//slow crescendo of oscillator when talking

(

{

var src= SoundIn.ar;

var pch= Pitch.kr(src);

var amp= Amplitude.ar(src, 1, 2);

LFSaw.ar(pch[0], 0, amp);

}.play(outbus:1);

)

//swap clear sounds from mic with noise and noisy sounds with sine tone (using Pitch's clar)

(

{

var src= SoundIn.ar;

var pch= Pitch.kr(src, clar:1);

Pan2.ar(XFade2.ar(LFSaw.ar(pch[0], 0, 0.2), WhiteNoise.ar(0.2), pch[1].linlin(0, 1, -1, 1).lag(0.5)), 0);

}.play(outbus:1);

)