anneVideotracking

clean-up #23:

The screenshot below is of a simple little mac application I made in MaxMSPJitter. It takes realtime input from a web or DV-camera, chops it into six regions and tracks activity in each of those regions separately. The result is sent over to SuperCollider via OSC.

The analysis/tracking is just a simple average luminescence per frame. That is, it takes the mean of all the red, green and blue values in one frame and then calculate the luma with (0.299*red)+(0.587*green)+(0.114*blue). The luma will be close to 0 for dark frames and close to 1 for very bright video frames.

To tune this system to produce maximal output, one should first make sure the camera sits still and has sufficient and stable light. Then in the software there are colour, brightness and contrast controls to help get even better readings. (Completely inverting all the colours might help in really dark places.)

The luma values (float 0.0-1.0) can be rounded off to nearest number and also smoothed out. This is often necessary to avoid flutter and noise. And most important - there is manual calibration of the luma values - either per region or globally.

To calibrate one should first empty the camera's view of any objects and then press the 'calibrate min' button in the lower-left corner. Then move some objects around to see what is the maximum reading. Last put that number into the 'set max' number box. One can also calibrate each region separately with the min and max buttons next to the preview windows.

Download the mac OSX standalone from /code/apps/#annevideotracking.

and here is SuperCollider code to read the data and make some simple sounds out of it.

//to start: select all + enter
//to stop: cmd + .
(
n= 6;
s.latency= 0.05;
s.waitForBoot{
  var dlast= 0.dup(n);
  d= {Bus.control(s, 1)}.dup(n);
  e= {Bus.control(s, 1)}.dup(n);
  OSCFunc({|m|
    var index= m[1], val= m[2], diff= (val-dlast[index]).abs;
    //m.postln;
    d[index].set(val);
    e[index].set(diff);
    dlast.put(index, val);
  }, \anneVideoTracking);
  CmdPeriod.doOnce({d.do{|x| x.free}; e.do{|x| x.free}});
  SynthDef(\annetest, {
    var src= Mix({|i| SinOsc.ar(i*100+400, 0, LagUD.kr(In.kr(e[i].index), 0.01, 0.1))}.dup(n));
    Out.ar(0, Pan2.ar(src));
  }).add;
  s.sync;
  Synth(\annetest);
};
)

Updates:

• 190121: there is a new much-improved version 4: /f0blog/annevideotracking4/