Work with Mark: iStreet - Online

Taking the Intelligent Street project further, Mark d'Inverno wanted me to try to get it online. The idea was to let people surf to a webpage, send commands to a running iStreet system and hopefully collaborate with other online users to compose music. Just like in the original SMS-version of the piece, everybody could make changes to the same music and the result would be streamed back to all the online users.

The first prototype was easy to get up and running. We decided to use Processing and write a Java applet for the web user-interface and then stream the audio back with Shoutcast. The users would listen to the music through Itunes, WinAmp or some similar program. This, of course, introduced quite a delay before they could actually hear their changes to the music. But that was not all too bad as we had designed the original iStreet in a way that latency was part of the user experience :-)

(The commands sent there via SMS took approx 30 seconds to reach us from the Vodafone server and that could not be sped up. The users were given no instant control over the music - rather they could nudge it in some direction with their commands.)

So our internet radio/Shoutcast solution worked just fine and we had it up and running for a short while from Mark's house in London.

That was a total homebrew solution of course. We wanted it to handle a lot of visitors, deal with some hopefully high traffic, be more stable, permanent and not run on an ADSL connection.

So at UoW we got access to a OSX server cluster and I started to plan how to install SuperCollider, a webserver and iStreet on that. Little did I know about servers, network and security and I had to learn SSH and Emacs to get somewhere. Rob Saunders helped me a lot here.

Then there were some major obstacles. First of all the cluster didn't have any window manager installed - not even X11. I spent many days getting SuperCollider and Stefan Kersten's Emacs interface for sclang called scel to compile.

We also had some minor issues with starting the webserver and punching hole in the university firewall etc. but the major problem turned out to be to get the audio streaming going. I didn't have root access and wasn't allowed to install jack on the cluster. To stream, I needed a Shoutcast client and some way to get audio to that from SuperCollider. I did find OSX programs that could have worked but none would run windowless on the console. So stuck.

The only solution was to write my own streaming mechanism. The resulting SuperCollider class for segmenting audio into MP3s is available here: /f0blog/work-with-mark-istreet-recording-mp3s-for-streaming/. A Java gateway handled the communication between SuperCollider and the Java applet that would stitch these files back together. (The Java gateway program also distributed all the other network data like the chat, checking online users, pending/playing commands etc. It used NetUtil by Sciss).

Unfortunately, I never got the streaming thing to run smoothly. Nasty hickups in the sound made it impossible to listen to. The hiccups were probably partly due to my crappy coding but I think the main error was in the ESS library for Processing. Either ESS (releases 1 and 2) can't do asynchronous loading or Java is just too slow to load MP3s without dropping audio playback. Very annoying.

After that defeat, I also spent time with flash and did a little player there that could load and play back MP3s smoothly. With the help from my flash expert friend Abe we also could talk to the flash thing from my Java applet via JavaScript. But time ran out and this would have been a too complicated system anyway.

So the iStreet never made it online. But again I learned a lot about networks, Unix, Java and some tools got developed in the process. RedGUI - a set of user-interface classes for processing, ISRecord and ISGateway for SuperCollider, and the ISgateway.java.

Screenshot of the Java applet running iStreet online...