Instant Broadcasting System is a software solution for receiving and mixing several video streams from cellphone cameras and broadcasting it live on a screen and/or to the internet. It’s an academic research project from Mobile Life in Stockholm under the lead of Ph.D student Arvid Engström. This project has been going on for about two years and I’ve been helping out on and off during these two years. The software has gone through a few name changes during the project. I’ve written about it here before under the names WeJ and Swarmcam.

The solution has two components - a Symbian application that is running on a Nokia N60 cellphone and a Max/MSP mixer application that is running on Mac Pro. During this demo we’ve been using 3 cellphones but the system can for sure do 8, probably up to 15. Each input takes a lot of CPU to process so you need a powerful computer to do it. The program has been tested on a MacBook Pro but it hits the ceiling pretty fast. And another problem with having many inputs is that it’s hard to overview all the preview windows. The cellphones streams the video over wifi or 3G to the computer. The resolution of the video is 320×240 at a top frame rate of 15fps. This isn’t very exciting but with Turbo 3G already here (in Sweden at least) and 4G at the doorstep we can expect much better image quality. Some new phones have a better codec for video compression which also gives better video quality. The software on the phones will be able to run on other Symbian enabled phones as well as the Androids. The iPhone has had it’s video functionality under lock, but since the recent release of the Bambuser app it seems like it would be possible to cover the iPhone too.

The interface has gone over a huge overhaul and finally got some design to it. A quick run trough:

In the left column you have a preview window and output settings as well as audio settings, you can broadcast audio from one phone a the time.

The middle column is the home of the A and the B unit as well as the preview windows for each camera. You can see four preview windows but we were only using tree for this demo. Below are the image settings for the A and the B bus as well as a the “Backchannel” which is a system for communicating with each camera person. The A unit of the mixer hosts the cellphone camera streams. You switch stream with the number keys or by clicking the preview windows. The B side hosts the more artistic side; pre made clips from the video bank and loops (recorded from the live streams). You can crossfade between the A & B as well as mix them through blend modes.

The right column has the video bank on top and the loop module underneath. You can feature either the loop module or the video bank as well as mix them together with blend modes. A nice thing with the loop module is that it has a recording buffer. This means that when you hit rec, it actually starts the recording back in time making sure you don’t miss the action. The in and the out point of the loop can be adjusted and you can save the loops for later use.

The interface could be controlled by a midi controller. We have had the Novation Nocturn mapped for this demo.

So would I use this software for VJ-ing? No, not at the moment. It sure is an interesting technology but over wifi and 3G, the image quality just isn’t quite there yet. I am however very curious to see if Arvid will add more VJ features like beat matching and on-the-fly midi mapping.

For for a few years, I’ve been a part of a research project that is driven by Arvid Engström at the Interactive Institute / Mobile  Life Centre, in Stockholm. I’ve written about this project here earlier.
Arvid has been researching around a VJ application using video streams from mobile phone cameras over the HSPA net (commonly known as ”Turbo 3G”). There can be many contributors (cameras) and one VJ that will decide what is being broadcasted. The VJ can give feedback to the people filming by showing that they are ”on air”, vibrate their phones for attention or send messages. The application goes under the name ”SwarmCam” for the time being.

The opensource software Movino is used to receive video over the HSPA-net and Max/MSP is used to mix the video sources and present the final output. In this case these softwares are run on a Macbook Pro. The software that must be present in the mobile phones to be able to connect to SwarmCam is written for Symbian. Unfortunately there are just few phones where user are allowed to install their own software.
The HSPA-net unfortunately has pretty big limits in upload speeds in it’s current state, which results in a pretty pixelated video quality. But with future cellphone nets we will be able to have much better video quality.

When I was testing the application for the first time with my DJ colleague, Henrik Berggren, the user interface wasn’t quite done yet. It was merely smacked together the night before. Even if this early version left a lot to wish for, one could clearly see it’s potential. The main functionality could be summed up like this:

• Crossfader between source A/B and hard cuts between A/B.
• Crossfader between B/C where C is pre made clips from the media bank.
• Tempo control for clips from the media bank.
• Media bank for pre made clips (that can be mixed with the live streams) and loops from the loop editor.
• Image controls for source A/B.
• The loop editor.

What impressed me the most of all these functions was the loop editor. The app will always buffer a few seconds of video. When you see something in a live stream that you want to record, just hit ”rec” and the recording will start a few seconds back in time (using the buffer). In that way you won’t miss the beginning of the action. The start and end of the recorded sequence can then be trimmed and saved as a loop in the media bank. This method of recording starting back in time is very smart and I haven’t seen it either in video or audio applications ever before.
On march 4 me and Henrik will demo SwarmCam when Mobile Life hosts an open house evening at the Interactive Institute. I’m hoping for a few additions like blend modes and MIDI-mapping that Arvid promised to build if there was enough time left.

Last spring me and Hinke Berggren did a VJ demo and discussion at the Interactive Institute in connection to the kick start of Arvid Engström’s project “Mobile Collaborative Live Video Production”. Dejakru has also been involved when Arvid later did interviews with VJs regarding how we work in live situations.

The first reports are now available for download. In the big main report, Engström investigates how VJs work, how teenagers use their cellphone cameras and how to build an application that will let you mix live video from cellphones in a VJ fashion. The application goes under the name “SwarmCam” (an earlier name was “WeJ”). After a pause in the project they are now back up and running and we are hoping to see some samples this spring.

For you theory heads out there the report is up for download in pdf-format from this link.

You can find more reports from Arvid Engström here.