notes from the bigfug

PatchBox is becoming quite the powerful tool for architectural video mapping but sometimes even I miss a certain sense of immediacy that I’ve grown accustomed to from my years of VJ’ing.

It was with much delight then that I received a link from my old chum (and favourite Stylophone player) Tom Hume last week pointing me towards a site where a chap based in Helsinki called J?rgen has created some software for using your mobile phone as a spray can, called – surprisingly – MobiSpray.

He’s been travelling around with a laptop, video projector, and generator, drawing all over buildings, fighter planes (!), rocks, and bags of rubbish all while wearing a military outfit.

Good man!

The system works by taking the tilt information that some models of mobile phones generate (like a Wiimote) and sending it wirelessly to a laptop that does the painting.? It can also use photos taken on the phone – nice touch!

However, it wasn’t really the interface that excited me – it looks a bit fiddly (sorry J?rgen!) – but the flat colour style of his drawings and what a very powerful and immediate effect they, and the drawing process itself, created.

I knew I wanted to achieve the same end result but using a different hardware interface, namely my nice little Wacom graphics tablet with all of it’s pressure sensitive goodness.

I’d seen a CGFX shader months ago in the NVIDIA Shader Library that could do painting within the shader itself.? I re-wrote it so it uses a seperate render target (for reasons that will become apparent later), and added the support code into PatchBox.

Initially it utilised the mouse but as a proof of concept it worked well, so I started looking at Raw Input (hence the comedy Google post earlier today) but ran into a few walls trying to integrate it with wxWidgets, so instead I put in the support for Wintab, giving me direct access and control of the tablet information.

So, at about midnight, it was ready for the first projector test:

First test of the new PatchBox paint mode

Aw, pretty flowers and… is that a… kind of squirrel?? The lower part with the lady dancing is a video screen.

This was shortly followed by test #2:

Second Paint Mode Test

And the actual screen grab from PatchBox:

Screen grab of the second test

I’m really excited about the results!? There’s a couple of features still to add (saving the textures out, hence the render target), but I’m already thinking I need to project on something… bigger…

Tags: cgfx, PatchBox