frock stars celebrates the australian fashion week with an exhibition put on at the powerhouse museum. i was fortunate enough to contribute to the event with two interactive installations which allow the user to style the hair and makeup of a virtual model with the use of real physical styling tools.

in the video you can see people using objects such as the hair dryer and curling tongs as the controllers to the interactive experience. it was important that these objects were as close as possible to the real thing, which in the early scoping eliminated the use of markers and eventually lead to the use of colour tracking as the main computer vision method for identifying and tracking the objects.

colour tracking proved to be quite accurate thanks to the use of flouro colours which stood out more from the surrounding environment and were less likely to clash with people’s clothing (unless the 80s made yet another comeback!). each of the two interactives were fitted with a affordable fire-i camera, which allows to configure most capture settings including the ability to turn off the EVIL auto-gain.

the above photo shows the colour tracker passing data into flash. the colour tracking software was written using openFrameworks and communicated with flash via a socket connection. openFrameworks did all the tracking while flash took care of all the pretty graphics, animations and video. there is some more info under this OF post.

here is the source code to the colorTracker.

timebomb cube from lukasz on Vimeo.

the opportunity came to exhibit the installation again and instead of doing the same old, maddi and i decided to push the idea further and take timebomb away from the 2d flat canvas into a 3-dimensional cube format. it also meant that the cube would be able to facilitate a rear-projection set up and ultimately a multitouch surface.

in the couple months before the event i did my research and developed a basic prototype of a diffused illumination (DI) projection. this is one of many multi-touch methods and one that works particularly well for large interactive surfaces. the downside of using DI is it is not as accurate as some of the other methods, although the timebomb interaction is based on movement detection and does not require a huge amount of precision as long as it could detect a change in the image.

there is a great thread on DI multitouch surfaces on the NUI website and is a great place to start when researching into the topic. the whole method works on infrared technology and pretty much boils down to the IR camera which blocks out visible light, so it doesn’t see the projection but it can see the people touching the surface of the projection.

i ended up going for a firefly mv black and white camera from point grey. its a firewire camera which has quite a lot of support in c++ land and appears to be a favorite amongst openframeworks users. the camera only comes as a body and so i also had to purchase a lens and an IR-filter to block out visible light. if you live in australia and need to kit up your computer vision camera, i recommend a company called adept, very knowledgeable and friendly chaps.

to complete the kit i got a couple IR spotlights from ebay for pretty cheap and i had a chance to test them against some top range IR lights that cost in the thousands ( thanks to the guys from adept ) and the results were surprisingly pretty much identical. just hop on ebay and search for ir illuminator but pay attention to the range and spread of the light, you’d want something with a decent angle of say 60 degrees and that can reach at least 40 meters.

finally you need something to project onto, a diffused material that can catch the light from the projector and be opaque enough to let IR light through so that the camera can see peoples movements on the other side. there are many specialised projection films which cost in the thousands to cover a large surface, some of which are outlined in this article, although since we only had a limited budget we went with drafting film, very cheap and a decent picture contrast.

building of the giant 2.5 meter cube was intense. spent 2 weeks worrying the whole thing would not hold and fall apart on some unsuspecting user, so i started reinforcing every panel of wood with metal brackets, especially the interactive surface which was constructed from 2 giant 10mm thick sheets of acrylic and i made sure the frame would be able to support it. in the end, the cube was SOLID! even the most enthusiastic of multitouch users could not rock it in the slightest.

on the day of the festival it was great to see people interacting with timebomb on the multitouch surface. people were really taking their time with the installation, trying to understand and harness its usability to control the time distortion effect to uncover the specific parts of the interactive graffiti wall. the one to one interaction scale was really the trick that made it all work and its whats so appealing about multitouch surfaces. its bridging the virtual gap and drawing the digital within the reach of the tangible.

light chasm from lukasz on Vimeo.

had the pleasure to team up with the very talented flutter lyon (robyn wilson) to collaborate on a audio / visual performance. we only had a week to prepare and decided to go with a black and white, op-artish theme for the projection visuals, matched with robyn’s very striking black and white attire which during the performance became part of the projection surface.

i wanted to create a custom vj app for the performance and have been getting more proficient in openframeworks so decided to give it a go as a technical and creative challenge. i began by coding the basic visual elements such as the black and white sun rays, the checkered background and playing around with blend modes in opengl. it took me a good 2 hours to work out how to do inverse blending… no reference for it anywhere! so here it is,

glEnable( GL_BLEND );
glBlendFunc( GL_ONE_MINUS_DST_COLOR, GL_ONE_MINUS_SRC_COLOR );
// draw your shapes here.
glDisable( GL_BLEND );

i wanted to control the visuals through my iphone via OSC, that way i could put my laptop away and become part of the crowd… so i wrote a wrapper class for OSCemote (an OSC iphone application) that told my program what buttons were being pressed and the slider values. hope to extend it further for accelerometer values which i think will be far more interesting way of controlling visuals and with the possibility of using body movements in a performance to react visuals.

next i plugged in some FFT classes from the openframeworks forum and viola, audio-reactive visuals. it goes to show the maturity of openframeworks where you can grab a bunch of classes created by the community and plug them straight in without any hassle.

here is the source if any one is interested in running it, made for of_0.06

really enjoyed building vj software from scratch just because you have so much more control over what you can make. im now toying with the idea of taking it further and creating a nodel based interface, so something like quartz composer or pure data interface but for openframeworks. the mind boggles…

bowie cam from lukasz on Vimeo.

the back story is… some friends at the world’s end studio have a biegel named david bowie and it jumped out of a 2 storys window and broke its leg. the dog that thought it could fly… so they put on a david bowie fund raiser party for the pooch’s operation.

ive been itching to get my face tracking projections out there and thought the night was a good opportunity to do something quick and fun. bowie cam is just that… it pastes a face of david bowie over yours so you can get down just like the freaky old bastard. there a four face mask modes it cycles through – two of mr bowie, one being of the pooch and a ziggy lighting bolt across the face. the interactive projection was good fun and people got it straight away, although i suspect some people were just happy to see themselves on the big screen.

it was also a good lesson never to depend on venue lighting. for those that have been to q-bar in sydney will know that its a pretty dark venue, something out of a david lynch film with all the red lighting. so had to improvise and mount a lamp to expose people in front of the camera a little better. even then, it was a fine line… if they got too close, their faces were washed out and the haar finder was unable to pick up their features and when they got too far, they would disapear into the shadows. so i think overall the accuracy of the face tracking was down to about 20% of its normal accuracy. still, good to try it out in the wild.

get the ofxCvHaarTracker source for openframeworks with example.

Time Bomb – Interactive Graffiti (StupidKrap & Holler) from Mike Hill on Vimeo.

timebomb is an interactive graffiti installation that I collaborated on with Maddi Boyd (KissKiss) from StupidKrap. check out the promo video to see what its all about…

here’s a little technical insight into the project.

the timebomb application is based on memo’s fluid dynamics library for openframeworks, ofxmsafluid. its a fantastic opensource library that memo has done a lot of amazing things with and i’ve been able to use it as the basis for fluid time distortion.
interaction input comes from a webcam, the live feed is analysed and used to create ripples in a fluid image. those fluid values are then applied to the pixels of a video, the pure white pixel (0xFF) equals a pixel taken from the top frame of the video and a black pixel (0×00) equals a pixel taken from the bottom frame of the video, all other shades of grey represent pixels between the first and last frame. confused? well hopefully the video of the debug screen will explain it better.

the video on the top left is the actual time-lapse we shot over 5 days.
next to it is the final output with time distortion applied to it.
all other frames below show the camera input and the ripples created in a black and white fluid image.

timebomb debug from lukasz on Vimeo.

here is the final effect on its own.
this is what is going to be projected onto a blank canvas at the exhibition.
people will be able to time-distort the graffiti piece through their body movements and uncover the hidden layers that lay beneath the surface of the finished piece.

timebomb from lukasz on Vimeo.

had the opportunity to exhibit the latest dear samantha work at the dark hall exhibition yesterday as part of the glorious undead festival. marcel and bronwyn did a great job transforming the kudos gallery into a forest of digital glow spotted with iridescent installations. the most impressive was a sculptural video installation by daniel kirkwood titled ‘dc city, glistening (a seasonal cycle)‘ using projection mapping, shown in the gif animation above. i’ve seen a lot of projection mapping on the interweb but this was the first time to witness it first hand and it was a real treat.
have to try it out myself very soon.