recently i was involved on a interesting project for Heineken which involved building a iPhone app for the 2011 Rugby World Cup. the project was headed by Holler Sydney. myself and Nathan de Vries joined the team to do the iphone build. part of the app was to create a interactive 3d game that allowed user to take live kicks as they were being taken in the actual world cup. in this blog post i wanted to concentrate just on the live kick game and go into some technical detail about the making of.

first of all, please download the app and have a play!
here is a link to the Heineken Clubhouse on the iTunes store.

the live kick game was built using a collection of open source libraries and frameworks, most notably using openframeworks. in the past ive used openframeworks for a couple iphone and ipad releases (horizons and haeckel clock) and the nice thing was that OF sets up a EAGLView for you by default and you just have to worry about drawing your graphics/animations. its great for getting OF apps running quickly on the iphone but for the heineken project i needed to build a native iphone app and somehow get openframeworks working inside and be able to create and destroy instances of the game. this took a bit of hacking and essentially i ended up with a very bare bones version of openframeworks with only its openGL functionality (ofGraphics, ofTexture etc). i even removed ofImage and loaded images using UIImage before converting them to ofTexture to be used inside the game. this meant i had a very light version of openframeworks with minimal dependencies and still have access to the great openGL code that OF abstracts.

the game uses some nice stadium models and baked textured which were created by a 3d designer. it was crucial that the scale and size of the models was exactly the same as used inside of the game. you can imagine the game working with boxes and flat planes to describe the 3d environment and the model is the final decorative layer placed on top to make the whole thing look real. it took a bit of trial and error getting the models to work and eventually i ended up going with the .obj format for the simple reason that it was the only one i could get to work! i converted the .obj models into static arrays of vertices, texture coordinates and normals using this obj2opengl script.

the 3d physics in the game was achieved using bullet physics library. it was my first venture in 3d physics land and it was a little daunting but i managed to get some of the basic examples running and from there it was just a matter of getting familiar with the library. it is very comprehensive and has been used in 3d films like Shrek and Megamind, so initially i associated it with being a chunky and processor intensive library although as i found out, its very modular and handles simple geometry pretty well on the iphone. i ended up using a single infinite plane for the game field, three boxes for the goal posts and the ball was generated as a 3d convex hull shape generated from the vertices of its model (done by some handy functions inside the bullet physics lib). adding these elements into the physics simulation i was able to kick the ball by apply a force to it and bullet handled the ball bouncing and obstacle collision with the goal posts.

the game design required some nice dynamic font rendering using drop shadows and strokes which was beyond the simple font support in OF. these effects had to be done natively and overlaid the 3d game. having semi transparent text over the EAGLView came with no performance penalty which was good despite reading reports that this could slow things down but i assume this has been addressed with newer iphones.

lastly, couple cool opengl tricks ive learnt on this project were mip mapping and multitexturing. mip mapping came in very handy when rendering the grass texture on the field. it basically means that textures further away from the camera are seen as a bit more blurry, removing the flickering and noise you’d otherwise see when using high resolution textures on small models. multitexturing allows for adding multiple textures onto the one plane with the added option of blending the textures in various ways. this can be seen again on the grass field texture where first the grass texture is drawn and then the field lines as another texture on top.

if you have any other questions about the app, please post below and i’ll do my best to answer them.

beginning to explore volumetic rendering using marching cubes and in the process creating some 3d metaball gif loops. made in openframeworks using ofxMarchingCubes and ofxDither.

this is a animated gif piece i contributed for the fa-g.org group show. the theme of the exhibition is FRIENDS.

recently ive joined forces with a group of artists of various talents, running our new and exciting operation from the headquarters of toby and pete [ http://tobyandpete.com ]. just before we all came together, toby, pete and esjay had already been planning to hold an exhibition at lofi gallery, titled ‘pretty ugly’ and so a bunch of us from the collective tagged along and contributed with some new work.

toby and i decided to collaborate on a audio-visual piece for the show, toby creating some very dark and brooding electronica while i created generative visuals which were projected onto one of the gallery walls. having our laptops hooked up and talking over OSC, toby ran his music live using ableton and fired out audio triggers when certain samples were played. i also analysed the live audio spectrum and had that hooked up to the visuals as a second interactive input.

other work at the exhibition featured print, stencil, video, wood carving and a giant neon gallows!
more pics and documentation here >>> http://prettyuglylofi.tumblr.com

haeckel clock is a interactive clock app for the iPad inspired by the work of Ernst Haeckel, a famous biologist and artist. the clock elements which represent the time are based on radiolarians, a group of single celled organisms which were often the focus of haeckel’s studies and his intricate illustrations.

the haeckel clock shows the time in two different modes which the user can toggle between by double tapping on the screen. the first mode arranges the units of time into a circular nucleus where the digits of the current time are pulled into the center while the unused digits orbit around the nucleus. as the seconds tick by the digits rearrange themselves in a kind of evolutionary dance, creating a unique and generative representation of time. the second mode is a little more self explanatory and shows the digits arranged into columns. the digits of the current time stack up in columns above the number which they represent while the unused digits stay afloat at the top of the screen.

the clock was developed using openframeworks and box2d (a popular 2d physics library). each digit is interactive so you can grab it and flick it around which i think makes for a fun and tactile experience. once the user releases a digit it quickly springs back to its resting position and helps keep the clock elements in order. the iPad accelerometer is also used to add extra motion to the digits by swaying them in the tilt direction.

the haeckel clock was created in collab with the clever kids @ holler sydney

nawlz is a online interactive comic created by stu campbell which he has been running for a number of years. recently stu has undertaken a bold task of taking the online flash comic and porting it over to the iPad. being a massive fan of nawlz for some time, i was thrilled when stu got in touch with me to collaborate on interactive visual effects for the comic. you can download the effects below and get a sneak peek at whats to come for the iPad release.

DOWNLOAD [MAC OSX]

the majority of the effects use MSAFluid library for simulating 2-dimensional fluid forces which are applied to particles released on the screen. the particles are steered by the vector field from the fluid simulation but also move around randomly using shiffman’s wander algorithm. the fluid forces are also used to distort texture meshes as seen in the first whale example.

[SOURCE CODE]

ive made the source code open and it can be downloaded above.
you will need to run the code inside of openframeworks.

lately ive been fascinated with gif animations that ive seen popping up online which employ tiny colour palettes, mostly just black and white, and use dithering as a technique of creating the appearance of more colours being used then there actually are. in particular ive been enjoying the work of francoise gamma from the computers club net art collective. its a lovely blend of highly detailed 3d animations but then rendered down into a old low-fidelity format you’d more likely see on a old Apple SE/30 then a modern computer.

there are a few different methods for dithering a image, ordered dithering of which has to be my favourite purely for its structured, crosshatch aesthetic. the appeal for the pattern is probably the result of endless hours playing early games like monkey island and space quest and now its permanently etched into my visual cortex.

i put together a ofxDither addon which is now on github. at the moment the addon supports ordered, floyd–steinberg and atkinson dithering algorithms. i had some help from jesús gollonet whos also done some experiments with dithering in openFrameworks. his addon ofxHalftoner is also on github.

the aim while exploring these dithering algorithms was to create 1-bit gif loops, one a day for a week. i got up to four and then some other work got in the way and spoiled the plan but it was good to work on short generative animations instead of usually spending days on trying to create more complex visual effects. ive included the code for these gifs which you can download here. there are three examples which you can swap out in the main.cpp file.

ecologic exhibition at the powerhouse museum is a look at the science behind climate change. it comprises of several interactive, one of which, a interactive weather map, i was involved in creating together with alex christian on design.

the idea behind the interactive weather map was to allow participants to explore the last 100 years of the australian climate, visualising the change in temperature, rainfall, la nina / el nino, and showing when and where extreme climatic events have occurred. the climate data was provided by the bureau of meteorology (BOM) who were amazingly helpful with lending their knowledge and resources. they were also very keen on the interactive as temperate, rainfall and el nino patterns have never been mapped out together in the one visualisations and they were curious to see what visual results the interactive would reveal.

the UI was designed with simplicity in mind as we wanted the user to be able to intuitively browse the weather data. the timeline is the main interactive element which when dragged will take the user to a particular year for which they can view the temperature and rainfall. the play button also allows to automate this process and play through the 100 years of weather on the timeline. extreme weather events are indicated on the timeline as icons and when reached, display another popup on top of the map, pointing to the location where the event has occurred.

the user interface was built in flash and imported into openframeworks using the ofxFlash addon which ive recently been developing. ofxFlash is very handy for building UIs in flash and importing them into c++ projects as it saves a lot of time laying out assets and creating button interactivity. it also works on the iphone and ipad… flash on the iphone you say?! and they said it couldn’t be done ;)

now onto the interesting bit and the meat of the project.
i had a lot of fun working with the data provided by the bureau of meteorology, essentially i had to create a contour map animation of changing weather patterns, pretty much the same animation you see on the weather channel. it was a great moment when i realised that it could all be done using openCV and in real-time!

BOM have a few different formats for supplying weather data, all of which i came across were in text file format. so the first step was getting the data from a text file and putting it into a image file that openCV could understand. parsing the data into a image file created something like the below, in total 100 image files, one for each year.

contour maps are great for visualising areas of common value, like height or temperature, and effectively show the spread and change of values over an area. the first step in transforming the raw image data into a contour map is to band the values. so to split the raw image into 10 bands involves duplicating the image 10 times and for each image setting the pixel threshold a notch higher until the complete color range is covered. using openCV i then run contour analysis on each of the banded images to get back a bunch of blobs which are my contours for that layer. i then add them together, colour code each layer and get something like the below.

openframeworks is a C++ toolkit for creative coding which is open source and supported by an avid online community that openly share their code and knowledge. it was time to bring out the OF community in sydney out of the woodwork and chat shop.

rene and i finally got it together after a couple months of inconvenient date swapping. we thought rather then jumping straight into a knitting circle, it would be a good idea to start from the beginning with a introductory workshop into openframeworks.

the turn out was great, a mix between experienced OF’ers and those who were just getting into it. we covered quite a big array of topics starting from the basic OF structure, to creating simple particle systems, to green screening via pixel manipulation, to openCV and blob tracking. we eventually finished with a demo of a interactive table sending TUIO data as input into a particle system we covered in a previous example. all this in a 4 hour session!

all the source to the examples we covered in the workshop can be downloaded here.

we’re planning to make the next one a lot less formal, more of a get together to chat OF.
you can follow the latest developments in the OF sydney community here,

http://www.facebook.com/group.php?gid=147185535296993
http://groups.google.com/group/ofsydney

frock stars celebrates the australian fashion week with an exhibition put on at the powerhouse museum. i was fortunate enough to contribute to the event with two interactive installations which allow the user to style the hair and makeup of a virtual model with the use of real physical styling tools.

in the video you can see people using objects such as the hair dryer and curling tongs as the controllers to the interactive experience. it was important that these objects were as close as possible to the real thing, which in the early scoping eliminated the use of markers and eventually lead to the use of colour tracking as the main computer vision method for identifying and tracking the objects.

colour tracking proved to be quite accurate thanks to the use of flouro colours which stood out more from the surrounding environment and were less likely to clash with people’s clothing (unless the 80s made yet another comeback!). each of the two interactives were fitted with a affordable fire-i camera, which allows to configure most capture settings including the ability to turn off the EVIL auto-gain.

the above photo shows the colour tracker passing data into flash. the colour tracking software was written using openFrameworks and communicated with flash via a socket connection. openFrameworks did all the tracking while flash took care of all the pretty graphics, animations and video. there is some more info under this OF post.

here is the source code to the colorTracker.