dusk is an interactive installation exhibited at the underbelly arts festival which draws upon the connections between people, technology and environment. participants engage with tiny creatures of light which respond to movement, flock around to avoid rocks, follow the queen and take refuge in a tree.

ive been doing a bit of work lately with the chaps from angryPixel (scott brown and eli murray) and this has been the first interactive installation we’ve worked on collectively. we all have our own individual skill sets and have been able to contribute to the project in unique and complemantry ways. everything was tied together through the magic of OSC allowing for communication between openFrameworks <=> max msp <=> ableton live.

we had great luck with the space allocated for our installation, a disused warehouse large enough to facilitate a top-down projection. you can see the dimensions in the photo above, with eli included for scale (he is quite tall). there was also a very convenient mezzanine level where we were able to set up our control center and have a good view when testing the projection.

the tracking of people walking around in the space was done using openFrameworks with simple background subtraction + blob detection. the camera we used for the task was a point grey firefly mv with IR pass filter which did a beautiful job of capturing a crisp image with little to none noise.

the input from people walking around was then used to steer a flock of fireflies which flew around the space avoiding people and rocks. there was also a queen firefly which had a love / hate relationship with the rest of the colony. she followed her minions around but would scare them off if she got too close. the position of the queen in the space also controlled the direction of artificial shadows being cast by the rocks and panning of sound through the surround sound system.

the fireflies were also able to take refuge in a tree, backlit by DMX controlled LED lights. the tree would give off a stronger glow depending on how many fireflies were taking refuge in it at the one time.

in the end it was great seeing people chase the fireflies around the space and gang up to heard them into the tree. kids especially took to this activity and i think thats always a good indicator of a success interactive installation… if kids understand and enjoy it then the kid inside of us all will enjoy it too.

over the last week ive been playing around with openFrameworks on the iphone and getting a handle on openGL ES for a work project. its been lots of fun and suprisingly tricky to find decent resources, especially when mapping textures and geo coordinates onto a sphere. ive developed a nice basic example which i think is a good start for anyone tackling an earth application on the iphone using openFrameworks or even working with Objective C.

the example contains code for drawing a sphere in openGL ES and giving it an earth texture which is provided in the source. theres a Trackball class which allows you to intuitively rotate the textured sphere and a CurveHop class which draws a curved ribbon from location to location as it hops along on the surface of the earth. theres a cool little trick you can do in google maps to get the latitude and longitude of any location, so you can add it into the curve hop chain. just search for a location in google maps and copy the following code into your address bar,

javascript:void(prompt(”,gApplication.getMap().getCenter()));

this will bring up a popup with the latitude and longitude of the center of the map.

download the source code.

my two favorite things, abstract geometry and audio reactive visuals.
working on the new beck’s beer site was a treat with a great base design and massive scope for innovation.

i won’t bang on about the creation process but i will say that the cube was modeled using blender, exported as a DAE file, painlessly plugged into papervision and off we went…
if you are new to blender like i was i recommend reading through the manual, its the best place to start and will allow you to tinker and create as you are learning.

had a few days spare towards the end of the project so thought it would be nice to add an audio reactive element to the site. it is the beck’s beer brand and it is heavily associated with electronic music so i reproduced the geometry of the cube segments and added them to a 3d backdrop layer… the space junk fragments float in darkness, illuminated and pushed on by the beat of the music.

still trying to work out what to make open source from this project so if theres anything you’d like me to release, please let me know and ill do my best to post it on here.

on the weekend i started working on my latest piece in processing titled cilindrico collapse. had a good run, getting everything done i had planned to do. main things were, coding the 3D motif which ive named arc bars, and after many painful attempts ive finally worked out how to use pure opengl lights inside a processing sketch. so now i can have all the pretty shapes and colours, with shading!

ive also inculded the ability to export the sketch to a amazingly large proportions using marius watz’s tile saver class. perfect if you want to print your work.

thought id make the source code available now, before it gets messy with experimentation.
you will also need toxiclibs to get it running.
enjoy.

in short, they allow for intuitive rotation around each axis.
normally if you rotate around one axis and then around another, you get a combination of those two rotations and its not really where you want to be.
quaternions allow to rotate around each axis without effecting the rotation of the other axis, so imagine a orientating a aeroplane model in a 3D flying game…
an aeroplane can be rotated around each axis and these angles are referred to as roll (x-axis), pitch (y-axis) and yaw (z-axis).

roll is then the aeroplane direction moving up and down.
pitch is the aeroplane direction moving left to right.
and yaw is the aeroplane swaying from side to side.

so finally worked out the mathematics behind this by reverse engineering some existing arcball code.
below is the processing implementation done with pure opengl, it is using some handy classes in toxi’s geomutils.

the steps are,
1) work out the rotation vectors around each axis, simply by rotating the vector around an axis.
2) work out the rotation quaternion for each axis, by getting the dot and cross product of each rotation vector from its origin.
3) multiply the rotation quaternions of each axis. – this will give you the overall rotation quaternion.
4) convert the rotation quaternion into a matrix and apply that rotation matrix.
5) draw your aeroplane! done.

			float DEGTORAD = PI / 180;

			Vec3D xrot, yrot, zrot;
			Quaternion xrotQuat, yrotQuat, zrotQuat, rotQuat;
			Matrix4x4 m;
			float angleX, angleY, angleZ;

			angleX = angleDegreesX * DEGTORAD;
			angleY = angleDegreesY * DEGTORAD;
			angleZ = angleDegreesZ * DEGTORAD;

			xrot = Vec3D.Z_AXIS.copy().rotateX( angleX );
			yrot = Vec3D.X_AXIS.copy().rotateY( angleY );
			zrot = Vec3D.X_AXIS.copy().rotateZ( angleZ );

			xrotQuat = new Quaternion( 1, new Vec3D( 0, 0, 0 ) );
			xrotQuat.set( xrot.dot( Vec3D.Z_AXIS.copy() ), xrot.cross( Vec3D.Z_AXIS.copy() ) );

			yrotQuat = new Quaternion( 1, new Vec3D( 0, 0, 0 ) );
			yrotQuat.set( yrot.dot( Vec3D.X_AXIS.copy() ), yrot.cross( Vec3D.X_AXIS.copy() ) );

			zrotQuat = new Quaternion( 1, new Vec3D( 0, 0, 0 ) );
			zrotQuat.set( zrot.dot( Vec3D.X_AXIS.copy() ), zrot.cross( Vec3D.X_AXIS.copy() ) );

			rotQuat = xrotQuat.multiply( yrotQuat ).multiply( zrotQuat );
			m		= rotQuat.getMatrix();

			FloatBuffer rotMatrix;
			rotMatrix = ByteBuffer.allocateDirect(4 * 4 * 4).order(ByteOrder.nativeOrder()).asFloatBuffer();
			rotMatrix.put
			(
				new float[]
				{
					(float)m.matrix[0][0], (float)m.matrix[0][1], (float)m.matrix[0][2], (float)m.matrix[0][3],
					(float)m.matrix[1][0], (float)m.matrix[1][1], (float)m.matrix[1][2], (float)m.matrix[1][3],
					(float)m.matrix[2][0], (float)m.matrix[2][1], (float)m.matrix[2][2], (float)m.matrix[2][3],
					(float)m.matrix[3][0], (float)m.matrix[3][1], (float)m.matrix[3][2], (float)m.matrix[3][3]
				}
			);
			rotMatrix.flip();

			gl.glPushMatrix();

			gl.glTranslatef( loc.x, loc.y, loc.z );
			gl.glMultMatrixf( rotMatrix );
			gl.glTranslatef( -loc.x, -loc.y, -loc.z );

			// draw here.

			gl.glPopMatrix();

the installation was for our xmas party themed gotham so i thought id put my flocking code to some good use and apply it to some winged rodents… no, to be fair i really like bats… have masses of them flocking over my house each evening to their nightly feeding spots and always enjoy the spectacle.

bats_01 from lukasz on Vimeo.

back to code speak, getting the bats to orientate towards the direction of flight was a tricky exercise in opengl.
in theory all i wanted to do was to rotate the bat around the Y axis (left or right) and around the Z axis (up or down)… easy enough, but that’s not the way it works in opengl. if you rotate the coordinate space around one axis, then all the other axis are also rotated.
with help from flashmonkey the solution was found in using a rotation matrix and a few other vector calculations.
there are probably many ways of arriving at the same result but this is what worked for us.

here is the source
also will need toxi’s geomutils library if you haven’t got it already.

here are some of the results, the first one is using my favourite jelly video that i break out at every occasion.

ribbon jelly from julaps on Vimeo.

this one was using a vj loop of space invaders, strong primary colours and you can see how it creates a completely different texture.

ribbon invaders from julaps on Vimeo.

thought id also share the source and really like to hear from you if you do something cool with it. ive exported the source straight out of eclipse which is my IDE of choice for working with processing, so it might take a few minutes to copy and paste it into the processing IDE but its pretty straight forward.
im also using toxi’s awesome geomutils library which you’ll need to run the sketch.

but what if you wanted to create a bespoke interface, something tangible and intuitive like a lever, a pulley rope or a valve to control what is happening on the screen. well then, you are now transcending into a strange world of resistors, currents, breadboards and smells of burning solder. luckily the folks behind the arduino microcontroller have made these electronic baby steps much easier and enjoyable.

a couple weeks ago i took part in a arduino workshop set up by little bird electronics here in sydney and set myself a very modest goal of changing something on screen using this little chip. i was ecstatic at the ease of use of the arduino board and within an hour i was twisting a variable resistor and changing colors in a processing sketch. mission accomplished! even though this is equivalent to a ‘hello world’ exercise in programming, it was still very gratifying to bridge these two worlds together and knowing i can apply this board to any future interactive projects.

here is a great example of work done by theodore watson ( muonics ) for a michel gondry exhibition. check out how a rope is used to open an close the characters eyelids.

over the last couple weeks I’ve been experimenting with slit-scanning, a technique which works on exposing different times of a video in the one frame, making it possible to glimpse into the past and future of the video. the above is an audio reactive piece ive built in processing where the video becomes temporally distorted by the audio. i didn’t want to get carried away ( as i often do ) so i limited the effects only to slit-scanning.

decomposing time from julaps on Vimeo.

in this sketch time moves across in a sine wave, exposing the past and present of the video in a smooth, undulating motion. the original video is of a mushroom decomposing, slowly decaying over time. by applying the temporal distortion the mushroom looks more like it is being blown away by an explosion of sorts. i can see many ways this technique could be used to create special effects in video, first by recording a time-lapse of something growing, expanding, decaying, disappearing etc, and then applying some form of time distortion to create the final effect.

here is the source for this sketch, its all pretty straight forward but churns out some nice results.
please note, you have set maximum available memory to 1024 in processing -> preferences.

temporal vortex from julaps on Vimeo.

the technique can be extended to change the video image on a per pixel basis. the temporal vortex sketch creates a hole in the video which allows to see into the future as it pours out into the present. there really is no limit on the types of distortion and final output you can create.

flong writes a great paper on the history of the technique.

been doing some experiments with QR (Quick Response) codes at work which are basically 2D bar codes and vaguely resemble conway’s game of life. they are convenient. take a snap of them with your mobile and the code is decrypted to something meaningful like a sentence, a link or even a clue to a treasure hunt… a great way of transmitting small amounts of data from the physical to the digital.

aesthetically the codes remind me of tile art, remnants of space invaders and have a certain binary beauty about them. i also like the idea of some meaning emerging from an encrypted message thats open to the public but only in reach to those that know what to do with it. and on a complete tangent, its what id imagine the robot army would be posting up over city walls if they ever took over the world, proclaiming that the humans are dead.

so ive decided to revisit the days of the throw-ups, sticking up your little piece of art on the back of street signs with the hope that someone has a chuckle. might as well make the most of this novel technology before it explodes commercially.

generate your code here.