i done a little sound reactive piece and while it didn’t have any problems performing in real time, as soon as i tried recording it, it got so heavy on the processor that it started dropping frames and was completely out of sync with the music. so i went looking and found a nice little solution that seems to be working for me which involves running fft analysis in a separate sketch, saving the fft data into a text file and then using this data in my piece. the idea is that on a frame by frame basis i can get the audio data for that frame, react the visuals to the audio data and save it out to a png. i then load the pngs as a sequence into quicktime pro and its nice enough to give me back a movie file. the final step involves overlaying the movie with the audio. i do this in final cut but every video editing tool features this functionality. i noticed a delay which is probably due to the time it takes for the audio to start playing before grabbing the audio data, but thats easily adjusted when overlaying the audio over the video… just move the audio layer up a few frames until its spot on.
good thing about this approach is that you are not limiting yourself to any particular sound library. ive used ESS and Minim in the past and each one gives you back a slightly different result in the fft data. in this particular case i used Minim because it pronounced the beat more over a wider spectrum and so it went better with the visuals.
arc rays source and audio data export source
noisefield from julaps on Vimeo.
Nice work here. Would you consider sharing the source? I assume you’re doing those arcs with a quad strip, and was wondering if there was an easy way to do this.
Either way, very nicely done. I also really dig enome and noisefield.
hey mark, the geometry code can be found in this sketch.
http://processing.org/learning/3d/geometry.html
You might try using MovieMaker – http://processing.org/reference/libraries/video/index.html
That will let you skip the annoying step of generating and composing thousands of PNGs. Use animation codec at best quality for compression free video.
Also setting framerate within processing to 29.97 will help with your audio sync.
[...] Check his thread for a simple method for sync P5 data and audio. Another good resource is here at the julapy blog. [...]
Very nice work!
I’d love to see how you accomplished the analysis in a separate sketch and using that with the “real” sketch…
the syncing can also be done by pausing the track after the analysis and let it play in the end of the loop again. in this way you can do really heavy rendering, eg. p5sunflow without getting out of sync