Michael Edgcumbe and I gave a presentation at the O’Reilly Strata Conference last month that takes stock of the myriad (and amazing) framework options available to creative coders and data visualization developers.
The slide deck, and the example zoo are online. Thanks to everyone who attended and provided feedback.
Also, the source code for Michael’s sample app in Processing is available on GitHub.
Lots of nice work at the ITP winter show this year.
I still think a little more curation to prune the “hello world” projects wouldn’t hurt, but such is ITP.
Here are some personal favorites:
Zach Schwartz, Martín Bravo, Nara Kasbergen, Rachel Slaff, and Ruth Spencer
Brilliant mashup of Google Maps and the not-as-famous-as-it-should-be Your World of Text. Written world lets you leave persistent messages on a map in real time. I really think it needs an address search bar to make it easier to explore other parts of the world — and being able to zoom out and see the density of conversation around the globe would be amazing. Regardless, it’s a great idea and the execution is polished. Go try it!
Matt Richardson
A servo-mounted laser methodically draws text (tweets) on a phosphorescent screen. The glow-in-the-dark screen holds a record of the laser’s path. So nice to get away from flickering screens (even if we can’t get away from tweets).
Lia Martinez, William Jennings A tiny catapult as an interface for cueing projected clips of bananas hitting a wall. Simple, but instantly gratifying.
Gavin Hackeling, Zena Koo
Wonderful side-stepping of QR-driven AR by using real images as tags in lieu of robot barf. Watch the video.
Luis Palacios, Kaitlin Till-Landry, and Peter Darche
An XY table that draws in a pseudo-zen garden sand pit. Instead of putting a pen or pencil on the business end of an XY table, SoZen’s creators use a strong magnet to drag a ferrous ball through the sand to build up a drawing. Making an XY table is a PComp rite of passage, but this is a great take on it and a nice use of magnetic indirection to keep the mechanism out of sight and mind.
Ivana Basic A plaster sculpture slowly destroys itself from the inside out in the absence of attention. More art should behave like a suburban adolescent.
Kate Watson
Finally, someone braves the ITP thesis archives. It’s the only real hope for deconstructing what, exactly, ITP is / was, while removing all excuses for creating the same projects over and over. Looking forward to seeing the evolution of Archived Visualizations next semester.
It’s an animation built from a tumblr blog documenting Kim Jong Il looking at things.
Animated Gif (Single Frame) – Eric Mika, 2011 It’s a single frame animated gif.
The file was hand written, per the GIF file specification. Although the original specification accommodates basic animation, this particular file uses an extension of the original format created and first supported by Netscape 2.0. Netscape’s addition remains out of the official spec — but seems to be the approach favored by most animated gifs around the web. While Animated Gif (Single Frame) may not follow CompuServe’s canonical format to the byte, it’s aligned instead with the vernacular of the real web. It renders, after all. The above image is simply solid black, and won’t appear to animate, but the file is structured in a way that identifies itself as a looping animation. Therefore, the parser rendering it in your browser is actually cycling between Frame 1 and Frame 1 every second, indefinitely.
It is intended as a “last work” for an exhausted medium.
Source code:
47 49 46 38 39 61 00 02 56 01 80 00 00 00 00 00 FF FF FF 21 FF 0B 4E 45 54 53 43 41 50 45 32 2E 30 03 01 FF FF 00 21 F9 04 04 64 00 00 00 2C 00 00 00 00 00 02 56 01 00 02 FF 84 8F A9 CB ED 0F A3 9C B4 DA 8B B3 DE BC FB 0F 86 E2 48 96 E6 89 A6 EA CA B6 EE 0B C7 F2 4C D7 F6 8D E7 FA CE F7 FE 0F 0C 0A 87 C4 A2 F1 88 4C 2A 97 CC A6 F3 09 8D 4A A7 D4 AA F5 8A CD 6A B7 DC AE F7 0B 0E 8B C7 E4 B2 F9 8C 4E AB D7 EC B6 FB 0D 8F CB E7 F4 BA FD 8E CF EB F7 FC BE FF 0F 18 28 38 48 58 68 78 88 98 A8 B8 C8 D8 E8 F8 08 19 29 39 49 59 69 79 89 99 A9 B9 C9 D9 E9 F9 09 1A 2A 3A 4A 5A 6A 7A 8A 9A AA BA CA DA EA FA 0A 1B 2B 3B 4B 5B 6B 7B 8B 9B AB BB CB DB EB FB 0B 1C 2C 3C 4C 5C 6C 7C 8C 9C AC BC CC DC EC FC 0C 1D 2D 3D 4D 5D 6D 7D 8D 9D AD BD CD DD ED FD 0D 1E 2E 3E 4E 5E 6E 7E 8E 9E AE BE CE DE EE FE 0E 1F 2F 3F 4F 5F 6F 7F 8F 9F AF BF CF DF EF FF 0F 30 A0 C0 81 04 0B 1A 3C 88 30 A1 C2 85 0C 1B 3A 7C 08 31 A2 C4 89 14 2B 5A BC 88 31 A3 C6 8D FF 1C 3B 7A FC 08 32 A4 C8 91 24 4B 9A 3C 89 32 A5 CA 95 2C 5B BA 7C 09 33 A6 CC 99 34 6B DA BC 89 33 A7 CE 9D 3C 7B FA FC 09 34 A8 D0 A1 44 8B 1A 3D 8A 34 A9 D2 A5 4C 9B 3A 7D 0A 35 AA D4 A9 54 AB 5A BD 8A 35 AB D6 AD 5C BB 7A FD 0A 36 AC D8 B1 64 CB 9A 3D 8B 36 AD DA B5 6C DB BA 7D 0B 37 AE DC B9 74 EB DA BD 8B 37 AF DE BD 7C FB FA FD 0B 38 B0 E0 C1 84 0B 1B 3E 8C 38 B1 E2 C5 8C 1B 3B 7E 0C 39 B2 E4 C9 94 2B 5B BE 8C 39 B3 E6 CD 9C 3B 7B FE 0C 3A B4 E8 D1 A4 4B 9B 3E 8D 3A B5 EA D5 AC 5B BB 7E 0D 3B B6 EC D9 B4 6B DB BE 8D 3B B7 EE DD BC 7B FB FE 0D 3C B8 F0 E1 C4 8B 1B 3F 8E 3C B9 F2 E5 CC 9B 3B 7F 0E 3D BA F4 E9 D4 AB 5B BF 8E 3D BB F6 ED DC BB 7B FF 0E 3E BC F8 F1 E4 CB 9B 3F 8F 3E BD FA F5 EC DB BB 7F 0F 3F BE FC F9 F4 EB DB BF 8F 3F BF FE FD FC FB FB 6D FF 0F 60 80 02 0E 48 60 81 06 1E 88 60 82 0A 2E C8 60 83 0E 3E 08 61 84 12 4E 48 61 85 16 5E 88 61 86 1A 6E C8 61 87 1E 7E 08 62 88 22 8E 48 62 89 26 9E 88 62 8A 2A AE C8 62 8B 2E BE 08 63 8C 32 CE 48 63 8D 36 DE 88 63 8E 3A EE C8 63 8F 3E FE 08 64 90 42 0E 49 64 91 46 1E 89 64 92 4A 2E C9 64 93 4E 3E 09 65 94 52 4E C9 4A 01 00 21 FE 2E 41 4E 49 4D 41 54 45 44 20 47 49 46 20 28 53 49 4E 47 4C 45 20 46 52 41 4D 45 29 20 2D 20 45 52 49 43 20 4D 49 4B 41 20 32 30 31 31 00 3B(Bytes responsible for animation in red.)
Flapping Toasters is an absurdist reconsideration of the classic screensaver Flying Toasters. You embody the toaster, flapping your arms to stay aloft, clapping to release toast, and tilting to roll away from danger. Created in collaboration with Michael Edgcumbe.
Clockwise from top left: Basic flapping / Multi-player / Tilting to roll / Clapping to release toast.
The screensaver is forbidden territory for those seeking to elevate the status of new media art. A screensaver is lowbrow, unserious, and fetishizes aesthetics over content. I’ve explored this hostility a bit further in another post. At the same time, early screensavers represent a raw, unpretentious play with the screen’s possibilities combined with a novel set of constraints. (E.g. a screensaver must have no static content, run for an unlimited duration, etc.)
Michael and I decided to embrace the kitsch, and compound the absurdity of the ever-criticized screen saver with the hype and widespread obsession with the Kinect. Beyond trying to show some love for the screensaver, we were conscious of how many camera + presence driven “interactive artworks” present an interaction model that reduces the participant to maniacal arm waving and overwrought gestures designed to satisfy a finicky computer vision algorithm. We wanted to poke some fun at this trope by creating an interaction model that explicitly demands that the participant is reduced to crazed arm-flapping.
We built the software in openFrameworks. Since we wanted skeleton tracking, we used the ofxOpenNI plugin instead of ofxKinect (since the latter only provides RGB-D data).
Detecting flaps was a bit trickier than expected, since there’s an issue of ranging, direction detection, correlating the animation to the arm position, direction reversal detection, etc. We ended up solving these issues with brute force by tracking velocity of the hand points, filtering heavily, and setting up a dynamic bounding system that would allow people with various degrees of enthusiasm / gesticulation to fly successfully.
We also added support for rolling in both directions. The player initiates a roll by tilting their arms in ether direction — we used an angle threshold on an imaginary line drawn between hand points to trigger it.
Life as a toaster would be unfulfilling without the opportunity to toast bread, so Michael implemented hand clap detection to trigger the release of golden-brown toast.
Ideally there would be some kind of system for recording sample motions, saving them, and then automatically firing an event when a similar motion is detected along with a confidence value. I’m sure this kind of thing exists on industrial-strength Kinect development platforms (and must have existed somewhere in the Wii’s development stack for ages). Would be great to have a similar library for OF or Processing… something for the to-do list.
OpenNI makes tracking multiple players relatively trivial, so we implemented support for up to 15 simultaneous flappers (OpenNI’s internal limit).
The most recent version of the ofxOpenNI plugin includes support for OpenNI’s native hand tracking algorithms. They weren’t available during our initial development, so we used the raw skeleton data instead. These algorithms seem a bit more reliable for locating hands then the skeleton points alone, particularly when hands cross the torso, go out of frame, and so on — essentially, it uses a more generous set of inferences to give you the best guess about hand position.
Since flapping toasters is all about hand tracking, it sounds like this new feature would be great. So, I tried a quick reimplementation of Flapping Toasters using these points instead of the end of the lower arm joint. It turned out not to work with more than one player, since I couldn’t find a way to associate two hands with a single person. For example, if three hands are detected, I had no way of knowing which two belonged to one person, and to whom the third might belong. There might be a way, but it wasn’t immediately obvious from the API.
The graphics are scraped sprites from After Dark’s original Flying Toasters. We created PNGs from each frame of animation and re-implemented the basic design of the screensaver.
The final element, which was surprisingly the hardest to get right, was the physics simulation. It’s your basic gravity / acceleration / velocity simulation, but getting the power of a flap just right was tricky. It still doesn’t feel quite like I imagined. Flapping is a bit too much work, but it’s also too easy to flap all the way off the top of the screen. The solution to this probably involves spending a bit more times with the sliders to get the numbers just right, as well as taking into account the vigor of the flap. (E.g. use some combination of average arm velocity during the down-stroke and the relative size of the flap as a coefficient to the flap power value.)
After Dark’s version of flying toasters included two possible soundtracks — Wagner’s Ride of the Valkyries, and a slightly less compelling theme song with karaoke-style lyrics. Michael and I opted for Wagner, and rendered out a midi version of the song as a WAV for use in openFrameworks. I’d have preferred to make the recording using the midi synths from a Macintosh of the appropriate vintage, but we just couldn’t get access to one in time for the class deadline.
After flapping for a few minutes, most people ask what they’re trying to do. We haven’t implemented any kind of game logic — it’s just you, the toaster, and the joy of flap-induced flight. In one sense it’s nice to keep things minimal. But if we really wanted a game dynamic, there are a bunch of possibilities.
Some kind of collision detection between players could make things more interesting. You’re also given an unlimited supply of toast. Perhaps you could try to “catch” un-toasted pieces of bread, carry them for an amount of time (toasting them in the process) before clapping to release. Carry them for too long, and you’ll burn the toast. (The horror!)
The application was written in openFrameworks. The source code is available on GitHub. See the repository’s readme file for additional notes on development / compilation. A Mac OS binary is also included in the GitHub repo.