Toward the Bare Metal: From Flash to Processing, OpenFrameworks, and Beyond

July 8th, 2009

Note: this is being cross-posted, with additional content, from the Adobe Experience Design site, Inspire.

There is a common pattern in the contemporary interactive digital art world: artists’ tools are getting more hardcore under the hood.

A lot of the current stars cut their teeth (and still make their living) on Flash. I was a very active member of a messageboard in the late nineties geared towards web and interactive design and development – a distinction I share with many of the current stars. The messageboard (dreamless.org) is long gone, but most everyone is still in touch, still sharing code and competing or collaborating. It’s nice to a fairly constant community, even if their tools have changed.

A lot of people moved from Flash to Processing – a great Java-based framework and IDE – as their interests moved away from wide distribution (via the Flash player) towards lower-level access to the hardware. Being able to work (almost) directly with OpenGL, for example, was a huge benefit. Processing provides a set of capabilities that is different than that offered by the Flash player, making it a great supplement for digital artists.

These days, the most adventurous (Flash-using) artists are venturing outside of both Flash and Processing. They are leaving their virtual machines behind, diving down into C++, OpenGL, OpenAL, and raw C code. This is a scary place for a lot of digital artists, who often lack CS degrees. Instead of taking the usual CS approach of learning from the hardware up, touching higher level languages (like Java and Actionscript) and virtual machines (like the Java VM or Flash Player) dead last, artists start at the top and move down, stepping into dark, complex, fault-intolerant waters only when they need access to more powerful APIs or direct hardware access. Knowledge is built pragmatically instead of comprehensively – a fact that keeps their perspective unique and their work fresh.

The top toolset for the hardcore digital artists right now is OpenFrameworks. It’s a great C++ library with a passionate community, not at all unlike, or separate from, the Processing and Flash communities. This summer, The Barbarian Group will be announcing a (possibly open) framework with a similar purpose. Flint is a framework for C++ and Objective-C development, allowing cross-platform development that is still platform-specific (if that makes any sense), and includes support for a certain popular mobile device…

These frameworks are vital for the progression of digital artwork because they take very tedious, daunting tasks and boilerplate code and wrap them up in attractive, semantically-appropriate domain-specific languages. In other words, they solve for the user experience of the digital artist. They help minimize painful context switches and free artists up to focus on the art. Instead of trying to remember all the arguments and gotchas with creating double-buffered windows on different operating systems, the artists can spend time creating textures and thinking of new interaction models.

For a framework to succeed, it has to become almost invisible to the coder. This is true of all assistive technologies, and, in my opinion, all technologies should be assistive.

Check out Robert Hodgin’s newest work, using Flint.

  • Toward the bare metal, frameworks for digital artists:

    http://bit.ly/scEBa


    This comment was originally posted on Twitter

  • RT @tobyjoe: New cross-post from my week on the Adobe XD site: http://bit.ly/3jzvf4 – top level on the move from flash to java to c++ fo …


    This comment was originally posted on Twitter

  • Chuck McQuilkin
    Fascinating article. Thanks.

  • Toby Joe’s: Toward the Bare Metal: From Flash to Processing, OpenFrameworks, and Beyond http://bit.ly/byGTn


    This comment was originally posted on Twitter

  • The tools are getting more hardcore - where hardcore = lower level. The frameworks and abstraction layers, the semantics, are staying (roughly) the same.

    That's what *allows* folks to move lower without pain. Without even realizing it, sometimes... and on-demand versus forcing them to know every detail of their GPU driver.

    Does that make sense? I'm not saying that the artists are becoming more 1337, or that anyone should. Just that more power is being made available while keeping semantics relatively constant. It actually sounds like you and I have the same view.

    I know that plenty of folks have been hacking on hardware and software forever. I'm not implying that anything is new here, except that a particular group (those who originated as web-based artists) are moving into tech that many of them have always considered off limits, or difficult, or ugly (such as C++), and they're able to incorporate those tools because such great frameworks have emerged that allow them to get started quickly and take rough ideas to the CPU/GPU faster than if they were opening a text editor.

    Like I said - nothing scandalous or exclusive.

    A non-emotional (for some reason discussions about frameworks always seem to spark emotion and defensiveness in people) example is Grand Central Dispatch in Snow Leopard.

    Though OpenCL has been around for a while, many graphics/media programmers have avoided or ignored it. For those working in Objective-C and Cocoa, though, Grand Central Dispatch suddenly makes OpenCL not just accessible but semantically relevant. I think folks will start to naturally think in terms of multicore without having to really worry about thread management, hardware disparity, etc.

    Note that I am approaching this from a UX angle. Solving the UX of the digital artist means giving them more hardcore tools without making them suffer. That's what frameworks can provide. A diverse set of frameworks means that a particular artist can find the DSL that fits his/her thought process the best.

    Let me reiterate: the "hardcore" label wasn't being applied to the person under the banner of "lower-level = more hardcore = more capable = more worthy" or whatever may have come across.

    I'm saying that the tools are getting more hardcore - faster, lower-level, more complex - while keeping the DSL higher and more assistive.

    A good analogy is OS X. Over the years, we've had an almost constant user experience but the kernel and hardware drivers have gotten far more hardcore. Laptops with 8 cores? Jeez. Thankfully GCD means we can take advantage of them without really caring that we're taking advantage of them.

    Simple, really. Sorry for any confusion. Perhaps my big note box at the top giving context should have been bolder, or I should have explained what Adobe is up to. I hope it's more clear now :)
  • Thanks for the considered reply -- I hadn't noticed that this was an article for a flash article, it does make more sense in that context. If I'd have noticed that, I'd have kept quiet.

    I still don't agree with your argument though. If we were controlling logic gates manually, wouldn't that be more hardcore, not less hardcore?

    My point was that computer art has been around since computing, and all computer languages have been turned to the purpose of art. Artists were using C++ before openframeworks came along. For them openframeworks makes things less complex, by tying together diverse libraries into a coherent whole. This is a good thing.
  • Hi, Alex. I'm not sure what you mean about "writing about [myself]..."

    I'm definitely not one of the artists I'm referencing. I know a lot of the particular folks quite well, but my programming work is more about creating the tools than using them for art. I fail at art. :)

    I agree with everything you said, though. Where Flash is concerned, please keep in mind the purpose of this article: the Adobe XD site. I wanted to use Flash as a starting point because, programmatically, it *is* the starting point for that audience!

    Without getting too specific with the names (other than mentioning Robert and Erik), I was really talking about a certain set of folks that followed this path, expanding their toolsets along the way.

    I also agree that a language is more than a tool. It impacts the work deeply. That's where frameworks can be powerful: like languages, they influence the ideas, the implementation, and the elements of surprise (both good and bad). Instead of creating entirely new languages for people aesthetically displeased by, say, "raw" C++, frameworks can act as a sort of secondary layer to those languages, giving the ability to introduce new semantics.

    Think of the simple DSL in Processing: draw(), line(), etc. Those semantics probably influence the art as much as the power of Java and OpenGL.

    When an artist finds semantics that fit his or her thought, I believe the framework kind of disappears and lets them more efficiently transfer thought to code. When languages or frameworks get in the way – through bugs/gotchas, or simply through semantics, the framework *can* become a detriment.

    Andrew Bell, the lead visual architect at The Barbarian Group, has a sick amount of talent. He also enjoys coding in C++ exclusively, without much in the way of frameworks (other than STL and Boost). His brain just works best that way.

    Robert wants his tools to help him, but stay out of the way. The syntax in Processing has become so familiar to him that it's invisible. The best way to give him new, more powerful tools is to try to create a DSL that fits his brain while introducing new complexity under the DSL.

    The complexity is there, when he wants to dig in – but for many things he doesn't have to.

    This is true, I think with all frameworks. Hell, it's true with macros.

    It's really not a shocking or contentious article, and the central theme (that these new artist-specific frameworks let folks dig in deeper, as needed, than they otherwise would) applies to frameworks and libraries in all languages.

    In fact, it applies to all languages. Otherwise we'd be controlling gates manually.

  • Funny how some see flash as the genesis of digital art rather than a blind, momentary trough in a longer history, e.g. http://is.gd/1s8iD


    This comment was originally posted on Twitter

  • http://bit.ly/scEBa


    This comment was originally posted on Twitter

  • Aren't you in danger of talking about yourself using the persona of the world? Digital art started before flash, and digital artists have always used a wide range of languages, from machinecode up. Also I think a language is more than a tool.

    That said I agree that processing and openframeworks are fantastic and it's great that language environments made for/by artists are around now.

  • RT @walkandre According to @tobyjoe, the new pattern in digital art is artist tools getting more hardcore http://is.gd/1rsCO via @flight404


    This comment was originally posted on Twitter

  • RT @generatorx According to @tobyjoe, the new pattern in digital art is artist tools getting more hardcore http://is.gd/1rsCO


    This comment was originally posted on Twitter

  • So ready for a taste of Flint from The Barbarian Group. Processing-caliber APIs ‘for a certain popular mobile device.’ http://bit.ly/3jzvf4


    This comment was originally posted on Twitter

  • I’ll be interested to see Barbarian’s Flint Framework – http://is.gd/1rvMJ – though these days I’m coding in machine code (via @flight404)


    This comment was originally posted on Twitter

  • The flint framework certainly sounds promising – http://bit.ly/scEBa ( via @flight404 )


    This comment was originally posted on Twitter

  • RT @generatorx According to @tobyjoe, the new pattern in digital art is artist tools getting more hardcore http://is.gd/1rsCO via @flight404


    This comment was originally posted on Twitter

  • A brief teaser about Flint C++ framework @tobyjoe: http://bit.ly/scEBa (via @flight404)


    This comment was originally posted on Twitter

  • According to @tobyjoe, the new pattern in digital art is artist tools getting more hardcore http://is.gd/1rsCO (via @flight404)


    This comment was originally posted on Twitter

  • A brief teaser about Flint C++ framework @tobyjoe: http://bit.ly/scEBa


    This comment was originally posted on Twitter

  • New cross-post from my week on the Adobe XD site: http://bit.ly/3jzvf4 – top level on the move from flash to java to c++ for artists


    This comment was originally posted on Twitter

  • New cross-post from my week on the Adobe XD site: http://bit.ly/3jzvf4 – top level on the move from flash to java to c++ for artists


    This comment was originally posted on Twitter

blog comments powered by Disqus