Tutorial Day at ETech - Stamen and Food Hacking
March 3rd, 2008by Sean Gorman
We’ve made our way to sunny San Diego for ETech and kicking the conference off with tutorial sessions (where you actually get to learn how to do the stuff presenters are forced to gloss over in 15 minute talks). Today we have six total tutorials in two blocks, so you get to pick two to participate in. I went for Stamen’s “Live, Vast and Deep: Web-native Information Visualization” and “Kitchen Hack Lab: Food Hacking for Techies”. The viz tutorial is right up our alley and the food hacking is purely for fun.
The Stamen presentation covered lots of good ground with several examples for creating useful visualizations of data. One of the first themes was “show all the data” whenever possible, so that you can create context for the user. If the user can see all the data they have a reference to start with - like the Zipdecode project. A great visual example of this is Curtis Jordan’s work visualizing volumes of trash. Here are the 2 million plastic bottle Americans consume every five minutes:
And zoomed in for perspective:
Two great examples of how visualizing large amounts of data can make a very dramatic point. Doing this on the web is more challenging, especially when you start to delve into very large data sets. Stamen has several innovative experiences with this whether it is Cabspotting or Oakland Crime Crimespotting. As we’ve been plugging along on the second generation of GeoCommons we’ve been wrestling many of the same issues - boiling down to how could you enable anyone to make a Crimespotting style map in just a few clicks with any variety of data. Democratizing the data is one side of the equation, but equally challenging is democratizing the ability to make dynamic visualizations of the data without needing to be a cartographer or a very clever Flash developer.
Regardless of the end audience - Stamen pointed out some very useful ground rules for data visualization capabilities:
HTML/Javascript - handles 100-1000 data points - loads in .1 seconds
Flash - handles up to 10,000 data points - loads in 1 second
Java/Processing - handles up to 100,000 points - loads in 10 seconds
OpenGL - handles upwards of 1,000,000 points - loads in 100 seconds
A fairly rough rule on thumb , and indicative to Stamen’s use of Flash and Processing. I’d heard a bit on Processing, and after the discussions and demos will definitely need to investigate more. An open question - is it a wash between Flash and Processing for practical web implementation? If Flash handles 10,000 points in one second and Processing handles 100,000 in 10 seconds does that extrapolate to Processing handling 10,000 points in one second as well? If that is the case is it essentially a wash between the two since no user is going to wait 10 sends for anything (yes we learned that the hard way ;-). We’ll be learning a lot more over the next couple of months and will share what we sort out.
Popularity: 11% [?]
March 6th, 2008 at 3:08 pm
Thanks for the positive feedback on the talk. I’m glad you guys were there, you have a really interesting platform!
Rules of thumb are tricky, especially when they’re based on my subjective judgement. I’ll try and clarify…
I should reiterate from the talk that the loading times I gave are made up times, meant to represent the ideal time to load the data (not the plug-in or software). And the number of data points isn’t raw capacity (this isn’t a benchmark) but rather a rule of thumb for what I would personally be comfortable manipulating on the respective platforms.
The point I’m making isn’t really about whether you prefer to develop for the web in HTML vs Processing vs Flash (vs Silverlight) etc – but about how choosing a more powerful platform on the web quickly gets hit by diminishing returns.
My point is that although the move from HTML or Flash to an applet seems like a step up in terms of raw performance, the raw performance isn’t any use to you until the data is there. This pretty much levels the playing field, and for me that means Flash is at the sweet spot.
Hope that makes sense!
March 10th, 2008 at 5:41 pm
Thanks for the feedback Tom. It has us convinced we just need to find a good Flash person. Know anyone in DC looking to work on a cool project? With some luck we’ll have a big chunk of public data available for folks to play with - both content and styles - fairly shortly. Great workshop - worth the price of admission by itself.
March 12th, 2008 at 4:02 pm
what about using flash and the effects of the search engine spiders to index your content? That should be worth looking into.
March 17th, 2008 at 3:58 am
King has a good point, however I don’t think it’s any different to current AJAX maps.
There are ways to expose your data to search engines if that’s what you care about. Progressive enhancement seems like the smartest principle here: start with a static map and an image overlay (for heatmaps) or tabular data (for pushpins), and then replace those images using javascript and flash as appropriate. It’s not easy, but sometimes it’s worth it.
Google just exposed their static map interface to the world… I’d love it if their default maps API examples swapped out a static image for the javascript one instead of just instantiating the javascript one alone.
Oh… if I hear of any good Flash developers in DC I’ll send them your way… so long as they don’t want to move to San Francisco
March 21st, 2008 at 10:10 am
Agree on the Google Maps API, and they are responsive to suggestions. We asked about the the ability to do static maps about six months ago and it eventually came full circle. Sure other folks asked as well and it did not hurt to point out that Yahoo! offered it already
October 22nd, 2008 at 10:53 am
[…] on the data rendering capabilities of a variety of approaches. The readers digest version of the workshop went something along these […]