Foreign Policy’s Tuesday Map Blog

July 31st, 2008by Sean Gorman

Mookie came across a cool blog run by the Foreign Policy journal that features a new map every Tuesday. The maps cover not only mash-ups but also print cartography and all have a foreign policy angle of some sort. Since F1 spun out of GMU’s School of Public Policy it is very encouraging to see maps playing such a prominent role in arguably the top tier policy journal.

The breadth of maps and cartographic approaches is quite impressive. Everything from this nice cartographic map of crime in Leeds UK:

(nice Ordnance Survey base data but would have been cheaper to use OSM ;-))

To map mash-ups like this collection of pushpins illustrating where journalists have been attacked:

A nice resource if you can’t quite get enough map porn in your daily web surfing.

Popularity: 12% [?]

While getting ready to launch Finder! we had an internal debate whether or not to put limits on dataset downloading. There were several options, ranging from requiring a user to be logged in before they downloaded to limiting the number of downloads a user could make in a day. A lot of the argument centered around the value of raw data - echoing the O’Reilly manifesto that “data is the Intel inside“. This belief holds that the value of the NAVTEQ’s and TeleAtlas’s of the world is derived from the proprietary data they collected.

One side of the company felt that by not limiting access to data we were giving away the family jewels. The other side felt that open access was the best way to create a network effect for data by making it as accessible as possible. At the end of the day the open access philosophy prevailed, and from the sound of comments to James Fee’s post after GeoWeb, access to data is still an important facet to both GIS and GeoWeb users.

Now that Finder! has been out for a little while we’ve begun to see a big surge in downloads. I noted last week we hit 18,000 downloads and just a week later we are now over 28,000. This has caused us to take a second look at our access policies. “Knock on wood”, the system has scaled like a champ handling the traffic, but as we get ready to launch Maker! some concerns have come up about potential abuse and its effect on user experience.

The biggest concern is around systematic downloading of data and the potential for that to impact other users experiences on the site. The question is how to make the content available without impinging on the collective user experience. Wikipedia approaches this by making content available as one big tarball and asks users “Please do not use a web crawler to download large numbers of articles. Aggressive crawling of the server can cause a dramatic slow-down of Wikipedia. Our robots.txt blocks many ill-behaved bots.”

I’m not sure a giant tar ball of data is the best way to go for us, especially since the data is available in a variety of formats. A second option is to provide third party access to the data via an API. This API could also work for both download and upload. Andrei had an interesting suggestion in our last post:

“The two-way API will definitely help with the number of uploads. The cool thing to do, would be to add (”Add to Finder!”) a URL request:

…finder.com/add?file=file.kml&type=kml&name…”

If people have other ideas on how they could better access the data in bullk without impinging performance we’d love to hear them. Also thoughts on what the line is between fair use of content and abuse of the commons. It is a bit of gray line in my mind. Is systematic downloading (manually hitting every dataset) abusive? Is scraping datasets with bots abusive? The main goal in my mind is to provide the best service possible without creating a “tragedy of the commons“.

Popularity: 17% [?]

Links List 7.25.08

July 25th, 2008by Sean Gorman

Not only can you drive with Google Maps directions, you can now take a walk with them. Google added walking directions to its Google Maps product today. The walking directions ignore one-way streets and Google Maps tries to give pedestrians the most direct and flat route possible.

Speaking of Google Maps, CNET raises the question of whether or not Google Maps can (and should) be used for good or evil. One argument states that Google Maps helps awful people find you, but the other argues that Google Maps helps you find awful people. Then again, consider the 51 things you aren’t allowed to see on Google Maps. It looks as though some data isn’t as open as we’d like it to be. From government restrictions to personal-privacy lawsuits, there’s just some information that we can’t see on GoogleMaps.

Andy Powell asks, “does metadata matter” in a recent presentation. He provides history on metadata and wonders if his organization is justified in its current focus and significant interest in this area. Is yours?

Having trouble keeping crowdsourcing straight? Check out the many names of crowdsourcing GIS from GIS Lounge. From neography to collaborative GIS, it’s all covered.

WunderMap released a hurricane tracking service this week, which is perfect timing for hurricane season. WunderMap displays the current position, strength and movement of every tropical storm across the world that threatens to become a hurricane, typhoon or cyclone. For more in depth hurricane information, check out these Google Earth files from Hurricane Hunters. The files allow you to watch live hurricane recon missions, or see the results from recent missions.

Will mobile phone companies purchase GIS companies to offer conventional GIS services? It’s not clear now (and obviously won’t be the iPhone), but it will be interesting to see who’s on first. Has GIS really gone mainstream though?

Popularity: 18% [?]

When we took Finder! out of private beta July 2nd, Matt Constantine also added a tool for doing statistical analysis of data downloads. We’ve been running it for 21 days and thought we’d share a little bit of what we’ve learned with the community.

1) Users love to download and have done so 18,143 times, but are a bit less excited about uploading, only 775 times (roughly 4% contribution rate which really is not all that bad when it comes to web participation).

2) When it comes to downloading KML dominates:

3) When it comes to uploading Shapefile dominates:

4) Quite a few people look at metadata (5,164 views) but prefer FGDC to ISO 19115:

The stats draw some stark yet relatively obvious conclusions. A good percentage of the data users are interested in seeing, natively resides in GIS file formats, shapefile specifically, but users much prefer consuming this data in GeoWeb formats, specifically KML. This is probably not a surprise to anyone, but for those in the GIS industry this could serve as a meaningful number for how the market wants to consume data.

Also I think the metadata statistics could serve as an indicator that the broad market does have an interest in metadata and understanding the veracity and accuracy of data they consume. This has been a topic the GIS side has been quiet vociferous about and increasingly looks to be something the GeoWeb will need to address.

A second bit of trivia is the top ten most popular data sets by download:

1. STARBUCKS, Store closures in the lower 48 states, USA, 2008
2. Priceviewer.com, WalMart Store Locations, USA, 2007
3. bicycleshops.us , Bike Shops by address, USA, 2008
4. WITS, Shia Violence in Iraq, Iraq, 2004 - March 2007
5. CARMA, United States Power Plant Emissions, United States, 2000/2007/Future
6. HRSA, Critical Access Hospitals in the lower 48 states, USA, 2007
7. Census, State Government Tax Collections, USA, 2007
8. CDC, Outbreak of Salmonella Saintpaul from Tomatoes Update, USA by State or County, June 14th 2008
9. 500 Year Flood Plain - Johnson County, Iowa
10. Openstreetmap Geofabrik, German Subway System,Germany , 2008

Popularity: 17% [?]

Dataset of the Day: Starbucks Closure Data

July 18th, 2008by Bill Greer

Sometimes it seems like there is a Starbucks on every corner, and sometimes it’s true! It looks as if they have finally reached their saturation point and are now closing 616 stores throughout the United States. This Finder! dataset shows the locations of the closures. We also uploaded a dataset that shows the almost 9,000 Starbucks locations around the Globe. With this point data, you can see that many of the locations being closed are very near to other Starbucks locations. Perhaps it makes sense to close stores that would cannibalize your own market however, there are many other ways of looking at the problem. We aggregated the data out to the Zip code and to urban areas. In case you were wondering, here is a sneak peak of the locations most impacted by the closures:

By Zipcode
1. 89108 Las Vegas, NV (5)
2. 63103 St. Louis, MO (4)
3. 77102 Houston, TX (4)
4. 92101 San Diego, CA (3)
5. 63102 St. Louis, MO (3)

By Urban Area
1. Dallas Fort-Worth Arlington (25)
2. Los Angeles-Long Beach-Santa Ana (22)
3. New York-Newark (22)
4. Chicago (18)
5. Las Vegas (15)

Lastly, we decided to map out some of the Starbucks locations with a competitor to see if perhaps that played a role in the closure decisions. Below is a map from New York to Philadelphia showing Starbucks locations (transparent green dots) and Dunkin Donuts locations (transparent magenta dots). The black dots are Starbucks locations which are on the Closure list.

Starbucks

Popularity: 24% [?]