I thought it would be fun to take a different angle on the virtual globe competition and look at the content and data made available by two of the players - Google and ESRI. From a technical perspective I think most would agree that ArcGIS explorer is pale emulation of Google Earth especially when it comes to user experience. I’ll put aside my gripes around difficulty in set up and version updates with Explorer and focus on content. The ESRI supporters have made the argument that ArcGIS Explorer is about access to GIS content and has a different mission than Google Earth. Actually, I would say the missions (exposing geographic data to larger audiences) are the same but the approaches are much different.

I’ll assume that some day ESRI will nail the technical side of Explorer, and look a little further down the road - to the content available for Google and ESRI’s thick client applications and how easy it is to access. As the GeoWeb evolves and we’ve rendered ever more amazing three dimensional worlds I think an increasing premium is going to be on the scope of data and content that can be delivered to these applications.

So where do the two ends of the GeoWeb spectrum sit on the topic? From my observations pretty far apart. For practical purposes I’ll break the comparison into four topics 1) data formats 2) data sources 3) data search and 4) data packaging

Data Formats

This is one category where ArcGIS Explorer ups Google Earth cleanly. Explorer allows you to load ArcGIS Explorer files (.nmf why they felt the need to create another proprietary file format that works with nothing else is beyond me), servers (WMS, IMS, file server), geodatabases, shapefile, raster, and KML. This is an impressive list, although a little bit less impressive when you consider that half are open standards KML, WMS and raster and the others are really ESRI proprietary formats at the end of the day. On the upside the raster support is quite extensive (30 or so different formats), although they require a spatial reference file.

On the Google side you have KML and KMZ (once proprietary now turned over to the OGC - jury is still out on how open it will be) for the free version of Google Earth, and within the free Google Earth 3-D warehouse support for SketchUp “.skp” and COLLADA “.dae”. In Google Earth Pro and Enterprise versions there is vector support for ESRI shapefile and MapInfo .tab, and for imagery support for TIFF (.tif), including GeoTiff, National Imagery Transmission Format (.ntf), Erdas Imagine Images (.img), Atlantis MFF Raster (.hdr), PCIDSK Database File (.pix), Portable Pixmap Format (.pnm), Device Independent Bitmap (.bmp). While this is an extensive list you have to pay to get the file support, in ArcGIS Explorer the support is currently free, so ESRI does have a distinct edge in the category.

Data Sources

In addition to supporting different file formats there is also baked in content for both applications. For Explorer you can access ArcGIS Online directly through the file server option, which allows you to access a file directory of cotnent. A second option is you can access the “Resource Center” website through the help tab, where you can download content in the .nmf file format. The “Resource Center” is definitely the more user friendly of the two with a nice user interface categorizing content into useful categories like, “imagery, street, physical features”.

On the downside the content is very limited - twenty four layers supplied by ESRI and four contributed by the community. ArcGIS Online has more content, but was a pain in the ass to access. You have to get a user name and password from ESRI, read through the incorrect direction to access it, then you get a list of UNIX style titles with abbreviations and underscores, like UNEP_WCMC_WDPA2006_2D. Not exactly user friendly, but you do get close to fifty additional layers of data once you jump through the hoops.

In Google Earth there are two sources of baked in data from the application, “Layers” including (terrain, geoweb, roads, traffic, 3-building, borders and labels, gallery, global awareness, and places of interest) and the ability to search for businesses. You can also click the “help” tab and be taken to the Google Earth Community web page. On the Google Earth Community page site alone there are 638,213 KML or KMZ files.

While finding this content (they are all file attachments to bulletin board posts) is pretty clunky and often frustrating, it is a LOT of content. Especially when you compare it to the four user generated files on the ESRI equivalent. The quality and source of this content/data varies wildly, and it is difficult to tell what is good and what is bad, but the potential is there. Actually the metadata support for both is pretty sparse. This is ESRI’s metadata for a .nmf imagery layer from the “Resource Center”:

“Displays satellite and aerial imagery at a 15m minimum resolution worldwide, and 1m resolution for the U.S. World boundaries, place names, and transportation layers are also included. Use this map to view man-made and natural features, or as a base map for overlaying associated data layers.”

For an ArcGIS Online layer this:

Layer Name: Imagery
Layer Source: ArcGIS Globe Service Layer
Layer Type: Draped
URL: http://services.arcgisonline.com/v92
Service: I3_Imagery_Prime_World
Sub Service: Imagery
LOD Tile Fetch: True
Hidden: False

Neither super useful.

Data Search

Neither application directly supports data search, but both have communities or services built around them that do. On the ESRI side there is a huge number of geospatial data repositories that have shapefiles in them and the search capabilities of them vary widely. Probably the largest is the Geospatial One-Stop that has a connection to most Federal geospatial data. Although in reality it is more often access to the metadata than the data itself. Still a large amount of content that could be conceivably viewed in ArcGIS Explorer, although largely disconnected in a large number of different repositories.

Google Earth has not only a good number of community aggregators like the official GE Community site above and unofficial like Mapufacture, GE Library and GLayers, but also the ability to search all KML files indexed on the web. I’ve heard numbers north of 10 million KML files indexed by this approach, but have nothing official. One way to search this content base is to type filetype:kml

Popularity: 51% [?]

The Utility of Maps in Hazard Forecasting

The recent wildfires in Southern California remind of us of just how important hazard forecasting has become in helping to ensure the safety and welfare of the public and the role that mapping can play in the process. Short-term forecasts of fire direction and intensity were pivotal in containment and evacuation efforts; Mapping played a prominent role in generating forecasts and in disseminating and sharing information about potential risk.

The usefulness of maps in visualizing and and generating forecasts extends well beyond the California fire event. In the area of climate prediction, numerous sites provide regularly updated maps of long-term and short-term forecasts of a variety of conditions and in some cases, valuable watches and warnings to the public based on the forecasts.

Some Points for Discussion

While the information that is currently out there provides great utility, there are some limitations in the way that the information is is disseminated and formatted that are worth noting. The points are intended to be food for thought and to get us thinking about how we can increase the value of forecasting even further - particularly in an interactive, web-based mapping environment.

First, forecasts are scattered across multiple websites and even within websites, requiring some effort and time on the part of the consumer to find, extract and process information. The sites and links vary in terms of the information they provide. In terms of fire forecasting, some sites focus on drought conditions, others on smoke generation and yet others on combinations of factors to characteristic future fire potential. The forecasting horizons also vary considerably from site to site.

Second, much of the maps provided on the web are in a “hard copy” format and not in an interactive mode where the user can pan, zoom and perform other functions. Some sites do have map viewers however, they are currently limited in the amount and type of data that can be displayed.

Third, and related to the second point, is that the possibility for “layering” data to create custom maps with richer information relevant to the needs of the user is limited. For example, someone may be interested in seeing if an environmentally sensitive or protected area is in the path of a projected wildfire.

Fourth, there lacks a mechanism for consumers and providers of the forecasts to interact and share information. Interaction could be very useful in understanding forecasts but also in terms of improving current predictive models. In the book Making Climate Forecasts Better, Stern and Easterling write: “The utility of forecasts can be increased by systematic efforts to bring scientific output and users’ needs closer together. These efforts may include both analytic efforts to identify the climatic parameters to which particular sectors or groups are highly sensitive or vulnerable and social processes that foster continual interaction between the producers and the consumers of forecasts.”

Fifth, not all information is publicly available and perhaps it should be? In climate forecasting, having access to the “best” information is in the national interest: it can save lives. And in some cases, the private sector is the keeper of such information. A recent study by ForecastWatch, found that in terms of recent historical forecasting of next day rain and snow, government sites had a 21% greater error rate than some of the private companies that do similar projections.

What Could the Future Hold?

The new web is fertile for the development of a system by which forecasts can be provided to the public in a more usable, digestible and efficient manner. Sites like Geocommons could be a one-stop location for viewing forecasts, such as those related to hazards and climatic conditions. In such an environment, visitors could interact with each other or the producers of the forecasts, discuss the validity of the forecasts or provide additional information to augment the projections, all through a wiki or blog-style environment. They could also create custom forecast maps with overlays of additional information that is of most useful to them for solving a problem, understanding a situation or simply planning ahead.

Popularity: 21% [?]

We've been playing around lately with some new data visualization techniques, so I started poking around to see what approaches have been generating buzz of late. In the process I came across Stamen Design's very cool "Oakland CrimeSpotting" via Brady's post on O'Reilly Radar. We'd been really impressed with what they did at Where 2.0 with Trulia and their latest further pushes the frontier.

Once I was done being wowed by the visual I found, Stamen had some thoughts similar to ourselves on the importance of public data being more accessible to...well the public. Specifically,

"We’ve found ourselves frustrated by the proprietary systems and long disclaimers that ultimately limit information available to the public. As citizens we have a right to public information. A clear understanding of our environment is essential to an informed citizenry."

That is the biggest mission of GeoCommons and it is encouraging to see other folks feel similarly. The sad thing is that in order to gain easy access to much of this data folks pay third party providers. One of our developers, Minh, passed along a list of data you can buy from Pushpin, including ESRI supplied Census data. Census data comes from the government, all our tax dollars pay for it - why do we pay twice? I'm not intending to pick on ESRI or Pushpin - the whole industry does it, and for many free data sources other than Census.

The good news is there has been a real ground swell to change the market. Whether it is OpenStreetMaps, ShapeWiki, or OpenAerialMaps crowdsourcing geodata or Swivel and WeoGeo opening up public data for easy consumption there is some real momentum to change the market place and business model. Specifically on the Census front Bill, aka Mr Data McFindsAlot, just posted up Census tract level data for the entire US by state. You can download any of it as KML or access it through our data and mapping API's. Census data, though, is just the tip of the iceberg of whats out there. My latest favorite "Pirate Attacks" a detailed dataset with lat long locations of real deal pirate attacks last year.

Popularity: 16% [?]

An interesting press release floated across GISuser yesterday on ESRI striking a strategic partnership with Jane’s to make their data available through ESRI tools. What I found intriguing was a service like Jane’s, which targets a non-technical audience, using ESRI, which has a largely technical user base, as channel to disseminate their data. The question it raises is ESRI working on something new to seriously expand their user base beyond GIS professionals, or is there enough demand by the US government (and others) for Jane’s data to be available to its GIS users to drive the deal.

The press release gives a few clues:

“For Jane’s electronic data, subscribers to its Web and desktop services can fuse and share the geospatial information in a common operating picture via ESRI’s software….For users new to the power of GIS, this easy-to-use capability will add considerable value to Jane’s products.”

So, is this a new “easy to use capability” or are they referring to ArcGIS 9.2 and ArcIMS?

Looking at the quote from Jane’s side is sounds as if the target is to make the data available to GIS professionals, “this partnership opens up a wealth of opportunity for Jane’s customers to view information in a GIS environment.” I think this brings up the compelling question of when does GIS include everyone, or when does web mapping (aka Google, Microsoft) encompass GIS. The line is definitely getting more blurry every day.

Popularity: 8% [?]