NPR ran a story on Monday’s Morning Edition entitled “Security Officials Seek to Block Some Online Maps”. The story centered around local government officials refusing to release electronic maps of what they call “critical infrastructure,” such as water mains and fire hydrants. Specifically the story of Steven Whitaker’s futile quest to obtain infrastructure data from the Greenwich, CT local GIS repository. As part of the story NPR came by to ask my opinion on the matter because of our history of creating security concerns using open source data.

The story has a nice quote of me saying it was an impossible task to try and control all the geodata out there and who has access to it. The part that did not air is that no one even knows what data is accessible and not accessible to the public. While we do have a good index and census of most of the web pages that exist, we have much less understanding of the databases including geospatial databases connected to the Web (often called the Deep Web). The indexes run by Google and others do a great job finding web pages but databases are a different game. A Cal Berkley study by Bergman found that, “the deep web consists of about 91,000 terabytes. By contrast, the surface web, which is easily reached by search engines, is only about 167 terabytes.” While it is uncertain how much of this data is geospatial in nature it is fair to assume it is a considerable amount of data that we largely have little clue about. Often times government agencies do not even realize what data they have online available to the public, and we definitely do not have a comprehensive way to understand the entire universe of geospatial data. What raised so much alarm with our original research were the authorities realizing that that the data was available open source. Everyone clamored the work should be classified, but the source data is all still out there hidden in myriad local, state, federal and NGO data repositories. This begs the question, how are we going to control a world of data that we have so little comprehension of?

In order to move towards greater security I believe we actually need to open up more so that the entirety of geospatial data can be indexed. We will have no true idea as to what geospatial data available to the public is potentially dangerous until know what is out there. The move towards making KML an OGC standard is a great first step as a standard geospatial data format for the Web. Although KML natively is geared towards providing a geographic framework for text, html, pictures etc., and not structured information like databases. We’ve been working on changing that by ensuring a mechanism exists by which to include feature attribute data in the schema tag of KML . Some of this work has carried over into KML 2.2 as “extended data“.

Once you begin to index the geospatial data out there you are in a much better position to have a logical debate about what data is a security threat and what data contributes to the public good. For instance you may want to know where there have been hazardous pipeline accidents, but not divulge where critical pipeline routing junctures are. By opening up geospatial data, not only do we have a foundation to better insure dangerous data stays out of the hands of bad guys, but we also have the positive externality of a whole wealth of data being made available to the public to solve a wide range of problems.

Read the rest of this entry »

Popularity: 14% [?]

One of our core missions at FortiusOne is to enable many more people to explore, create, and share maps – to democratize geospatial capabilities. Even in government markets, which have been big users of GIS tools, the expense and technical sophistication required often cause bottlenecks in the preparation of maps. Government customers are also facing critical challenges in making information from the field accessible throughout their organizations – increasing collective intelligence from the edges of the network.

We have teamed with Lockheed Martin to leverage our intelligent mapping services to address these problems in government markets. Lockheed has a long history in the geospatial space and has been very progressive in embracing advanced Web 2.0 technologies such as Intelligent Mapping and Wikis.

I thought it might be helpful to provide an example of the kinds of problems we are addressing with Lockheed. Let’s take a fictitious scenario of a military operation dealing with terrorist attacks in Iraq. Suppose I'm Sergeant Gorman and I've uploaded data on a spree of attacks that my patrol collected over the past week.

A GIS analyst at headquarters, in reviewing my data along with historical data from the last three years, notices a pattern of increasing Shia activity around Samarra and sends an alert to field units. The alert prompts me to scan for data on attacks tagged Shia and Samarra, where I find a photo from a previous attack that shows one of the locals we had suspected of being a Shia ring leader.

I post a geo-blog noting that this individual has been suspected of coordinating attacks in my sector. A flurry of responses from other patrol leaders indicates that the same individual has been seen in proximity of other attacks. A GIS analyst at headquarters validates the findings and generates a command report, which results in the order to apprehend the suspect. On our next patrol into Samarra, we locate him and discover a complex cell of terrorist Shia activity in the area.

While the account above is completely fictional, hopefully it conveys the power of democratizing geospatial information throughout an entire organization. The same principles apply to a variety of other environments, such as disaster response, homeland security, and intelligence, where enabling the entire organization to explore, create, and share geospatial information can enhance mission effectiveness. We are excited about the partnership with Lockheed Martin to bring these capabilities to market.

Popularity: 13% [?]

One of the natural questions to follow a tragic event like the I 35W Minneapolis bridge collapse is where are there other bridges that could suffer a similar fate. In our last post we talked about the National Bridge Inventory, the valuable information contained in it, and the difficulty working with it.

My original hope was to load up a dataset with lat longs for the dangerous bridges across the country, but after Raj spent many hours trying to sort out the geo-coding inaccuracies in the data set it became apparent that would not be possible. The latitude and longitude coordinates simply did not map to reality, the entire state of New Jersey was missing coordinates, and many time the last two digits of resolution were zeroed out. So, Raj aggregated the data to the next best level of resolution - counties. The bad news is you cannot tell where in that county the dangerous bridges are, but you can tell which counties have the riskiest bridges and if it is one you drive through. We are still working on trying to derive a finer grain picture and will definitely post those up if we can come up with something accurate.

Where are the the most dangerous bridges:

The top 5 most dangerous counties and the total number of dangerous bridges are:

Garfield Oklahoma 78

Attala Mississippi 45

Allegheny Pennsylvania 42

Washington Pennsylvania 37

Montgomery Pennsylvania 36

So, how was this all calculated? First we took the National Bridge Inventory and grabbed the safety ratings for all bridge's superstructure, substructure, and decks (found here):

"The NBI database contains ratings on the three primary components of a bridge: the deck, superstructure, and substructure. A bridge deck is the primary surface used for transportation. The deck is supported by the superstructure. This transfers the load of the deck and the traffic carried to the supports. Within the superstructure are the girders, stringers, and other structural elements. The substructure is the foundation of the bridge and transfers the loads of the structure to the ground. The superstructure is supported by the substructure elements, such as the abutments and piers."

For each of the key bridge components, decks, superstructure, and substructure we provided counts by county for the number of bridges by their safety ratings (ranging from "failed" to "excellent"). We also created three indexes: 1) dangerous bridges (the sum of the number of failed, imminent, critical, and serious bridges), 2) risky bridges (the sum of the number of poor, fair, and satisfactory bridges) and 3) safe bridges (the sum of the number of good, very good, and excellent bridges).

The embedded map show the number of dangerous bridges rated by their superstructure. In the Minneapolis bridge collapse the superstructure was rated "poor" and all the bridges mapped are bridges in worse condition by safety rating. This is just one slice of the data. The best way to see how your location rates is to head to GeoCommons and create a map by clicking the "mymaps" tab and adding one of the three datasets. Just search "bridges" or by type "superstructure", "substructure", "decks". Once you've made the map you can also click on the "about this dataset" link and create top ten lists for whatever data attribute you think is most critical. We hope this allows a mechanism for the public to discover if there are dangerous bridges in their backyard.

Popularity: 16% [?]