When you do a general search in Finder! you now have the option to sort your search results by date and relevance. Here’s an example: Let’s start by going to the Finder! homepage and doing a search for ‘Bridges’.

Above we see the search results page. On the right hand side of the page you will now see that there are options to sort by relevance, sort by newest first, and oldest first (highlighted in red). This is especially helpful when looking for a dataset that was recently uploaded.

Now let’s discuss some helpful search tips. Let’s say that you want to search for data on pollution but you notice that many of the results lack proper metadata and you are unsure where the data is sourced from. To make sure your data is reliable and from a quality source try the following search technique.

When filling out metadata in Finder! after you upload your dataset there is a source field where you can indicate the source of your data. By entering into the Finder! search field ‘source: ______’ you are able to collect search results that only have the indicated source that you have entered. This will help filter out extraneous items and provide you with more focused search results. The example above searches for data from the EPA (Environmental Protection Agency), a well-known source for quality pollution data.

The same technique to search for a specific source can also be used to search for results for data from a particular GeoCommons user and for datasets that have a specific tag associated with it.

Above you can see that the search ‘user: Bill’ and the search ‘tag: pollution’. The user search allows you to search for data uploaded by a specific user and the tag search allows you to find data that has been tagged with a specific tag word.

Another unique search option in Finder! is the ability to cancel out certain search results. Let’s say that you are looking for data on the recent earthquake in Haiti but notice that you are receiving a lot of results about the earthquake in Chile. To make sure you get only results for Haiti try the following:

By putting the minus sign in front of the word chile in your search field you eliminate the possibility of receiving any data on Chile. This again is another technique to clean up your search results and allow you to eliminate extraneous results.

We are still in the process of making our Finder! search better here at FortiusOne, but hope these latest features are beneficial to a better search. Stay tuned to see new search features that will again better filter your search results to help you find exactly what you are looking for in our data repository.

My take away from the last couple of weeks is the convergence of the various forms of location based information, on and off the Web, are converging are rapidly. Disasters often act as crucibles to cause these forces to be accelerated. We've seen this in Haiti and now in Chile, where crowd sourced data is playing an integral role in relief efforts. It is a role, though, that complements the traditional sources of geospatial information. The largest impacts we've seen is in the fusion of the two sources. The combination has incredible value providing baselines, context and temporal adaptability - resulting in a malleable abstract that can be molded to solve a myriad of disparate challenges.

The malleable qualities of the fused locative Web are highlighted in disasters, but they are just as powerful over the long cycle. Platial was one of the iconic instigators to foment location as an integral part of the Web. Di-Ann's statistics tell the story "5 MM (million maps) have been created. Our members maps are visible on over 500,000 sites around the Web." "The People's Atlas" lived up to it's moniker - democratizing personal cartography at a scale not seen before. Some times the economic cycle and market don't grace innovation with good favor. This does not diminish what Platial accomplished and will continue to accomplish. The content and community created by Platial has provided a seed for the rest of us to reap the harvest from, and we are all in debt to it. When I say "seed" I mean it literally. The open and adaptive nature of the Web allows us to take the Platial seed and diffuse it across the online world. I believe we'll see Platial reborn in many new forms as a result.

Thanks to late night efforts by Andrew and the F1 engineering team Platial's data has been migrated to GeoCommons for access and redistribution. You can access Platial data through a dedicated link on GeoCommons. We are still working on details of retaining icon cartography and getting the data all migrated but stay tuned for more detail on the effort.

This video highlights just one example of a growing number of online services making geodata available as a feed:

Valentine's day is this Saturday so I thought I'd made a few maps in honor of this lovers holiday. Since one thing lovers do is travel, I made a map of the 50 most romantic places in the world...at least according to Travel + Leisure in 2005. Each icon on the map contains the name of a hotel recommended by the magazine for that place.

There may not be enough time to plan a trip to one of these places by Valentine's day so you might want to leave that until next year and just pick up some artesian chocolate. The next map shows the locations of the chocolate shops in the Metro DC area.

Italy has the reputation as the as being the most romantic country in the world, so I made some maps showing where in Italy one would have the best chances of meeting single men or women.

Before now I would have told these users that they would have to use complicated and expensive mapping software. This would allow users to combine spreadsheet data with the desired shapes that they want to view on their map.

I am now happy to announce that with GeoCommons you no longer have to rely on the ways of the past. Now FortiusOne has created the new feature of GeoJoin which allows you to move beyond points and easily visualize regions. Below is a walk-through of the process or click this link to view a video that will visually assist you.

First, I have a spreadsheet of data in excel. The data is of various States in the USA with a corresponding value associated with each State.

Now I want to take this data and visualize it proportionally as the actual shapes of the States on my map. So, after saving the excel spreadsheet as a csv file I then upload it into Finder!

After I upload the file I proceed to the next steps:

Pending layers list. Click Next.

In Step 2 of the upload process click “Join with a boundary dataset”. This is the step I choose to perform the GeoJoin process.

The next part of step 2 allows me to search the Finder! database to find the appropriate boundary dataset to join to the data in my excel spreadsheet. In this case I want to find a boundary dataset of States in the USA. I can either search for the right boundary dataset by searching in the search bar or I can use the categories on the left hand side of the page to navigate to the appropriate dataset.

After the appropriate boundary dataset is chosen, my next step is to choose what attributes in the datasets I want to join together. In this case I’m matching ‘state’ from my data with ‘State name’ in the selected layer. I pay close attention to the message on the right hand side of the box to see how successful my GeoJoin match is.

I proceed through the rest of the upload steps of review, describe, and then map. When making a map in Maker! I choose to map by visual theme and can now view my map proportionally as it appears below.

GeoJoin is a great new feature of GeoCommons. To see a listing of what boundary datasets are available in Finder search ‘referenceboundary’ in the search bar on the Finder! homepage. There is a wide range from international borders to neighborhood boundaries of cities in the USA. Check it out and start GeoJoining today.

OpenStreetMap in the
World Bank Haiti Situation Room

Historically, these tools have been useful for the public and media to cover an event - but the question often arose if they were beneficial to responders and citizens on the ground. It's been clear that tools such as OpenStreetMap have had a clear and positive impact on the response and recovery efforts. Organizations from the World Bank, to MapAction, to the Fairfax Search and Rescue teams have all been using OSM as a primary means of maps and routing. OSM volunteers have even been performing rough damage assessement and IDP camp identification.

While these tools have been useful for responders, there was still a disconnect on the availability of this data to the Haitian Government to access and assess the situations. The best data previously available before the earthquake is primarily from 2002, and at the same time most ministries are either non-existent or severely understaffed.

Bringing the Commons

Portable Hard drives with data
and map tools for the Haiti Government

A major hurdle, however, was that there were no local copies of the raw imagery data. Thanks to the incredibly hard work of a number of organizations, especially Georgetown, SDSU and Internet2, we were able to move the 1.25 TB of raw data from the Hypercube server in San Diego to Georgetown's servers here in DC in a little over a day. These were loaded onto a series of 6 USB, self-powered hard drives. We chose the self-powered drives so it didn't depend on external plugs or fluctuating power sources.

In addition, we built an offline version of the Haiti Browser that can run by double-clicking an HTML file to run in a browser off of a hard drive or USB stick. You can get the code from here to see how it's done. We also included Delta State's MGRS Atlases for printing map books, and the GPS map images that can be used on Garmin handheld units.

There will also be several engineers deploying with the World Bank to assist in the dissemination of the data, working with the Haitian Ministries and President on using these tools and handling requests for more analysis and data with CNIGS (Centre National de l’Information Géo-Spatiale), Haiti's GIS department.

Enabling Self-Sufficient Government

The goal is to provide the Haitian government with their own capability of using the available tools for situational awareness and decision making. Beyond the immediate response and recovery efforts they are now enabled to utilize these tools for long-term reconstruction and infrastructure. The capabilities and tools should be sustainable and the goal is to close the data loop within the government as well as between external data collection and Haitian data needs and collection. We are moving beyond the traditional problems of leaving disasters without their own data or ability to be independent of the temporary organizations. The entire project is a model of how crowd-sourced data and tool development can have a beneficial impact both on remote support as well as directly to local citizens and government.

In the US and Europe, we're seeing a growing embrace of Open Government and transparency sharing data to citizens for engagement and collaboration. In Haiti, the community is performing a reverse-Gov2.0 - they're providing data and information to the government to respond, and rebuild their society.

A tremendous thank you to the numerous individuals and groups that helped and provided tools or data: World Bank, San Diego State University / Calit2, Internet2, Georgetown University, DigitalGlobe, Delta State University, Sahaha, Crisis Mappers, OpenStreetMap, NOAA, Ushahidi, DevelopmentSeed, TelaScience, STAR-TIDES, CrisisCommons, USAID, GeoCommons, OpenSGI, GeoEye.

Crises often act as crucibles to distill problems rapidly and Haiti in my mind did this in spades. While CrisisCommons gathered an immense talent pool and many brilliant quick spin applications were created, there was no vehicle to sustain and integrate the applications with those in need. There were government agencies and NGO's that had need, and the community had useful solution, but it was incredibly challenging to connect the two together. A few of the issues I saw that created obstacles:

1. Communication - simply connecting technical solution with those in need, and knowing how to navigate the bureaucracy of the organizations with need. Fortunately great websites and wikis were built, media coverage was abundant and lots of volunteers were able to leverage their social networks to make connections.

2. Institutional competition - the problems that arise in crises are fairly well known. Not surprisingly many government agencies have spent money, often a lot, to solve these problems. For many innovations that come out of community efforts there is already an incumbent solution in place. Even if the community solution is better or complimentary, there will be institutional and vendor resistance. No one want to look like the schmuck whose multi-million multi-year project was just trumped by a group of hackers over a couple of weekends.

3. Contract vehicles - even if you get past the issues communication/awareness and institutional resistance there are funding problems. If there is not a mechanism to make funding available it becomes exceedingly challenging for an innovation to survive past the weekend volunteers and more importantly get adopted by the user community. This was seen at the most basic level with CrisisCommons during Haiti support efforts. A government agency saw a big need the community could address and wanted to get several of the Geo-Heroes on site to help. Just getting their travel and meals covered was a major feat of bureaucratic magic.

In order for volunteer tech efforts or app contests to succeed long term I think these problems need to be addressed. My half addled brain had a few ideas over the last couple of days. Communication is the easiest, but during a crisis it can be tough to quickly put these together. CrisisCommons went a long way in solving this and the result, I believe, was a far more useful response from the tech community that did make a difference on the ground. The phenomenal founders have some great ideas to institutionalize the efforts to make them repeatable and more efficient. On the app contest side where long range planning is an option there is less of an issue. Folks like iStrategyLabs and others have a well oiled machine playing that role for a variety of great efforts.

The bigger challenges comes from solving the problems of institutional competition and funding. To really capture the value of the innovation coming out of open data and community efforts there needs to be the ability integrate applications into the IT baseline of agencies and NGO's. This is the only way, in my opinion, that the innovations will be sustainable over time. Otherwise we end up with a junk yard of brilliant apps that never received the support they needed to make a difference. We run the risk of creating a self-congratulatory cycle that does not actually deliver the needed capabilities.

Over time even the best and most popular apps have inevitable costs associated with them. Maintenance, hosting, bandwidth, just to start with. Even in an open source model a little bit of funding can make a huge difference in the success or failure of a project. I think we can all agree the cream of the crop apps resulting from these initiatives deserve funding, which brings us to the next problem - institutional competition. The best apps solve well known problems in novel ways. One of the toughest tasks getting these innovations into an IT baseline is removing or integrating with the existing solution to the problem. In any institution there are employees and vendors with a vested interest to keep the status quo.

The flip side of this is the same employees and vendors have the all important "contract vehicles" which are the barriers to making change happen. Some may say this is a conflict of interest but it is the reality. So, what is a possible solution to it all. In short make it a win-win as much as possible for all involved.

Government (Big G) first needs to recognize the new model that is emerging. Haiti was a poignant example but more proselytizing is needed. Open data, open platforms and communities can out perform the status quo by orders of magnitude. In recognition "Big G" needs to make money available to seed these initiatives to cover basic costs. The app contests are a great examples of small incentives creating great return. For crises we should create "seed" slush funds that can quickly be accessed to help with community volunteer costs - like travel etc. Have a simple process for "Big G" to allocate "seed" money to the best projects that solve the most immediate problem. Create a simple application and have the process in place with all the relevant organizations ahead of time.

This is the easy problem. The tougher problem is how do you sustain the successful projects long term. This applies to both app contests and volunteer efforts around crises. The solution here requires more effort. "Big G" needs to allocate larger pots of funding to get community app innovations integrated into the baseline. Perhaps more importantly "Big G" needs to provide support and guidance to innovators. Support in the form of contract vehicles, and guidance in the form of how to navigate the bureaucracy of government procurement. We should also not forget support for basic small business skills.

The purist in me says a custom (simple) contract vehicle should be built through GSA for the effort and the SBA should set up guidance and grant programs to foster the business development side. The reality is this would take a long time and you would still be challenged by institutional competition, especially from the vendors. The solution here I believe is co-opting the vendors. Have the vendors (i.e. system integrators, beltway bandits, what ever you prefer to label them) partner with "Big G" to provide contract vehicles and have them provide integration support. "Big G" can run their typical vendor bake off's to select which firms become part of the program and allocate enough funding to make it interesting for them.

Some may consider this making a deal with the devil, but I think it is the compromise that gives the innovations and innovators the best shot at success and sustainability. Innovations by their nature are disruptive, and the more you can co-opt the status quo to embrace the disruption the higher the odds of the innovation surviving. Incentivizing the vendors to help incubate innovation coming out of the community is one of the most direct ways to combat institutional competition. It has to be a win-win for "Big G", vendors and innovators. The good news, in all this, for the citizens is we can get a much more efficient return on investment for tax payers dollars. Not only does innovation occur more rapidly, it gets deployed in ways that can help solve problems at much lower cost. Last but not least we get another driver for small business generation. An aspect that has been largely lacking from the broader stimulus response. This is something we've discussed as a positive externality of open data as a public good, but now we could have the momentum to seize upon it.

Last week President Obama presented the Congress and the American people with his examination of the state of the union after his first year in office. He outlined his achievements as well as some of his failures in the past year and presented his plans for the future. We thought we would make maps highlighting some of his major points in the speech. Some maps we made with datasets that are regularly updated in Finder! such as unemployment. For others we created new datasets and uploaded them into the database.

Recognizing that the economy is the one issue foremost in American’s minds, he focused much of his speech on the subject. Job creation and small business growth were major factors in his plan for improving the economy, citing the Stimulus Package as the engine for both.

The first map shows how the unemployment has changed in the past year. The green counties show areas where unemployment rates decreased since last November and the purple counties show areas with increasing unemployment.

It is no surprise that most areas of the US have seen their unemployment rates continue to increase but to put that into perspective, it is helpful look at the same scale for the 12 months change during the last year of the Bush administration (the next map). During Bush’s last year the unemployment also rose in much of the country however it appears that during Obama’s first year, more counties increased dramatically (by 4% or more) and less counties improved their unemployment rates at all. To address unemployment the president spoke of the Jobs Bill and a major component of that bill is to fund clean energy projects. The following map shows current wind energy projects in each state. If DC is going to promote more environmental projects around the US, than it might be interesting to see how green DC itself is. The next map shows commercial buildings and plants in downtown DC that are Energy Star approved. Another way that the President suggested he will boost job growth is by doubling our exports in the next 5 years and mentioned specifically trade with South Korea, Colombia and Panama. The map below shows the US Trade Balance for all countries for 2009. As expected we have a high trade deficit with China. It is interesting to see with what countries in Africa, Latin America and the Middle East we have a deficit or a surplus. This next map shows the total trade balance of each country in the world. The US clearly has the highest trade deficit. As part of the recovery act, Obama highlighted a high speed rail plan. These projects around the country should provide for at least some job growth. The next map shows where these projects will be on top of the unemployment rates. Although not the main topic in his address, health care did get a lengthy mention. The president reconfirmed his commitment to improving health care in this country, particularly for the uninsured and underinsured. The next map shows uninsured Americans by state. Check out these dataset and others in Finder!

When you open the csv, Finder! has already found the centroids of the polygons for you and you will see two columns for latitude and longitude. The next step is to save the csv and upload this to Finder!. Now you've got a file of points rather than polygons. You don't even have to change the attribute names because Finder! likes them in any format.

Another great option for the csv download is to sort based on one of the attributes. For example, if you want to see buildings by type, such as "collapsed" simply sort, select the collapsed buildings, copy and paste into a new csv. Then copy the header row and save. Reupload into Finder! and bam, you've got collapsed buildings as points in Maker!

Upload your own shapefiles and see what you can come up with using our easy download options.

Below are step by step screenshots of this simple process. Enjoy!