For the past 3 weeks, since January 12, we've been actively involved in many efforts to provide support and analysis of the Haiti earthquake recovery. From immediate OpenStreetMap efforts to get imagery open licensed and traced, to working through CrisisCommons to get new volunteers mapping, creating dashboard common operating pictures of population affected by the event, emerging IDP camp locations, distribution centers, and gathering normalized health facilities around the country.

Historically, these tools have been useful for the public and media to cover an event - but the question often arose if they were beneficial to responders and citizens on the ground. It's been clear that tools such as OpenStreetMap have had a clear and positive impact on the response and recovery efforts. Organizations from the World Bank, to MapAction, to the Fairfax Search and Rescue teams have all been using OSM as a primary means of maps and routing. OSM volunteers have even been performing rough damage assessement and IDP camp identification.

While these tools have been useful for responders, there was still a disconnect on the availability of this data to the Haitian Government to access and assess the situations. The best data previously available before the earthquake is primarily from 2002, and at the same time most ministries are either non-existent or severely understaffed.

Bringing the Commons

In response, at CrisisCamp DC on Saturday the World Bank initiated an effort to provide the massive amounts of data and tools directly to the government. The goal was to provide a browser, like the Haiti Crisis Map with the imagery and OpenStreetMap roads, data gathering and visualization tools like GeoCommons, public and quality checked sources of Hospital locations, camps, and damage assessment, and the World Bank's own flyover imagery on portable hard drives and onto the desk of the Haitian ministers.

A major hurdle, however, was that there were no local copies of the raw imagery data. Thanks to the incredibly hard work of a number of organizations, especially Georgetown, SDSU and Internet2, we were able to move the 1.25 TB of raw data from the Hypercube server in San Diego to Georgetown's servers here in DC in a little over a day. These were loaded onto a series of 6 USB, self-powered hard drives. We chose the self-powered drives so it didn't depend on external plugs or fluctuating power sources.

In addition, we built an offline version of the Haiti Browser that can run by double-clicking an HTML file to run in a browser off of a hard drive or USB stick. You can get the code from here to see how it's done. We also included Delta State's MGRS Atlases for printing map books, and the GPS map images that can be used on Garmin handheld units.

There will also be several engineers deploying with the World Bank to assist in the dissemination of the data, working with the Haitian Ministries and President on using these tools and handling requests for more analysis and data with CNIGS (Centre National de l’Information Géo-Spatiale), Haiti's GIS department.

Enabling Self-Sufficient Government

The goal is to provide the Haitian government with their own capability of using the available tools for situational awareness and decision making. Beyond the immediate response and recovery efforts they are now enabled to utilize these tools for long-term reconstruction and infrastructure. The capabilities and tools should be sustainable and the goal is to close the data loop within the government as well as between external data collection and Haitian data needs and collection. We are moving beyond the traditional problems of leaving disasters without their own data or ability to be independent of the temporary organizations. The entire project is a model of how crowd-sourced data and tool development can have a beneficial impact both on remote support as well as directly to local citizens and government.

In the US and Europe, we're seeing a growing embrace of Open Government and transparency sharing data to citizens for engagement and collaboration. In Haiti, the community is performing a reverse-Gov2.0 - they're providing data and information to the government to respond, and rebuild their society.

A tremendous thank you to the numerous individuals and groups that helped and provided tools or data: World Bank, San Diego State University / Calit2, Internet2, Georgetown University, DigitalGlobe, Delta State University, Sahaha, Crisis Mappers, OpenStreetMap, NOAA, Ushahidi, DevelopmentSeed, TelaScience, STAR-TIDES, CrisisCommons, USAID, GeoCommons, OpenSGI, GeoEye.

Popularity: 2% [?]

In the process of helping out with CrisisCommons and various Haiti relief efforts I've noticed recurring challenges around getting resources to sustain all the great ideas and apps being created. As I looked around at all the app contests and volunteer efforts it struck me we are creating an app junkyard. This does not not mean the apps are junk. It means there is not institutional infrastructure to sustain the apps and innovations that are created. The apps are incredibly valuable but we have no mechanism in place to grow to seed into a tree.

Crises often act as crucibles to distill problems rapidly and Haiti in my mind did this in spades. While CrisisCommons gathered an immense talent pool and many brilliant quick spin applications were created, there was no vehicle to sustain and integrate the applications with those in need. There were government agencies and NGO's that had need, and the community had useful solution, but it was incredibly challenging to connect the two together. A few of the issues I saw that created obstacles:

1. Communication - simply connecting technical solution with those in need, and knowing how to navigate the bureaucracy of the organizations with need. Fortunately great websites and wikis were built, media coverage was abundant and lots of volunteers were able to leverage their social networks to make connections.

2. Institutional competition - the problems that arise in crises are fairly well known. Not surprisingly many government agencies have spent money, often a lot, to solve these problems. For many innovations that come out of community efforts there is already an incumbent solution in place. Even if the community solution is better or complimentary, there will be institutional and vendor resistance. No one want to look like the schmuck whose multi-million multi-year project was just trumped by a group of hackers over a couple of weekends.

3. Contract vehicles - even if you get past the issues communication/awareness and institutional resistance there are funding problems. If there is not a mechanism to make funding available it becomes exceedingly challenging for an innovation to survive past the weekend volunteers and more importantly get adopted by the user community. This was seen at the most basic level with CrisisCommons during Haiti support efforts. A government agency saw a big need the community could address and wanted to get several of the Geo-Heroes on site to help. Just getting their travel and meals covered was a major feat of bureaucratic magic.

In order for volunteer tech efforts or app contests to succeed long term I think these problems need to be addressed. My half addled brain had a few ideas over the last couple of days. Communication is the easiest, but during a crisis it can be tough to quickly put these together. CrisisCommons went a long way in solving this and the result, I believe, was a far more useful response from the tech community that did make a difference on the ground. The phenomenal founders have some great ideas to institutionalize the efforts to make them repeatable and more efficient. On the app contest side where long range planning is an option there is less of an issue. Folks like iStrategyLabs and others have a well oiled machine playing that role for a variety of great efforts.

The bigger challenges comes from solving the problems of institutional competition and funding. To really capture the value of the innovation coming out of open data and community efforts there needs to be the ability integrate applications into the IT baseline of agencies and NGO's. This is the only way, in my opinion, that the innovations will be sustainable over time. Otherwise we end up with a junk yard of brilliant apps that never received the support they needed to make a difference. We run the risk of creating a self-congratulatory cycle that does not actually deliver the needed capabilities.

Over time even the best and most popular apps have inevitable costs associated with them. Maintenance, hosting, bandwidth, just to start with. Even in an open source model a little bit of funding can make a huge difference in the success or failure of a project. I think we can all agree the cream of the crop apps resulting from these initiatives deserve funding, which brings us to the next problem - institutional competition. The best apps solve well known problems in novel ways. One of the toughest tasks getting these innovations into an IT baseline is removing or integrating with the existing solution to the problem. In any institution there are employees and vendors with a vested interest to keep the status quo.

The flip side of this is the same employees and vendors have the all important "contract vehicles" which are the barriers to making change happen. Some may say this is a conflict of interest but it is the reality. So, what is a possible solution to it all. In short make it a win-win as much as possible for all involved.

Government (Big G) first needs to recognize the new model that is emerging. Haiti was a poignant example but more proselytizing is needed. Open data, open platforms and communities can out perform the status quo by orders of magnitude. In recognition "Big G" needs to make money available to seed these initiatives to cover basic costs. The app contests are a great examples of small incentives creating great return. For crises we should create "seed" slush funds that can quickly be accessed to help with community volunteer costs - like travel etc. Have a simple process for "Big G" to allocate "seed" money to the best projects that solve the most immediate problem. Create a simple application and have the process in place with all the relevant organizations ahead of time.

This is the easy problem. The tougher problem is how do you sustain the successful projects long term. This applies to both app contests and volunteer efforts around crises. The solution here requires more effort. "Big G" needs to allocate larger pots of funding to get community app innovations integrated into the baseline. Perhaps more importantly "Big G" needs to provide support and guidance to innovators. Support in the form of contract vehicles, and guidance in the form of how to navigate the bureaucracy of government procurement. We should also not forget support for basic small business skills.

The purist in me says a custom (simple) contract vehicle should be built through GSA for the effort and the SBA should set up guidance and grant programs to foster the business development side. The reality is this would take a long time and you would still be challenged by institutional competition, especially from the vendors. The solution here I believe is co-opting the vendors. Have the vendors (i.e. system integrators, beltway bandits, what ever you prefer to label them) partner with "Big G" to provide contract vehicles and have them provide integration support. "Big G" can run their typical vendor bake off's to select which firms become part of the program and allocate enough funding to make it interesting for them.

Some may consider this making a deal with the devil, but I think it is the compromise that gives the innovations and innovators the best shot at success and sustainability. Innovations by their nature are disruptive, and the more you can co-opt the status quo to embrace the disruption the higher the odds of the innovation surviving. Incentivizing the vendors to help incubate innovation coming out of the community is one of the most direct ways to combat institutional competition. It has to be a win-win for "Big G", vendors and innovators. The good news, in all this, for the citizens is we can get a much more efficient return on investment for tax payers dollars. Not only does innovation occur more rapidly, it gets deployed in ways that can help solve problems at much lower cost. Last but not least we get another driver for small business generation. An aspect that has been largely lacking from the broader stimulus response. This is something we've discussed as a positive externality of open data as a public good, but now we could have the momentum to seize upon it.

Popularity: 3% [?]

Maps by Emily Sciarillo and William Benjamin

Last week President Obama presented the Congress and the American people with his examination of the state of the union after his first year in office. He outlined his achievements as well as some of his failures in the past year and presented his plans for the future. We thought we would make maps highlighting some of his major points in the speech. Some maps we made with datasets that are regularly updated in Finder! such as unemployment. For others we created new datasets and uploaded them into the database.

Recognizing that the economy is the one issue foremost in American’s minds, he focused much of his speech on the subject. Job creation and small business growth were major factors in his plan for improving the economy, citing the Stimulus Package as the engine for both.

The first map shows how the unemployment has changed in the past year. The green counties show areas where unemployment rates decreased since last November and the purple counties show areas with increasing unemployment.

It is no surprise that most areas of the US have seen their unemployment rates continue to increase but to put that into perspective, it is helpful look at the same scale for the 12 months change during the last year of the Bush administration (the next map). During Bush’s last year the unemployment also rose in much of the country however it appears that during Obama’s first year, more counties increased dramatically (by 4% or more) and less counties improved their unemployment rates at all. To address unemployment the president spoke of the Jobs Bill and a major component of that bill is to fund clean energy projects. The following map shows current wind energy projects in each state. If DC is going to promote more environmental projects around the US, than it might be interesting to see how green DC itself is. The next map shows commercial buildings and plants in downtown DC that are Energy Star approved. Another way that the President suggested he will boost job growth is by doubling our exports in the next 5 years and mentioned specifically trade with South Korea, Colombia and Panama. The map below shows the US Trade Balance for all countries for 2009. As expected we have a high trade deficit with China. It is interesting to see with what countries in Africa, Latin America and the Middle East we have a deficit or a surplus. This next map shows the total trade balance of each country in the world. The US clearly has the highest trade deficit. As part of the recovery act, Obama highlighted a high speed rail plan. These projects around the country should provide for at least some job growth. The next map shows where these projects will be on top of the unemployment rates. Although not the main topic in his address, health care did get a lengthy mention. The president reconfirmed his commitment to improving health care in this country, particularly for the uninsured and underinsured. The next map shows uninsured Americans by state. Check out these dataset and others in Finder!

Popularity: 3% [?]

The beauty of Geocommons is the ease of use and ability to manipulate data quickly. Sometimes we have shapefiles of data that contain polygons, say for buildings. What if I want to view those buildings in Maker! by points rather than polygons. Easy! Simply upload your shapefile, fill out the metadata, and then download the data as a csv. You can find this option to the right of the layer information, which is the bottom of three options.

When you open the csv, Finder! has already found the centroids of the polygons for you and you will see two columns for latitude and longitude. The next step is to save the csv and upload this to Finder!. Now you've got a file of points rather than polygons. You don't even have to change the attribute names because Finder! likes them in any format.

Another great option for the csv download is to sort based on one of the attributes. For example, if you want to see buildings by type, such as "collapsed" simply sort, select the collapsed buildings, copy and paste into a new csv. Then copy the header row and save. Reupload into Finder! and bam, you've got collapsed buildings as points in Maker!

Upload your own shapefiles and see what you can come up with using our easy download options.

Below are step by step screenshots of this simple process. Enjoy!

Popularity: 5% [?]