Dealing with Data Accuracy in the GeoWeb: The Day China Annexed Taiwan

One of big gremlins in Geography that often gets over looked is data accuracy. On the surface data accuracy and validation is a pretty dull topic, but when data is wrong or controversial the repercussions can be huge. Google Earth has been a lightning rod for much of the controversy, especially when it comes to the definition of borders between countries. A border controversy in Chile over inaccurate data in Google Earth created a large media stir as did a disagreement between India and Pakistan over the border in the Kashmir region. Interestingly Google outsources the whole volatile issues of borders to Europa Technologies who “supply the international borders, national borders, coastlines, airports and places data featured in Google Earth.” When you have 250 million downloads it is hard for Google Earth to not be a lightning rod, and the truth is there are data accuracy issues all across the GeoWeb and traditional GIS.

GIS has a long history of tackling issues of data accuracy and have created a set of standards like “National Standard for Spatial Data Accuracy” as part of official FGDC metadata. While these standards worked for dealing with data accuracy in traditional GIS, are they sufficient or appropriate for the GeoWeb. Further are there capabilities emerging in the GeoWeb that could help make the entire world of geographic data more accurate. OpenStreetMaps has made a strong case for “crowdsourcing” creating more accurate and timely maps (if not more complete) than tradition GIS approaches. Is there potential in using crowdsourcing to make all geographic data more accurate and timely. Is there a possible symbiotic relationship with existing metadata standards to harness the historical diligence of GIS metadata with the real time responsiveness of crowdsourcing?

I think part of making this leap is recognizing that there is a need for it. Just because geographic data has official metadata and comes from an official source does not mean that it is accurate or current. Take Mikel Maron’s well documented example of a New Orleans bridge destroyed by Hurricane Katrina but remaining as a geographic entity in NAVTEQ and TeleAtlas official datasets (as exposed through Google and Yahoo maps). We recently came across a similar example on GeoCommons. An open source data set of official country boundaries for China released by a large GIS vendor for 2006 included Taiwan as part of China.

Looking at the 2002 dataset Taiwan was not included as a province of China:

Either it is a mistake or the data provider is following the claim by the People’s Republic of China that Taiwan belongs to China. Google has come under pressure for recognizing Taiwan as a province of China, so there is a case to be made.

Politics aside data accuracy and timeliness is a gremlin that is going to plague all parties in the geospatial space be they GeoWeb, GIS, or increasingly a fusion of the two. My question is in the fusion of the two are their opportunities to help both sides out. OpenStreetMaps has presented an approach for increasing the timeliness and relevance of geographic data through crowdsourcing, but the commercial data providers often quip about spatial accuracy and completeness. That said the number of times I’ve used those data sets and been sent down a road that does not exist or sent to a point of interest (POI) that is mislocated is far from rare.

TomTom has been one of the more proactive vendors along these lines allowing users to correct mistakes they find in directions and POI s. Their acquisition of TeleAtlas is a strong indicator of the direction this all takes, and is only further reinforced by Nokia’s acquisition of NAVTEQ. As GIS and the GeoWeb further converge can the crowd help make the world of geospatial data richer, more accurate and more timely? OpenStreetMaps and TomTom have show it can work and reap huge benefits for streets and POI’s, but what other varieties for data or analysis will work? What is next?

Popularity: 3% [?]

On October 2nd, one of the most emailed New York Times articles in the United States was an op-ed highlighting several coinciding, racially-charged events. These included the slandering of a black female executive for the Knicks, the beating of a popular female Evangelical minister by her husband, the most recent arrest of O.J. Simpson, and, finally, the Jena 6 “miscarriage of justice.”

The article also broaches the gross discrepancies in the rate of black male incarceration compared to white male incarceration in the the United States and looks to uncover the causes behind our penal system’s gibbous population.
As of June 2006, there were 2,245,189 prisoners behind state and federal bars. The incarceration rate of black men was 4,789 for every 100,000 black males in the population compared to the 736 per 100,000 white males.

Just for reference, a quick international comparison: South Africa under apartheid in 1993, imprisoned 851 black males per 100,000 black males in the populous. The United States (under George Bush) in 2006, imprisoned 4,789 black males per 100,000 black males in the population.

Below are maps heat sourcing the state prisons’ population composition.

White Male Incarceration

Black Male Incarceration

Here is a link to the chart with full numeric values of incarceration by race in each state prison:
chart

The states with the largest discrepancies:
Iowa: White Males 159 Black Males 2818 Difference 2659.
DC: White Males 87 Black Males 2720 Difference 2633.
Connecticut: White Males 139 Black Males 2296 Difference 2157.
Pennsylvania: White Males 108 Black Males 1826 Difference 1718.
New Jersey: White Males 115 Black Males 1526 Difference 1411.
Minnesota: White Males 59 Black Males 1383 Difference 1324.
Illinois: White Males 98 Black Males 1395 Difference 1297.

The states with the least discrepancies: (still twice as many black males as white males)
Hawaii: White Males 219 Black Males 579 Difference 360
Vermont: White Males 178 Black Males 451 Difference 273

Rockefeller

As Paul Smith said, “The level of crime and the levels of inequality in a society go hand in hand, statistically you are more likely to commit crime the more oppressed you are.”

Many sources will say the direct cause of our unbalanced prison population is the ineffective laws currently governing drug apprehensions. The total number of people imprisoned in the United States in 1980 is just over the number of people currently imprisoned for only drug offenses. “A major reason for the overflow of African American inmates in New York State prisons is the so-called ‘Rockefeller Drug Laws,’ enacted in 1973. They require judges to sentence even a first-time offender to a minimum of 15 years in prison for selling two ounces of narcotics.”

These current drug laws are largely considered to be harsh, naive, and ineffective money drains. The New York State Corrections Commissioner, Thomas A. Coughlin, III, says “…the punitive effect of the Rockefeller drug laws, as well as the rigidity of the second felony offender laws, do not deter, they do not lessen, nor do they cure drug addiction. Instead, as far as prisons are being filled with low-level drug offenders, more street level sellers and abusers appear to take the place of those individuals who get caught up in the criminal justice system.”

A Heat Map of Drug Use in the United States

Unemployment

An alarming effect of the deluge of black American men incarcerated since the enactment of the current drug laws, is that upon release, these men are considered mostly unemployable. Employment discrimination is prevalent enough without adding a criminal record to render a black man undesirable for hire. According to the Community Service Society, “The Wall Street Journal reported on a study that showed white men with a criminal record have a better chance of being asked back after a first job interview than black men without a record.” Not to mention the fact that they have already lost the right to vote (“Thirteen percent of the black adult male population has lost the right to vote because of felony disenfranchisement laws.” Human Rights Watch and the Sentencing Project, Losing the Vote.)

Having a criminal record doesn’t just bar you from high level, hard to attain jobs; in places like New York, it eliminates your ability to be employed in occupations such a as a licensed plumber or barber. Furthermore, these restrictions are not modified in accordance with the degree or type of crime, they fall across the board, firmly tying the hands of many released felons.

A map of unemployment rates in the United States

Family Structure

What’s clear is that there is a devastating and ever increasing disparity in our penal system’s racial composition. What is unclear is why that is. There is strong support for the idea that our nation’s crippling drug laws are the culprit, however, The New York Times article spawning this blog states that the obvious—but ignored—reason is the increasing failure in African American families due to male/female dissension. According to the article, the abuse of black females is leading to the disintegration of a healthy, prolific family structure. Then there is the all encompassing and all so vague summary that “Although racial bias may play a role in individual cases, most researchers believe racial disparities in the criminal justice system are primarily the result of indirect discrimination; the impact of race-linked (e.g. poverty, education, neighborhood of arrest) disadvantages compounded through out the criminal justice processing system; specific ‘social structural contexts;’ and such legally relevant race neutral variables as the existence of prior records,” provided by David Cole.

So is it the drug laws, pure and blind racism, familial dysfunction, indirect discriminations, or misfiring drug laws? Probably all of the above. The more important question; what can be done?

Popularity: 3% [?]

How big a deal is it?

In numbers, most recent reviews estimate a prevalence of one to two cases per 1,000 people for autism, and about six per 1,000 for Autism Spectrum Disorder (ASD), with ASD averaging a 4.3:1 male-to-female ratio. Estimates of 1 to 1.5 million Americans are effected with disorder. With a 10 – 17% annual growth, autism will become more common than pediatric cancer, diabetes, and AIDS combined.

What is it again?

Autism is a complex neurobiological disorder that typically lasts throughout a person’s lifetime. Autism impairs a person’s ability to communicate and relate to others. It is also associated with rigid routines and repetitive behaviors, such as obsessively arranging objects or following very specific routines. Symptoms can range from very mild to quite severe.

The Controversy – is it really more prevalent or just more diagnosed?

The cause of autism is unclear and highly contentious. Current theories propose that childhood vaccines, environmental toxins, diet, and genetics are linked to the cause. What’s frustrating is that there are no effective means to prevent autism, no fully effective treatments, and no cure.

In this map created from Individuals with Disabilities Education Act (IDEA) data, the percentage of school age children diagnosed as autistic are shown. Some states reflect much higher rates of autism than others. Is it that autism is more readily, or improperly, diagnosed? Is it that autism is better defined in some states or are more children in different areas suffering from autism?

And Autism will continue to make news…

Here are some recent headlines concerning autism. New issues on the horizon are 1) understanding autism better, 2) figuring out who should pay — insurance companies, federal tax dollars, schools, and families.

*Pennsylvania tries to get private insurers to foot the autism costs of therapy. The bill (HB 1150), introduced by the Speaker, would create a new “Autism Spectrum Disorders Coverage” provision of the Pennsylvania insurance law.

*Should medical insurance be forced to cover autism therapies, even if they haven’t been proven scientifically effective and will raise premiums?

*Should tax dollars pay for Special Ed private schooling? A case currently in front of the US Supreme Court (Board of Education of City of New York v. Tom F., 06-637) is expected to clarify the parent’s and school board’s role in the child’s individual education plan.

*By Harvard School of Public Health, It can cost about $3.2 million to take care of an autistic person over his or her lifetime.

Popularity: 3% [?]

It is my great pleasure to announce that an update to GeoCommons was pushed out yesterday, which has a number of new features, enhancements, bug fixes, and a completely restyled user interface.

Here are the highlights:

• New “preview” feature in dataset listings — see the heatmap without having to make a map first!
• Generally enhanced and restyled user interface that should be more intuitive and easier to use
• New “blog style” presentation for user profiles
• New “map it” buttons alongside dataset listings — get data into your maps faster than ever before!
• GeoRSS import support
• Numerous bug fixes and stability enhancements, specifically with respect to data import and Internet Explorer

We hope you enjoy the new features and interface, as always, don’t hesitate to contact us if you have any questions, concerns, or if you find that it just breaks horribly .

Continue reading for screenshots and more details.

Technorati Tags: georss, fortiusone, geocommons

Read the rest of this entry »

Popularity: 4% [?]