There has been an interesting discussion going over on James Fee’s blog on the merits of ESRI’s new javascript API and Flex API. James has thrown his lot in with the JavaScript API, and a host of Flex/Flash developers have been exposing their technology’s merits. While we don’t use either of ESRI’s APIs internally we did have to make a choice between Flash and JavaScript/HTML when we were developing Maker. At the end of the day we ended up blending the two approaches - implementing JavaScript where it made sense and utilizing Flash when we needed powerful vector rendering capabilities.

One of the most useful references for me in this process was a workshop Tom Carden gave at ETech last year on the data rendering capabilities of a variety of approaches. The readers digest version of the workshop went something along these lines:

HTML/Javascript - handles 100-1000 data points - loads in .1 seconds
Flash - handles up to 10,000 data points - loads in 1 second
Java/Processing - handles up to 100,000 points - loads in 10 seconds
OpenGL - handles upwards of 1,000,000 points - loads in 100 seconds

For Maker we wanted to be able to handle 10,000+ points/polygons and there was no way JavaScript was going to be able to handle it. Of course rendering the data was just one of many problems. Not only did we have to render the data but also parse it from the server out to the client while running the mathematical operation enabling you to take advantage of the structured data being sent. The team came with lots of clever tricks to pull it off, but the level of performance afforded by using Flash for rendering the vector data was not available with JavaScript. Processing could be a very cool option as the technology matures. Silverlight could also be a great option if they can get the plug-in universally embedded into browsers as with Flash.

While Flash was a great option for the tiling and vector rendering we did not want to build out the entire application in Flash for a variety of reasons. In GeoCommons everything outside of the map itself is JavaScript/HTML. This is probably rudimentary for many folks, but reading the debate on James’ blog I think sometimes developers lose sight of picking the best tool for the job. Oftentimes it is easy to get wedded to an approach just because it is what you know well. We were complete Flash rookies when we started, but got some great help from Tom with Modest Maps, Axis Maps with the Flash development and cartography, hired some full time resources, and learned a lot on our own. It ended up being a great approach for the specific problems we were facing. As long as you are using standard interfaces in your development, you should be able to fluidly adapt to the technology that makes the most sense for your set of problems.

Popularity: 23% [?]

As the continuing turmoil in the financial sector and a mind boggling trillion $ bailout are together contributing to the general feeling of economic gloom and doom across the nation, the distressing news of a big bank failure last Thursday made it worse. The Office of Thrift Supervision (OTS), announced that it had closed Washington Mutual (WaMu), a big bank with over 45,000 employees, more than $300 billion in assets, 2,200 branches in 15 states with over $100 billion in deposits. Billed as one of the largest bank failures in the U.S. history, WaMu was taken into receivership by FDIC (Federal Deposit Insurance Corp), the federal regulatory agency and promptly sold to the highest bidder, JP Morgan & Chase Co for as little as $2 billion.

According to OTS, WaMu, one of the largest savings banks in the country, became unsafe after deposits of more than $16 billions left the bank in just last two weeks. WaMu, which specialized as a mortgage bank was the latest victim after IndyMac, of the combined crises of downturn in housing sector, subprime and credit crunch. FDIC maintains an ongoing list of 100 risky banks. However, this is a secret list that will never become public and neither is the process by which such a list is created.

So we at FortiusOne decided to create our own list the 100 banks based on the level of their exposure to risky/bad loans on their books. For this analysis, risky loans are computed as the sum of the total value of loans on foreclosed residential properties and mortgage defaults (30 plus and 90 plus days late mortgage payments as well as the residential properties that have stopped paying mortgages but have not been foreclosed yet.) The data for the 2nd quarters of 2008, 2007, 2006, 2005 and 2004 was downloaded from the FDIC website and then geocoded. Only those commercial banks who specialize in real-estate mortgage business were selected and then ranked according to the level of risky loans. The top 100 banks for each of these quarters were combined into one single file, then sorted by the value of the risky loans for the 2nd quarter of 2008. The map below shows only those banks that have risky loans for each of the 2nd quarters, the pie-charts represent risky loans by the 2nd quarters between 2004 to 2008.

Pie-Chart Legend: Red = 2nd qtr 2008, Orange = 2nd qtr 2007, Yellow = 2nd qtr 2006, Green = 2nd qtr 2005 and Purple = 2nd qtr 2004

So how exposed was WaMu to risky lending practices? A lot, according to our analysis of FDIC’s banking statistics for last several quarters. See the cartogram version of the data below where the size of the pie represents the total value of the risky loans. Clearly, WaMu tops in the risky loan business for each of these quarters spanning the pre- and post-housing bubble. WaMu’s bad loans for the 2nd quarters for each of the years between 2004 and 2008 range in value from just a $2.32 billion (2004), $2.63 billion (2005), $4.76 billion (2006), $6.7 billion (2007) to $15.8 billion (2008).

Pie-Chart Legend: Red = 2nd qtr 2008, Orange = 2nd qtr 2007, Yellow = 2nd qtr 2006, Green = 2nd qtr 2005 and Purple = 2nd qtr 2004

However, what is scary is that Wachovia, Countrywide, E-trade and few others are not that far behind. Does that mean, they are the next in line to fail? Not necessary, because exposure to risky loans may just be one of the factors involved in a bank’s failure. Search for a more comprehensive bank data for all quarters between 2004 and 2008 on the Finder! with key word “FDIC” and its analysis in the near future. In the meanwhile browse for the data discussed in this blog here:

Risky bank loans 2nd quarter 2008
Risky bank loans 2nd quarter 2007
Risky bank loans 2nd quarter 2006
Risky bank loans 2nd quarter 2005
Risky bank loans 2nd quarter 2004

Popularity: 28% [?]

With so much of talk about energy policies on the Presidential campaign trail, I decided to look into the nuclear option that both the candidates support to different degrees. Lo and behold I found a listing of nuclear power plants in Finder!. Now that we have all these nuclear power plants with promises of more to come - what’s next - hmmm, yes, we need to store the highly radioactive spent fuel somewhere.

Senator McCain’s most frequent example that the US Navy uses nuclear powered vessels safely all the time was somewhat debunked by news that emerged from the USS Houston - a globe-trotting nuclear powered submarine, that has been leaking radiation for the past several years. Listening to Senator McCain here on YouTube describe how we will have nuclear waste lying around in a puddle of water in our street corner reminded me of the Simpsons movie where Mr. Burns could roll down nuclear waste in a truck and dump it into the lake just outside Springfield.

Of course the next thing I wanted to check was where we dump the spent nuclear waste. I added this dataset to Finder! with all the spent fuel storage installations.
The map below indicates where these dumping grounds (a.k.a storage installations) are located.  It also shows the locations where nuclear power plants are located from the dataset I found in Finder!

Given the spatial concentration of most of the nuclear power plants as well as the current storage sites on the Eastern seaboard it is intriguing to note that millions of tax payer dollars have been spent into building a mega-storage facility in the Yucca mountain in Nevada. This facility is still not ready - long past its scheduled 1998 opening date. Once it does open, then comes the question of transporting all these highly radioactive wastes across the country. This article points to how Senator McCain says “no” to allow nuclear waste to be transported through Arizona - the state that got him elected, but supports building the Yucca mountain site in Nevada.

Popularity: 15% [?]

Sean mentioned in his blog about how pooling together of efforts by Andrew, Sean, Bill et. al, the Fortifacture/MapuCommons folks were able to bring to you in record time the near-real time pollution data from Beijing. As we were working on this, we realized that there is a huge difference in the perceptions between the host nation and most of the western world/media on what constitutes severe air quality problem. For eg. see below the two pics, both dated 5th August, 2008. One shows Beijing “Clear skies” while the other has haze/smog blanketing Beijing. Wonder whether they are talking about different places and different days!
Xinhua Photo

The photo taken on Aug. 5, 2008 shows the clear sky above the National Stadium, namely the Bird’s Nest, in Beijing, capital of China. (Xinhua Photo/Li Ziheng)

BBC Photo

5 August PM10 reading: 104 micrograms per cubic metre. The World Health Organisation guideline maximum is 50 micrograms per cubic metre, averaged over 24 hours.

Knowing that many countries in Asia, including India and China share the dubious distinction of having the most polluted cities in the world, the media’s obsession with hazy skies should come as no surprise and that much of the media coverage of Beijing Olympics has been about the quality of air. See for example, this split picture of Beijing skyline on a clear and a hazy day on the BBC’sBeijing Pollution: Facts and Figures.

BBC has, for last several weeks, a daily pic of Beijing skyline with a running commentary on the hazy conditions, on their Beijing Pollution Watch site. So we at FortiusOne/Mapufacture decided to generate a daily map of the official stats on PM10 published by Beijing Municipal Environmental Protection Bureau (BMEPB) and compare it with BBC’s Beijing Pollution Watch. PM10, the airborne particles consisting of dust from construction,landfill sites, vehicle exhaust, industrial sources etc. of size 10 microns or less, are the main culprit behind the hazy skies /bad air days in Beijing.

The map below is based on the air quality monitors spread across dozens of Beijing districts along with the locationsof Olympic events (red circles). The six slices of each pie-chart show share of PM10 at each location between 5th and 10th Aug, 2008.

th

The second map shows today’s readings of PM10 (purple colored proportional circles) for each of the air quality monitoring stations, along with a pie-chart that has share of the SO2, PM10 and NO2.

For comparison, see BBC’s pic of the same day below.

BBC: 10 August PM10 reading: 278 micrograms per cubic metre. We test for 10 minutes at midday from a seventh floor balcony in central Beijing..

While the official readings in nearly half dozen air quality monitoring stations nearby have readings near 90, it has apparently, not had an adverse effect on the athletes thus far in the games. As BBC offers daily pics of the smog, we will have daily updates on the air quality all through the Olympics. In the mean while you may explore on the Finder! the air quality data (SO2, NO2 and PM10) for the last six days i.e, 5th to 10th August, 2008, the road network, and the “>district polys as well as Olympic Athletic Venues,and Olympic village. Search using keyword “Olympics.” You are welcome to download, add, update and upload these data back to Finder!

Popularity: 23% [?]

With the rise of post-partisan Obama on the national political scene, there have been sporadic stories in the print and on-line media , in Op-Eds, on the cable-news/YouTube and in the blogs; of how some influential Republicans have turned into Obama supporters, the so called Obamacans, reverse of Reagan-Democrats. Of course, not everybody is buying into the Obamacan story, considering it as a media creation or part of chaos theory. However, the recent claims by McClatchy newspapers’s that their “…. computer analysis, incomplete due to the difficulty matching data from various campaign finance reports, found that hundreds of people who gave at least $200 to Bush’s 2004 campaign have donated to Obama”, caught our eye at FortiusOne.

So, if there indeed are Bush donors who now have become Obamacans, the data-team wanted to find out where they are spatially speaking. Below are the maps of our efforts showing locations of possible Obamacans in New York City and Washington D.C. Why use the term possible? Because what is mapped are the results based on spatial join and attribute join, the later being a variation of spatial join. And the accuracy of the results of such joins is subject to the limitations imposed by the accuracy of the original data (donor addresses) as well as limitations of the geocoding operation. More on this towards the end of this post. So what is mapped are donor address matches and not individual donors.

Attribute Join
The attribute join is based on an identifier “XY” constructed from the concatenation of X and Y location coordinates of the Bush-Cheney and Obama donors, where the X and Y location coordinates are obtained by geocoding donor addresses. The attribute join resulted in 250 records across the lower 48 states, mostly concentrated in major cities of North-East and West-Coast. The results are shown below for New York city (lower Manhattan) and Wash D.C., where blue circles represent Obama donors (1,415 in D.C. and 1,825 in New York city); red circles represent Bush-Cheney donors (294 in D.C. and 419 in New York). The purple squares colocated with Bush-Cheney red circles are the XY “attribute matches.” There were 32 such locations in D.C and New York City had 85.

New York City: “XY” attribute join of Bush-Cheney donors with Obama donors

Washington D.C.: “XY” attribute join of Bush-Cheney donors with Obama donors

Spatial Join
Yet another way was to carry out a “spatial” join between location of each Bush-Cheney donor with all of the co-located Obama donors, resulting in more than 9,200 Bush-Cheney records colocated with more than 42,000 Obama records in the lower 48 states. The results are shown below for New York City (lower Manhattan) and Wash D.C., where again blue circles represent Obama donors, red circles represent Bush-Cheney donors, and the purple circles with varying sizes represent count of Obama donors that are colocated with each of the “spatially” joined Bush-Cheney donor. There were more than 1,500 Obama donors colocating with 248 Bush-Cheney donors in D.C. while the comparable figures for NY city are more than 2,030 Obama donors colocating with 303 Bush-Cheney donors.

Bush-Cheney donor locations spatially joined with Obama donors in NY City

Bush-Cheney donor locations spatially joined with Obama donors in Wash D.C.

Donor Data
You may find/download the mapped as well as other supporting datasets from the Finder! by using the key-word “Obamacans“. The supporting datasets also include spatial join of all Bush-Cheney donors for each of the Obama donors.

Read the rest of this entry »

Popularity: 17% [?]