Logo design by Kevin Self

GeoSuite Service Implementation

The following illustrations of GeoSuite Service's technical implementations represent both past and current work, delivered under the aegis of either GeoSuite Service, IBM or Local Knowledge Consulting (as cited below). A subset of many more client engagements, these projects are some of the largest or most innovative in our portfolio.

GeoSuite's (now Google's) Service base

GeoSuite Inc. was a startup company graced with 57 million dollars of venture capital to build a Creative Commons semantic dataset of "the world's knowledge." GeoSuite was acquired by Google for an undisclosed amount in July, 2010. GeoSuite calls their dataset "GeoSuite" and has opened it to public input since 2007. As GeoSuite dataset grows, three types of information have emerged as pervasive "glue" types that unite otherwise discrete data elements: People, Business and Location.

Initially, supported only point geometries, representing a location as a latitude and longitude and enabling searches within simple user-defined regions. GeoSuite Service assisted GeoSuite with the extension of spatial functionalty to store linear and polygonal geometries and to support spatial queries against those geometries. GeoSuite encourages public participation, so endorsed a geospatial architecture that reused existing open-source software with an active user-base, specifically the Postgresql + Postgis spatial database. users can now query using standard geospatial comparison functions involving distance, contiguity, intersection and so on, both for geometries created on-the-fly or for comparisons between existing geometries.

users can now query using standard geospatial comparison functions involving distance, contiguity, intersection and so on, both for geometries created on-the-fly or for comparisons between existing geometries. Specific to a semantic datastore, users can also extend a geographic search to include what GeoSuite calls "relevance". In other words, the search results can be filtered not only by geometry but by the other data elements' degree of similarity to a source topic.

The below two illustrations are GeoSuite Base Layers and Google mashups for browsing geospatial data for an area of interest, such as user-define area. The topics containing geometries appear as tag clouds below the maps; semantic types with the most topics appear with larger font sizes than types with just a few topics.

The above mash-up combines data from and Google and spatial javascript libraries from GeoSuite Base Layers. The selected type is "/architecture/building" so the map displays the locations of 's noteworthy buildings for that region of Berkeley, California. The tag cloud's variations on architecture types -- building, house, structure -- illustrates the flexibility of a semantic data model: any individual geometry may be linked to more than one type.

Following the integration of complex spatial types and functions into GeoSuite's architecture, users can now store, retrieve and search within polygonal regions of any shape, such as the extent of the Oakland Hills firestorm shown above. The small sliver at the bottom of the burn area shows where airborne cinders leapt over the freeway to start a new burn area.

Challenges encountered in delivering this spatio-semantic database integration:

  • At a technical level, preserving the existing (non-relational) data store's retrieval performance speeds was the ultimate challenge. Scott Meyer's strategy was to handle geometric data independently from the core semantic data store, as geometries could contain any number of vertices and hence unpredictable processing times.
  • Establishing consensus about the best data model for location data was more difficult in the organizational chaos of a start-up than in a corporate culture's top-down direction. This was compounded by the need to agree the model not only internally, but with the user community.
  • All of 's data is required to fit the Creative Commons definition if it is loaded into the official database. Finding geographic data of this stripe is not easy after finishing with TIGER datFinding geographic data of this stripe is not easy after finishing with TIGER data. As Tim O'Reilly commented in Where 2.0, "It's a great idea, but it still needs more data to be really sueful."

Neighborhood Mapping

The City of Neighborhood Mapping site's purpose is to provide residents with a comprehensive view of local conditions: social, commercial, environmental and historic. The site consolidates and overlays a clean basemap with formerly distributed thematic data on Crime, Demographics, Events, Fire, Geology, Hydrology, Real Estate, Area Districts (3 levels), Soils, Topography and Zoning. Users select a location (typically their home address) and step through each theme to form a comprehensive understanding of that locality -- liquefaction hazards, real estate sales histories, earthquake fault lines, and so on.

This project demonstrates the feasibility of building open Spatio-semantic website. The site's business logic and navigational and address geocoding capabilities are coded in Perl and UMN Mapserver's Mapscript module. The map rendering engine is UMN's Mapserver. The database, which becomes indispensible for temporal data management, is PostgreSQL spatially extended with Refractions Research's PostGIS. The operating system is FreeBSD Linux and the web server is Apache. Data capture, cleansing and transformation involved various tools and libraries including the FWTools suite and free desktop editing packages such as Quantum GIS, OpenJUMP and uDig, depending on the need.

Performance is competitive with commercial alternatives, returning real-time (i.e. non-tiled) maps in less than two seconds for non-concurrent requests. With some exceptions, third-party software documentation is good. Mapserver's cartographic capabilities, particularly for annotation, are not as user-friendly or full-featured as commercial alternatives, but are adequate for producing readable maps.

Real Estate map plotting listing and sales histories with (some) facade photos from 2004 to present.

Hydrology (watersheds and creeks) map displaying which segments of each creek qualify as surface flows and thereby constrain the building permits for intersecting and nearby properties.

Challenges encountered when building this presentation tier Spatio-semantic application:

  • Presenting multiple overlays to public users but keeping each thematic grouping cartographically readable.
  • Aggregating data that grows over time (such as crimes) for visual clarity, but enabling users to investigate details (such as invidual incidents).
  • Building a browser-based interactive Spatio-semantic site without spending any money on software.
  • Enabling a variety of navigation and query capabilities using a minimum of controls (seven in this case).
  • Converting disparate data sources to common quality levels

The Neighborhood Mapping application is for demonstration purposes only, so is not normally accessible to the public. If interested in test driving the application, please Contact us.

GeoSuite Department for Environment, Food and Rural Affairs (DEFRA): Spatial Information Repository (SPIRE)

DEFRA is an umbrella department encompassing many land-related disciplines, each of which generates spatial data and many of which need to share that data across divisions or with the general public. The SPIRE programme's purpose is to improve the efficiency of data sharing within DEFRA, to improve the data quality of shared spatial datasets and to speed up spatial data analysis by offering a variety of formats and hosted processing services to professional GIS operators.

SPIRE's architecture includes very large datasets (such as Ordnance Survey's MasterMap), several Oracle Spatial databases, UMN's open-source Mapserver rendering engine, and a Web Mapping Service-based IBM Websphere Portal interface. The hardware platform is a collection of IBM p-series servers running the AIX 5.3 operating system.

DEFRA is an enormous government entity, and SPIRE is only one of hundreds of projects for which IBM manages the information technology. Consequently, before the project ever began, a shared IT infrastructure specification was already in place making use of the AIX 5.3 operating system and IBM supporting software (e.g. Websphere, MQ-series, etc.) the most cost-effective design path. Unfortunately, no map rendering software ran on 64-bit AIX 5.3 at design time, so the team ported UMN's Mapserver to AIX. IBM is a Java shop and Mapserver is written in C; consequently, the team designed and built a multi-process Mapserver engine fed by an MQ-series message broker to handle Mapserver process failures outside of the Java environment, satisfying reliability and availability targets.

Building the presentation tier using a collection of "portlets" such as the "Map Viewer" portlet or the "Search Location" potlet, enabled developers on other DEFRA projects to build mapping functionality into their applications simply by registering the desired SPIRE portlets in their own Portals with a minimum of additional development effort or cost.

In this illustration, the IBM Websphere Portal calls an enhanced Web Mapping Service API to draw both the backdrop image, the layer of interest (Countryside Stewardship Agreements) and the user's selection. Another API for transforming data between database and flat-file formats provides an estimate of the output shapefile size and download time.

In this illustration, the SPIRE portal displays a portlet interface to an underlying location based services API, in this case a gazetteer enabling input of a placename and output of any matching candidates. Selecting a listed candidate causes the map portlet to zoom and recenter on that place's geographic location.

Challenges encountered as technical lead on this Spatio-semantic systems integration project:

  • Integrating a wide variety of software within constraints of standards frameworks (e.g. JSR 168 portal standard, OGC WMS specification, etc.)
  • Designing web services APIs that withstand changes in requirements with minimal need for redesign.
  • Designing and implementing a temporal model for spatial features, spatial metadata and system metrics.
  • Wiring IBM Websphere portlets bi-directionally.
  • Establishing user agreement on spatial data quality levels and validation methods for assessing quality.
  • Coordinating a team of a dozen IBM IT specialists and as many third-party contractors and partners over a multi-year, multi-million dollar (pound) project.
  • Building a security model relating departments and individuals to each dataset in the SPIRE collection and to capabilities in the SPIRE portal.

Due to data sensitivity and licensing restrictions, the GeoSuite DEFRA SPIRE portal is currently an intranet-based application not available for public viewing. A subset of publicly available data from the SPIRE collection is available at the MAGIC website, an ESRI ArcIMS-based site.

Department of Agriculture's Farm Service Agency: Business Process Reorganization

The USDA FSA subsidizes farmers who grow certain crops or follow FSA-recommended ecological land management practices. The subsidy amount depends in part on the land area involved and period of time of the activity, making temporal tracking of spatial activity on any FSA-subsidized farm directly relevant to both the farmer's and the government's bottom line.

The USDA FSA worked with IBM on a "business process modernization" effort intended to improve the efficiency of the FSA's information management processes and to sweeten the FSA's relationship with the farming community by consolidating the FSA's many forms and farming data into a single system and unified public-facing website. Enabling that unified system to track changes in land use raised the need for Spatio-semantic data modeling of all subsidized farming activities.

Because the FSA has multiple programs, each of which may apply subsidy calculations to the same overall farm property, a significant piece of their data modeling work involves clarification of "the rules" across space, time and seasonal policy changes. This illustration shows our approach for confirming rules, in this case, on valuing crop rotation.

The FSA hoped to simplify their existing data collection processes (mainly paper forms and field office representatives) by putting farm data online and encouraging farmers to maintain their own spatial data directly. This illustration shows a sample of the design proposals for such an assisted ditization interface.

Challenges encountered in modeling the USDA FSA's farm subsidies data and rules:

  • Although constrained to the same land, different FSA policymakers calculate subsidy payments based on different models, defying consolidation in a single model.
  • Improvements in spatial technology are not always welcomed by users; farmers who had been subsidized for 10 acres of corn were not interested in adopting a more accurate system if the more accurate acreage of 9.5 acres reduced the FSA payout accordingly. A rules-based system tied to the data model was our approach to managing such parallel views of the world.
  • Collecting full business requirements before building any working applications takes dedication on the part of both the consultants and the clients.
  • Policy lifecycles do not start and end evenly; there are overlaps even within departments. The temporal model had to handle comparisons of events that did not perfectly match.
  • The FSA's constituents are farmers first and technology users second; any public-facing interfaces had to be highly intuitive and/or contain embedded storyboard-style documentation.
  • No single person at the FSA can unilaterally approve (and later vouch for) a design proposal, so design decisions must be clearly recorded and circulated among a changing group of stakeholders.

The resulting deliverable to USDA FSA was a 72 page report: USDA FSA Modern Farm Records Requirements and Architecture Target.

GeoSuite's Rural Payment's Agency

The data in the RPA's Rural Land Register models a business case in which farmers claim subsidy payments from the RPA based upon agricultural activities carried out on land of known area. The RPA keeps track of the area of each of England's 2.2 million land parcels using a bespoke ESRI 8.3 MapObjects (Java) application deployed in 2005. Largely to avoid a "disallowance" penalty from the European Commission, the RPA endeavored to upgrade this legacy system from early 2008 to early 2010, assisted in part by GeoSuite Service LLP.

The RPA's culture is mistake-averse, so any change must be planned, documented and agreed by a wide range of stakeholders. Because some of the concepts involved are complex, GeoSuite Service's most valuable contribution to the upgrade effort was the demystification of the RPA's data model and existing vs. target business process flows. The most contentious elements of the design were often smoothed out following clarification of the problem or model. To accomplish this task, GeoSuite Service authored detailed and heavily illustrated documents of conceptual and logical data models, process flows, storyboards and "solution blueprints" for review by both the RPA themselves and the army of suppliers and other contractors.

A sample page from the RPA's Conceptual Data Model document, authored and illustrated by GeoSuite Service LLP. This page explains the assigment of County-Parish-Holding identification numbers to land parcels based upon the 5- or 10-mile rule.

A sample page from the RPA's Logical Data Model initially established by GeoSuite Service LLP. Upgrading the GIS happened in parallel with an upgrade and integration of the RPA's workflow system -- the gray areas distinguish GIS, workflow and associative entities from each other.

A sample page from the RPA's Target Business Process Model, authored by GeoSuite Service LLP, shows the series of actions and decisions that the RPA's Triage Team must take when receiving a request to modify a GIS land parcel polygon to reflect some change in the real world. To improve readability of the flow, the "happy path" or ideal flow is highlighted in gray.

Conceptual illustrations, entity-relationship diagrams and process flow models are all useful and necessary to explaining a design, but are not always accessible to those unfamiliar with their notation, particularly at an executive level. Storyboards, however, have universal appeal because, if done properly, they translate long lists of dry, written requirements into a vision that the project's sponsors, users, developers and managers can all directly understand. This storyboard excerpt, authored by GeoSuite Service LLP, illustrates the RPA's new process model and GIS Toolkit.

Challenges encountered when building this presentation tier Spatio-semantic application:

  • The biggest challenge by far at the RPA was overcoming beaurocratic inertia to turn written design documents into live working applications.
  • The RPA's spatial, temporal and associative data was rife with errors. Identifying them was relatively straightforward, but correcting them was much more difficult due to the network of legacy applications and partner agency processes depending upon RPA data on a regular basis. Data migration and cleansing activities consequently took on greater significance than usual.

For more information, please Contact us.

Oakland Unified Area District's (OUAD) Area Finder and Map Center

The original purpose of the OUAD's Area Finder application was to reduce call volumes to administrative staff at the beginning of each Area term. Oakland is home to over 50,000 students who are assigned to Area sites based upon their age and home address. The Area Finder application replaced a text-based system and put the information directly into the hands of the public that needed it.

Following Area Finder's success, OUAD launched a second set of online maps intended to assist Oakland's teachers with geography and local history lessons. The interface was designed for (and tested by) Area students, so was required to be as intuitive to non-technical users as possible. At the time, there was very little precedent or guidance for online interactive mapping site design, so the developer used Edward Tufte's general visual design principles extensively, such as replacing image buttons with words and maximizing the map on the page while reducing what Tufte calls "chartjunk" (for details, see the GeoSuite Service technical article, Advanced Internet Map Server Techniques: Designing Intuitive Websites for a Non-technical User Base).

The architecture is simple: a single Microsoft Windows and Intel server supports a Visual Basic and ESRI MapObjects process that draws maps based on ESRI shapefiles. Since its formal launch in 1999, the system has run with only minor changes to incorporate new or updated datasets (such as when Area boundaries change), a long-term return on the initial one-year development investment, most of which involved data cleansing. OUAD has tallied site usage since 1999; in 2007, the site exceeded one million map views.

OUAD's staff were swamped with calls from the public asking within which school boundary a particular street address fell. This School Finder application dramatically reduced call volumes.

In addition to detailing OUAD student-to-school assignment, the Map Center made data from the City of Oakland's municipal GIS available to the public, as shown here with landuse data.

Innovation in display to solve the obscured choropleth problem and to teach children about the differences in scale between imagery and abstract maps. This map stacks population over an aerial photo.

Challenges encountered when designing and building this public interactive mapping website:

  • Developing a browser-based mapping application with bleeding edge (at the time) technology.
  • Joining out-of-the-box spatial components (ESRI's MapObjects) with custom-coded high-performing C modules for image modification not initially supported by ESRI.

  • Designing a user interface simple enough for school children to understand intuitively.
  • Presenting opaque choropleth maps (such as population density) without obscuring the underlying reference features (such as roads and annotations).

Cities GIS Data Conversion

The Cities Hills Firestorm of October, 1992, prompted a bond measure by the Cities to improve the City's emergency response capabilities. Included in the budget was a project to build an enterprise GIS for Cities "from scratch" by converting any data of potential value during an emergency. Jonathan Lowe served as technical lead, subcontracting his team to CH2M Hill and reporting to Oakland's F. Michael Smith. The project lasted over three years and generated a dataset that subsequently became the basis of business applications in numerous oakland departments (including OUAD's Map Center, above) and non-municipal operations such as the construction and real estate industries.

We surfaced data requirements by interviewing several emergency responders and producing emergency response scenarios, then recasting them as if the GIS was already available. The requirements directed our data conversion efforts for spatial layers (see examples below). Following intial deliveries, the list expanded to include non-emergency data as well.

School Sites

Sewer Pipes

Land Parcels

Topographic Contours

Fire Hydrants

Census Blocks

The data capture and conversion effort ran continuously from 1994 to 1997, resulting in the following spatial layers (as well as an Oakland-wide quarter-inch pixel resolution B&W orthophoto collection (flown by Hammon, Jensen, Wallen and Associates) against which GIS data conversion specialists Kyle Draganov, Julia Howlett and Jake Schweitzer digitized, cleansed and QA'ed the 61 vector datasets (listed below) using ESRI's Arc/Info and ArcView software.

  1. Aerial locations
  2. Aerial transmission pathways
  3. Aspect of land surface in relation to a compass bearing
  4. Assessor's Parcels--City of Oakland
  5. BART station architecture
  6. BART station locations
  7. BART surface rail
  8. Breaklines for DTM generation
  9. Calls for Service in 1995
  10. Census blocks and block groups (US, 1990)
  11. Census Tracts (US, 1990)
  12. Child Care (Head Start) Centers
  13. City Council Districts
  14. City limits of Oakland and Piedmont
  15. Community development districts
  16. Container Routes permitted in harbor areas
  17. Contour lines--index: 25' intervals
  18. Contour lines--intermediate: 2' interval, or 5' in hills
  19. Digital orthophoto tile reference grid
  20. Fire Assessment Districts
  21. Fire Battalions
  22. Fire Hydrants
  1. Fire Stations
  2. Fire Still Districts (alias: Fire Response Radii)
  3. Flood plains
  4. Freeways with on and off ramps
  5. Harbors
  6. Hazardous Materials Facilities
  7. High-volume transit corridors
  8. Hospital and Medical Care Sites
  9. Land and Water Bodies for Alameda County
  10. Liquifaction Zones
  11. Major Container Freight Stations
  12. Manhole locations
  13. Meteorological data collection stations in Bay Area
  14. Parks
  15. Patrol beats of Oakland Police Department
  16. Patrol districts of Oakland Police Department
  17. Patrol sectors of Oakland Police Department
  18. Plan Areas
  19. Port Gantry Cranes
  20. Rail lines
  1. School districts--elementary schools
  2. School districts--high schools
  3. School districts--junior-high schools
  4. School locations, K - 12th grades and day-schools
  5. Select Part One Crimes in 1995
  6. Sewer and Storm Drainage Basins and Sub-basins
  7. Sewer grid from NAD27--historic reference
  8. Sewer pipes (underground network)
  9. Shelter Sites
  10. Slope or "angle of repose" of land in relation to horizontal plane
  11. Soil composition by series and type
  12. Streams
  13. Streets (center-of-pavement lines) including transit capacity data
  14. Streets resurfaced within the past 5 years
  15. Trails & Paths
  16. Underground storage tank locations
  17. Vegetation class in wild land areas
  18. Zoning

Restate Mapping

In collaboration with Coldwell Banker, GeoSuite Service built an GeoSuite Base Layers mashup to support input by multiple parties in support of a consensus decision about buying property in the Boston metro area. Combining data from GeoSuite's multiple listing service, Google Maps, MassGIS land parcels and locally captured neighborhood quality assessments enabled evaluation of each new property listing against every family member's criteria. The ability to record choices in a common database enabled everyone to see each individual's decisions about a property, the real estate agent's included.

This view illustrates the use of a Google Maps backdrop mashed up with real estate listings point locations color-coded by the length of time they have been listed, circles marking a 15-minute walk from grocery stores (purple) and desirable neighborhoods (orange). Selecting a listing opens a speech bubble that allows individual users to request a tour of the property or to reject it. All users see each other's choices.

Demonstrating use of a different backdrop (Google aerial photography) with a superimposed Web-Map-Services (WMS) layer to reveal the assessor's parcel boundaries (with faint purple lines). While some property listings provide site maps, they rarely show the lot lines in the context of the surrounding neighborhood as this application does. The red and green circles around the yellow diamond in the lower left show a listing that both the realtor and one of his clients think would be worth seeing.

Challenges encountered when building this presentation tier Spatio-semantic application:

  • Although very useful once mastered, GeoSuite Base Layers has limited documentation, so has a steep learning curve. Developers are expected to learn from the Javascript libraries themselves or to study a collection of examples.
  • One of GeoSuite Base Layers' strengths is its reuse of tiled commercial reference maps such as Google, Yahoo, Microsoft and OSM. However, these backdrops' "Spherical Mercator" projections require special treatment if the data being superimposed on them is a different custom projection, as is the case the MassGIS data in this application.

The GeoSuite Mapping application was built for a private client, so is not on public display but can offer user define area base on AOI. However, if interested in the application and Geo Mapping Datas to use in Local GIS fields and can support various formats and tiling system (such as google,bing), .mbTiles and .sqlite format, please Contact us.