Dec 21, 2011

Introducing python-sld and django-sld

By

python-sld

python-sld is a simple python library that enables some basic manipulation of StyledLayerDescriptor (SLD) documents.

What are SLD documents?  SLD is a standard defined by the Open Geospatial Consortium, or OGC. In their words:

The OpenGIS® Styled Layer Descriptor (SLD) Profile of the OpenGIS® Web Map Service (WMS) Encoding Standard defines an extends the WMS standard to allow user-defined symbolization and coloring of geographic feature and coverage data.

In layman’s terms, SLD is a common way to style your own maps that come from any map server that speaks WMS (another standard by OGC). Of all the GIS tools available, the WMS server ecosystem is exceptionally rich and diverse. There are many proprietary choices, as well as a plethora of open source options.

State of the Art

Recently in the course of developing new features for DistrictBuilder, we arrived at a point where we needed to generate SLDs dynamically. Looking around at the existing python libraries, we examined:

What we were looking for was a pure object model access to components in the SLD, as well as XML validation, with very few dependencies. None of the above projects really fit the bill, so we started working on our own.

Introducing python-sld

python-sld in an open source (Apache 2.0) library for dynamic SLD creation and manipulation. The project is hosted over on github, and the packages are in pypi (including generated inline documentation).

Features

Width python-sld, creating new SLD documents is as easy as creating a new instance of a StyledLayerDescriptor object:

>>> from sld import *
>>> sld_doc = StyledLayerDescriptor()

With this SLD document, all descendants are accessed as properties, and most child objects are created off the parent with “create_xxx()” methods:

>>> sld_doc.NamedLayer is None
True
>>> nl = sld_doc.create_namedlayer('My Layer')
>>> nl.Name
'My Layer'

For most complex types, the parent’s property is an instance of the class. In our example:

>>> isinstance(nl, NamedLayer)
True
>>> us = nl.create_userstyle()
>>> us.Title = 'Style Title'
>>> us.Title
'Style Title'
>>> isinstance(us, UserStyle)
True

A couple pythonic classes break up the monotony, too. For elements that contain collections of items (a FeatureTypeStyle element may contain many Rule elements, and Fill, Stroke, and Font elements may contain many CssParameter elements), they behave as pythonic lists.

>>> fts = us.create_featuretypestyle()
>>> len(fts.Rules)
0
>>> r1 = fts.create_rule('Criteria 1')
>>> len(fts.Rules)
1
>>> fts.Rules[0].Title == r1.Title
True

Another bit of pythonic syntactic sugar is the combination of Filters. By constructing filters (with the Rule as a parent) and combining them with “+” or “|”, they create logical “AND” and “OR” filters, respectively.

>>> f1 = Filter(r1)
>>> f1.PropertyIsGreaterThan = PropertyCriterion(f1, 'PropertyIsGreaterThan')
>>> f1.PropertyIsGreaterThan.PropertyName = 'number'
>>> f1.PropertyIsGreaterThan.Literal = '-10'
>>>
>>> f2 = Filter(r1)
>>> f2.PropertyIsLessThanOrEqualTo = PropertyCriterion(f2, 'PropertyIsLessThanOrEqualTo')
>>> f2.PropertyIsLessThanOrEqualTo.PropertyName = 'number'
>>> f2.PropertyIsLessThanOrEqualTo.Literal = '10'
>>>
>>> r1.Filter = f1 + f2

When the SLD object is serialized, it will render an “ogc:And” element that contains both property comparisons. You may have noticed that both the “PropertyIsGreaterThan” and “PropertyIsLessThanOrEqualTo” properties are assigned an instance of a PropertyCriterion class. This is the common class for all property comparitors. The name of the comparitor determines it’s logical comparison (less than, greater than, equal to, etc.), and the class has a PropertyName and Literal property, to control which property gets compared, and which value it is compared against.

Finally, serialization is performed on the main StyledLayerDescriptor object, with options to ‘prettify’ the output:

>>> content = sld_doc.as_sld(pretty_print=True)

Dependencies

The lxml library is required by python-sld. This is the library that provides the underlying parsing and serializing of the XML document, as well as the validation steps against the canonical SLD schema.

Limitations

At the current time, only a subset of the entire SLD specification is implemented. All SLD elements are parsed and stored, but only the following elements may be manipulated as objects in python-sld:

  • StyledLayerDescriptor
  • NamedLayer
  • Name (of NamedLayer)
  • UserStyle
  • Title (of UserStyle and Rule)
  • Abstract
  • FeatureTypeStyle
  • Rule
  • ogc:Filter (implicit ogc:And and ogc:Or)
  • ogc:PropertyIsNotEqualTo
  • ogc:PropertyIsLessThan
  • ogc:PropertyIsLessThanOrEqualTo
  • ogc:PropertyIsEqualTo
  • ogc:PropertyIsGreaterThanOrEqualTo
  • ogc:PropertyIsGreaterThan
  • ogc:PropertyIsLike
  • ogc:PropertyName
  • ogc:Literal
  • PointSymbolizer
  • LineSymbolizer
  • PolygonSymbolizer
  • TextSymbolizer
  • Mark
  • Graphic
  • Fill
  • Stroke
  • Font
  • CssParameter

All other SLD elements cannot be directly manipulated in python-sld, but are accessible (from a parsed SLD that is perhaps more complex) via the parent object’s _node property. This is the lxml.Element that the python-sld class represents.

django-sld

django-sld builds upon the capabilities in python-sld by enabling quick SLD generation from geographic models. This library is separate from the python-sld library because of the dependencies on django and pysal, the Python Spatial Analysis Library.

Primer on Geographic Models

I gave a quick background to geographic models in django to the Boston django meetup last week, and the slides of my presentation are available online as a presentation in Google Docs. The slides are embedded here for your convenience:

Introducing django-sld

django-sld is an open source (Apache 2.0) library for generating SLD documents from geographic querysets. The project is hosted over on github, and the packages are in pypi (including generated inline documentation).

Features

django-sld enables quick classification of geographic querysets by passing the data distribution of an individual model field into the classification algorithms built into pysal. Not all classification methods in pysal are available, however. At the current version (1.0.3), the following classification algorithms are supported:

  • Equal Interval
  • Fisher Jenks
  • Jenks Caspall
  • Jenks Caspall Forced
  • Jenks Caspall Sampled
  • Max P Classifier
  • Maximum Breaks
  • Natural Breaks
  • Quantiles

To classify a django queryset, use any of the as_xxx() methods in the djsld.generator module.

>>> from djsld import generator
>>> qs = MySpatialModel.objects.all()
>>> sld = generator.as_quantiles(qs, 'population', 10)

The above example assumes that you have a model named “MySpatialModel” in django’s models.py file. The result is a sld.StyledLayerDescriptor object, which may be serialized to a string with “as_sld()”

>>> sld_content = sld.as_sld(pretty_print=True)

The “pretty_print” option is available to format the SLD in a fashion that is more readable by us humans.

In addition to simple models, django’s support for related fields really shines, as it’s possible to classify the distribution on any related field, using the “__” (double underscore) format preferred by django:

>>> sld = generator.as_quantiles(qs, 'city__population', 10)

The one caveat is that the PropertyName in the criteria will be set to this field name (which is not the way most mapping packages refer to related fields). To accommodate this difference, you may use the ‘propertyname’ keyword to control the output PropertyName:

>>> sld = generator.as_quantiles(qs, 'city__population', 10,
... propertyname='population')

Dependencies

django-sld requires python-sld and the pysal library.

Comments (0)
Sep 23, 2011

Putting the Fun in FOSS

By

I went to the State of the Map (SotM) and Free and Open Source Software for Geospatial (FOSS4G) Conference in Denver, CO last week, where I was surrounded by geospatial users, developers, and architects. I had the opportunity to attend some workshops and learn about a slew of awesome projects — I’m itching to start incorporating many of these new tools and techniques into our solutions.

Node.js

I was able to attend some of the workshops — “You’ve got Javascript in your backend” with Node.js and Polymaps was a great beginner workshop, introducing lightweight servers and client mapping libraries. I was amazed that a basic web server in node.js is only 5 lines of code. Equally amazing was seeing what capabilities Polymaps had when it weighted in at only 32K (minified) vs. OpenLayers at 1.2M (minified default build).

i2maps + pico

Some exciting visualization tools are coming out of the National Center for Geocomputation at the National University of Ireland, in the form of i2maps. While it’s relatively immature (not much in the form of documentation), most the basic functionality builds off of OpenLayers.  Since I’ve already learned the OpenLayers library, I has a short learning curve, and was able to get up to speed pretty quickly.  Their library incorporates some awesome features like dynamically loading and evaluating rasters via canvas (this only works on modern browsers), and even agent-based modeling. I could have stayed in that workshop for a week.

A byproduct of the i2maps project is pico. Pico is a bridge between Python and Javascript, enabling you to call native Python methods directly from Javascript. It performs all the plumbing for you, allowing you to write a simple callback to handle your method’s return value. It also takes care of converting Python objects into Javascript objects, allowing you to pass all sort of data back and forth (including rasters!).

mod-geocache

Another new project from a contributor to the MapServer project is mod-geocache, a tile caching service as an Apache module. This skips a lot of overhead (no proxying, no interpreters, no CGI), and is very fast. In addition, the C implementation has excellent speed and performance. You can perform on the fly tile merging, quantization, and recompression. I’m excited about this module, and the promise of caching with an Apache server (looks like it has more features than mod_tile).

Geoserver

Geoserver‘s next release is also going to include some great features. The ones that really jumped out at me:

  • Time and elevation filters — e.g. storm tracking, where you can limit the features by a time field.
  • Styling SLDs in data units — e.g. “road is 5m wide”, and changes dynamically with scale. This greatly simplifies scale-dependent renderers.
  • Georeferencing of layers can be done in the admin interface.
  • Layers can be view definitions — you don’t have to roll your own views prior to creating the layer.
  • Virtual Services — partition the data layers by workspace.

These aren’t all the new features; take a look at the laundry list yourself, and prepare to be impressed.

Mapnik 2

I think the reason for calling it Mapnik2 is that it is literally twice as awesome as it was before. I learned about the new features in Mapnik2 in the lightning talks at SotM, and I think this was one of the few talks that made you feel like you were actually struck by lightning. I can’t remember half the slides in the talk, but the supported formats, reprojection, styling, and speed improvements left me with my head spinning.

Comments (1)
Sep 15, 2011

Django, contests and weekly voting

By

I’ve written before about how OpenDataPhilly uses a ratings module to drive a nomination system. Recently, we added a contest to the site to determine what kinds of data local non-profits and the public would like to see made available. Contests generally have a winner and, in this case, we’re letting the public vote on data sets nominated by non-profits. At first glance this isn’t much different from our current nomination system, but there’s one catch; we wanted users to be able to vote for one entry once a week. Turns out this was more novel than it sounds.

Django has a few modules for rating or voting on content, one of which we’re using for the nomination and comments systems. The inner-workings of the module boil down to the following rules:

  1. A user must be logged in to rate/vote
  2. A user can rate/vote for any number of items
  3. A user can only rate/vote for any particular item once (though they may change their rating/vote later)

Compare this with the rules we wanted to enforce for the contest:

  1. A user must be logged in to vote
  2. A user can only vote once per 7 day period
  3. A user can vote for an item multiple times, so long as rule 2 is preserved

Aside from the first rule, we were trying to do almost exactly the opposite of what our rating module enforced. Rather than retrofit the existing module to allow additional and sometimes contradictory behaviour, we decided to write a very small voting module of our own.

The code revolves around two decision points: is voting allowed and can a specific user vote now. The first question is answered by the contest object itself. A contest knows when it’s starting and ending date are, so if today is after the start date and before the end date, then voting is allowed.

The second question is a bit more complicated, but not by much. Because of rule 2 above, we need to know when a user last voted to know if they’re currently allowed to vote. The database storage for a vote contains a datetime object, a foreign key to the user object and a foreign key to the contest entry so if we sort a user’s votes by time we can retrieve their latest vote.

def user_can_vote(self, user):
    increment = datetime.timedelta(days=7)
    votes = user.vote_set.order_by('-timestamp') #latest on top
    if votes:
        next_date = votes[0].timestamp + increment
        if datetime.datetime.today() < next_date and dt.today() < contest.end_date:
            return False
    return True

The above code gets a user’s votes and orders them by time with the most recent first. If a user has ever voted, we need to check if they’re allowed to vote again yet or if they have to wait. We calculate the earliest time that a user can vote next and check it against the date and time now. We also check the end of the contest against the date and time now. If “now” is before the next time the user can vote or “now” is after the contest’s end date, we return false; the user can’t vote now. If a user has never voted before, or the dates are all ok then the user can vote. This check is done after a user clicks the “vote” button but before a vote is saved to the database. We also display a message saying why this check failed and when a user will be able to vote again.

So we’re taking advantage of all of the spam protection built into Django’s user registration process and running a contest on surprisingly little code: 3 database tables, 200 lines of python (blank lines included) and a few templates is all we needed!

Comments Off
Sep 01, 2011

Restricting Zoom with Multiple OL Basemaps

By

DistrictBuilder logoAs David recently posted, our team has been hard at work implementing DistrictBuilder, where we’ve been investing a great deal of effort on both performance and usability. One feature we added in the spring was the addition of basemaps to the user interface. Before this addition, users labored over drawing the perfect district configurations without a whole lot of context of the surrounding environment (e.g. roads, water boundaries, etc.). When the time came to add a basemap to the application, it didn’t feel right restricting it to a single type of map, or even a single provider. We wanted to allow for users to have the choice to select the best map for the task at hand. Could an application promoting democracy really have it any other way?

We set out to support several base map options as well as any combination of options, including:

  • Bing Maps (satellite, roads and hybrid)
  • GoogleMaps (satellite, roads and hybrid)
  • ArcGIS Online (any of several maps)
  • OpenStreetMap

Since DistrictBuilder needs to be flexible enough to meet the needs of users and administrators in a variety of situations, we decided on a two step approach to basemap configuration. First, the administrator specifies, in the configuration file, which of the combinations of map providers and map types are allowed to be selected. Then DistrictBuilder presents all of the configured options to the user, where they can be toggled among at any time while a plan is being viewed or edited.

Here’s an example of an instance configured with an OpenStreetMap road layer, a Bing hybrid layer, and a Google satellite layer:

Road, Hybrid, and Satellite

Here’s another example with only road layers — one for each of the three configured providers:

Roads for three vendors

DistrictBuilder currently allows the configuration of basemaps using permutations of each of the three vendors and three map types described above. Adding more options is a relatively easy task, however. With the launch of Fix Philly Districts, we wanted the basemap colors to be slightly more muted than the above options, and ended up adding support for the ArcGIS Online World Topographic Map. We also experimented with the Google Maps V3 custom styling API, which looked great, but introduced performance problems when panning and zooming (animations).

There were, of course, some hoops that needed to be jumped through in order to get all of these basemaps behaving correctly on the same map, which will be discussed below. I’ve extracted the logic required to do so into a small demo that can be viewed/downloaded here. The demo has also been embedded into this post, and can be interacted with without going anywhere:

Zoom Levels

Many of the challenges that needed to be overcome to get this working correctly were brought about because we needed to restrict the zoom levels to the area at hand. We wanted to eliminate superfluous zoom levels to ensure the user was always operating within the appropriate boundaries (note: it is not done in this demo, but in DistrictBuilder we also restrict the extent with the ‘restrictedExtent’ map parameter, so users can’t even pan outside of the area).

One difficulty with setting zoom levels on the different layers is that the layers don’t use zoom parameters consistently. In Bing (the VirtualEarth layer), minZoomLevel and maxZoomLevel are needed. In Google, minZoomLevel is needed, but it requires numZoomLevels instead of maxZoomLevel. And in OpenStreetMap (the OSM layer), well…no combination of those seem to work — we needed to slightly modify the XYZ layer (OSM’s base class) to allow maxResolution to be changed based on the minZoomLevel. To see how this is done, view the demo source. With that change in place, the list of required layer parameters is as follows:

  • Bing – minZoomLevel, maxZoomLevel, projection, sphericalMercator, maxExtent
  • Google – numZoomLevel, minZoomLevel, projection, sphericalMercator, maxExtent
  • OpenStreetMap - numZoomLevel, minZoomLevel, projection

Coordinate Systems

We also faced some problems related to coordinate systems. DistrictBuilder uses GeoServer and GeoWebCache to serve up WMS layers. The coordinate system of our data is one version of the the ever-changing “Popular Visualization CRS / Mercator” projection. We needed to match up the OpenLayers projection to the one used on our data, or else we were seeing slight offsets on our overlays. Unfortunately, the ‘projection’ layer parameter isn’t always used within the layers correctly. For example, any layer using the SphericalMercator class gets its projection automatically hardcoded to 900913. We needed to make a slight modification to the SphericalMercator class to allow the ‘projection’ parameter to carry through. This can be seen by viewing the demo source.

Bonus: Math Time!

One interesting part about implementing zoom restriction was that we needed it to work in any instance of DistrictBuilder — from large states to small towns, which may have vastly different extents. Instead of having an administrator figure out the proper minimum zoom level, we calculate it automatically based on the extent, which requires a little bit of basic algebra.

For Philadelphia, the extent of our area is:

[-8397913.926216, 4842467.609439, -8329120.600772, 4895973.529229]

In DistrictBuilder, we calculate this dynamically on the server side (using Django) by filtering all of the geounits in the database and calling the ‘extent’ function on the query set. For the demo, this is hardcoded. Here’s how to transform this extent into a Spherical Mercator minZoomLevel:

  • Find the width of the area in meters.
var studyWidthMeters = extent[2] - extent[0];
  • Find the width of the map in pixels. In the demo, this is hardcoded, because we are setting the div size of the map. In DistrictBuilder, the map takes up the whole screen, and this value is calculated on the fly based on the size of the div in which the map occupies.
var mapWidthPixels = 450;
  • Find the map resolution, or meters per pixel.
var resolution = studyWidthMeters / mapWidthPixels;
  • Find the maximum map resolution. In Spherical Mercator, the maximum resolution is one 256×256 tile taking up the entire circumference Earth. So dividing the circumference of the earth (~40,000km) by 256 gives us the maximum meters per pixel, which is a constant.
var maxResolution = 156543.033928;
  • Spherical Mercator zoom levels work like a pyramid. Each zoom breaks the current tile up into a 2×2 group of 256×256 tiles, essentially halving the resolution each time. Therefore, finding the resolution at a given zoom level looks like this:
maxresolution / 2^zoom = resolution
  • We know the resolution and max resolution already, and need to find the zoom:
zoom = log(maxresolution/resolution)/log(2)
  • Or in javascript:
var minZoom = Math.log(maxResolution / resolution) / Math.LN2;
Comments Off
Aug 22, 2011

Building Districts in Web-Time

By

DistrictBuilder logoMost recently, the Politics, Redistricting and Elections team has been working closely with the Public Mapping Project to build DistrictBuilder, an open source, web-based application that enables regular citizens to use powerful tools to draw their own legislative districts. If you’ve seen how badly the professionals can mangle districts (Exhibit A, Exhibit B, etc), it’s easy to imagine that any given citizen, given the right tools, could do it better.

We spent quite a bit of time making the application easy to use and responsive in modern desktop web browsers.  The “easy to use” part was tackled by our excellent UI/UX design team. The “responsive” part was the domain of  our engineers.  That’s where the fun began for me.

DistrictBuilder is designed to use any polygon shapefile, transform it into an internal data model, then make that accessible via map tiles and geometric features.  When serving map tiles, we use GeoServer and GeoWebCache to generate the tiles and cache them, respectfully. This performance is great — pre-generated map tiles are the best we can aim for with respect to the base map tiles. Serving geometric features at full resolution, however, introduces a slew of problems. A few that stood out right away:

  • Web Browser Limitations — 9 out of 10 experts agree: too many map features has a significant performance impact on web browsers, with the greatest impact on the Microsoft Internet Explorer browser.
  • Excessive Coordinates — delivering lots of polygon coordinate pairs that the user may never see consumes valuable bandwidth and rendering time.
  • Server Processing Time — recalculating state-wide geometric features consumes valuable CPU time.

Web Browser Limitations

First, we tackled the browser performance issues. A sluggish browser is the kiss of death in the web world, and we had to make the application experience as fast as possible before looking at the server processing time.

We originally gave users the power to create highly detailed districts at the statewide level, but realized that no modern web browser could handle the volume of polygon features that would need to be served to represent an entire state.  In order to mitigate this limitation, we limited the size and number of features sent to the browser. With some scale-dependent logic, a user zoomed in to a detail of a district can finely tune the boundary by moving smaller geographic features (e.g. census blocks), and a user zoomed out to the state-wide level can manipulate the districts by moving large geographic features only (e.g. counties). In addition, when editing the finest details, we limit the number of features a user can move in a single edit.

Excessive Coordinates

The next thing to go was the set of full resolution geometries. In DistrictBuilder, users never actually see the full geometries, but an adaptively simplified (sometimes called generalized) geometry; depending on the scale of the map view, the server will deliver geometries with appropriate coordinate resolutions. Simply put: as you zoom in on the map, you get more detail in the geometries.

By simplifying counties, the geometries are reduced from 166,958 points to 4,821. When a user is zoomed out, there is no noticeable difference between these geometries!  However, as the user is interacting with higher resolution maps, DistrictBuilder loads in higher-resolution geometries on demand. The following images demonstrate the difference in the geometry detail:

Low Resolution Transition

The zoomed in County layer, with a low resolution district overlay (orange line). There are currently 1,414 coordinates in this view of the district overlay.

High Resolution Transition

The zoomed in VTD layer, with a high resolution district overlay (orange line). There are currently 3,253 coordinates in this view of the district overlay.

You can notice the differences in the district detail if you look closely at the orange district boundary. This transition happens seamlessly in the application, loading in the higher resolution geometries as web users zoom in to areas of interest.

We also eliminated coordinates that you never see.  It made no sense to serve  coordinates that were located in the opposite side of the state where a user was editing, just like you wouldn’t expect to get an encyclopedia in the mail when releasing an RFI. With the OpenLayers library, Strategies came in handy here, particularly BBOX.

Server Processing Time

After we had optimized the performance of the user interface, we shifted our focus to the server-side processing.  One of the features that makes DistrictBuilder such a powerful tool is the accuracy of the underlying data and constant feedback of important district statistics. In order to calculate all these statistics on the fly, it is necessary to leverage some tricks already mentioned with respect to map tiles: caching and generalizing.

Computation of the district statistics must happen every time a district boundary is changed. A naive solution to this problem would be to aggregate the values within the boundary every time a change is made.  This approach results in horrible performance. Instead, we just determine what has changed — which areas were added, which areas were removed — and recompute the delta, or change, on the previous district value.

Another trick to optimizing performance is in the way we determine the changing boundaries.  I’ll describe the problem using the census geographies of counties, tracts, and blocks. The structure and detail of the underlying data yielded computationally expensive queries against the block geometries.  We came up with a method of searching for the geographies in a hierarchical fashion — searching the counties first, then continuing to the next smallest-scale geography only if there was any remaining geometry left in the query.  We did the same for the tracts, and took a shortcut at the block level to exclude the block geometries.  This increased server side performance considerably.

King William County

King William county is comprised of 22 Voter Tabulation Districts and 1,527 Census Blocks.

Consider the following scenario: a user wants to move King William County (highlighted in yellow) from District 1, which is over populated, to District 3, which is under populated. Changing the boundaries with all the blocks in King William County would require testing at least 4,000 blocks for spatial intersections, then aggregating 1,527 data values, and recomputing the spatial aggregate (union) of those 1,527 geometries. With our hierarchical approach, we can change the boundary of the district with the county boundary, and change the population totals by the county’s population. A few orders of magnitude fewer operations to perform, and much faster from the user’s perspective.

Lessons Learned

Throughout the DistrictBuilder development process, the same core performance challenge has arisen: the volume of data must be reduced. This applies to all aspects of the application:

  • Map Tiles: pre-render tiles to keep the number of rendered tiles to a minimum at runtime.
  • Map Features: deliver to the browser only as much information as you can see (perhaps even less).
  • Database Queries: do anything possible to ensure that geometric operations are performed on simplified geometries.
  • Aggregating Statistics: cache whatever you can, and only compute the difference from the last cache state.

The above steps reduced the sheer number of operations and volume of processing that both the server and browser need to complete when creating new districts. These are lessons that translate well to any “big data” problem, and are crucial in bringing sophisticated GIS operations to the web.

Comments (1)
Aug 19, 2011

Pending edit system using Django

By

A common concern when we talk to people about OpenTreeMap is how much to trust the public with an organisation’s tree inventory. Every implementation of this open source system has a different answer. The original site, UrbanForestMap.org, allows a logged-in user to edit almost every bit of information they gather about a tree. PhillyTreeMap.org requires a certain level of reputation before a user can edit everything, but even a new user has considerable edit capabilities. The most recent implementation (still a work-in-progress) introduces a bit of oversight to public edits. The managing group wanted to double-check changes to officially inventoried trees, but didn’t want to get in the way of people adding and editing their own trees.

Lets look at how this changes the user story first:

A logged-in user makes an edit to a tree. The system needs to decide if these changes are applied to the tree or placed in a pending queue. If this is a publicly-entered tree, the changes are applied to the tree. (Start new requirements) If this is an inventory tree, and the user isn’t a member of a management group, add the change to the pending queue. Display any pending changes reasonably near the appropriate current value. (End new requirements)

Most of this happens behind the scenes in the saving logic. I added a bit of code to the top of our tree updater to check if the pending system is active, the user’s permissions and the tree’s origins. If everything checks out, the change goes straight into the updater code. For changes that go into the pending queue, the path to becoming an official change is a little more tortuous.

Since we’re storing these changes for later review, they have to go into the database. I created a new table to hold onto the original tree’s id, the field being changed and the new value as well as the user who submitted it, a date/time stamp and a status field. Each pending change is stored separately; even if the user makes more than one change to the tree, each ‘pend’ can be applied individually.

The rest of the pending system is eye-candy and a bit of slightly tedious templating. Almost every field on a tree’s detail page now needs to check two new things: are there any pending changes for this field, and does this user have permission to approve/disapprove pending edits. If there are pending edits, the new values are added below the current official value. When a managing user views the page, small approve and disapprove buttons also appear next to each pending change. Throw in a management-access-only page for some bulk evaluation and the system is complete!

Comments Off
Aug 12, 2011

Bring on the data focused basemaps, Esri

By

It’s great to read that Esri is working on features and basemaps to include within ArcGIS.com to support data visualization.     Sometimes the map isn’t the focus; sometimes the data is the focus.  A few weeks ago, Bern Szukalski wrote an article for the Esri Insider blog that spoke about Esri’s efforts to create new basemaps including basemaps for data visualization purposes.   I think this is a great move for Esri.   Last fall, I suggested such a muted basemap via the ideas.arcgis.com portal, so I was quite excited to hear it is in the works.

A post today, also by Bern, explained a new feature within ArcGIS.com to allow the user to mute the basemap by adjusting it’s display transparency.    The HunchLab team stumbled upon this idea a few months back and it’s been a great way to use the existing topographic basemap.    In our demo instance of HunchLab, we are using the ArcGIS.com Topographic tiles set to a transparency of 60%.   You can see what it looks like below.

Kudos to Esri — keep the basemap options coming!

Comments Off
  • Azavea Journal
  • Azavea Atlas
  • Azavea Labs
  • Azavea.com