The data that we’ll be viewing is from the National Elevation Database (ND) developed by the U.S. Geological Survey. In a tutorial on the GeoTrellis documentation site, we saw how to get the data and convert the GeoTIFF rasters into the ARG format required by GeoTrellis. With GeoTrellis 0.8.1, when you use the geotrellis-server project to develop REST services with GeoTrellis, you have the option to also serve an administration tool on the /admin path of the Jetty server. You only have to make sure that some options in your application configuration are set correctly (or not set, as these are the defaults). The settings are geotrellis.admin.serve-site = “yes” and geotrellis.admin.serve-from-jar = “yes”. The first setting tells the GeoTrellis server to start the services required by the admin site. The second tells the GeoTrellis server to serve up the static files that make up the admin site. The javascript, css and html files that make up the site are packaged into the goetrellis-server JAR.

Starting the admin tool

If we run the server (see the tutorial or the README on the github page for the code for how to run the server), we can go to http://localhost:8880/admin/ and get to the admin site:

In this case, the admin site is showing data from a catalog that I have locally. The catalog’s name is “Catalog of Rob’s Data”, and it has a number of data stores (indicated by the folders on the left). The catalog that the admin site displays is the one set in the configuration as geotrellis.catalog. For more information about the GeoTrellis catalog, see the overview section on the subject in the documentation.

We can open up the ‘NED data’ data store and select Philadelphia to visualize the raster:

The raster fits over the base map perfectly. This is because both the base layer and the raster have a projection of Web Mercator (ESPG:3857); if the raster was not in that projection, it would be places in an incorrect spot on the map (although you would still be able to view it).

Changing the base layer

We can choose another base layer using the Leaflet layer chooser on the right side of the map:

Let’s change it to the Stamen Toner layer, and increase the opacity of the raster using the Opacity slider, in order to get a more clear view of the coloring:

Changing the Color Ramp

Seeing Raster Values

The Mythical City of Atlantis – just as legendary as GeoTrellis 0.8′s advanced geoprocessing power! (image credit)

The GeoTrellis team is very excited to announce the availability of GeoTrellis 0.8 (codename “Atlantis”), which is a major new release that is a huge step forward towards our goal of a general purpose, high performance geoprocessing library and runtime designed to perform and scale for the web.

First of all, you should check out our new documentation site — http://geotrellis.github.com/ — which features a great deal of new documentation and tutorial material. Credit is due to Rob Emanuele for the terrific new organization and layout.

Secondly, take a look at the Azavea Labs blog Adam Hinz recently published (example running above) which describes process of building a kernel density service using GeoTrellis that features some of the new 0.8 functionality. Adam’s post details step-by-step the development of a WMS service that should be helpful as you think about how you might develop your own services with GeoTrellis. It is a companion blog to the Azavea Atlas blog series being written by John Branigan that explores map algebra concepts through the process of creating a site suitability model using wildlife habitat preferences.

As you delve into GeoTrellis 0.8 in more depth, here are some new features you may want to explore:

• A new suite of raster operations in the focal category of map algebra.
  • For more information, see: http://geotrellis.github.com/operations/raster/focal.html
• Simplified raster rendering with a built-in suite of color ramps for data cartography.
  • For more information, see: http://geotrellis.github.com/overviews/rendering.html
• Zonal summary (by feature) operations, including a caching mechanism for tiled rasters for improved performance.
  • For more information, see: http://geotrellis.github.com/operations/raster/zonal.html
• New vector feature framework for operations on points, lines, polygons with a suite of vector operations, as well as re-engineered rasterization operations.
  • For more information, see: http://geotrellis.github.com/overviews/vector.html and http://geotrellis.github.com/operations/feature.html

Please let the team know — via the #geotrellis channel on Freenode IRC or the geotrellis-user Google Group mailing list — if you have any comments or suggestions. We will likely release a minor bugfix release (0.8.1) in the future.

Github: http://github.com/geotrellis/geotrellis
Maven repository: https://oss.sonatype.org/content/repositories/releases/
API Scaladocs: http://geotrellis.github.com/geotrellis/latest/api
Issue tracker: https://github.com/geotrellis/geotrellis/issues?milestone=3&state=open
Mailing list: https://groups.google.com/group/geotrellis-user
IRC: #geotrellis on freenode

GeoTrellis is released under the GPL V3 license.

GeoTrellis is a high performance geoprocessing engine and programming toolkit developed here at Azavea.

There’s an example showing the Western Jackalope sightings running at http://207.245.89.238/labs1-demo/index.html.

Scala Environment

We’re using the same made-up dataset that John used in the Atlas Article. The first step is getting our environment setup. If performance is key, we’ve found the Oracle JVM to be a bit faster than OpenJDK. For simplicity, we’re going to use the OpenJDK found in the standard apt repo:

To build scala projects we use “sbt”. If you don’t have it, you can install it:

To get moving we need two boilerplate files. The first is our project definition (which lives in the root in build.sbt)

The second file is needed to explain to GeoTrellis what classes we want exposed and on what port by updating the settings in the application.conf file

To make sure everything is looking good we’re going to setup a simple hello world service. We’re using Jersey/JAXRS for web services in this tutorial but you should feel free to use whatever.

Testing Service

At this point we can do “./sbt run” and see our service startup by going to http://localhost:8888/hello and you should see “Hello World”. Jersey uses annotation to determine what methods to expose. We use the “@Path(…)” annotation to represent that path of a server (“/hello” in this example). Decorating our method with @GET exposes it via GET requests.

Formatting Our Data

Before we can analyze the data we need to grab some data and prep our environment. We use GeoTools to parse our shapefile in to GeoTrellis features.

The last few lines set up our GeoTrellis server executor. Usually GeoTrellis reads a catalog file to determine where relevant raster data is on the server. Since we’re only generating rasters (not reading them) we used a blank catalog. The server is responsible for running operations.

The last few lines setup our color handling. All of the regular color ramps have their alpha values set to 100%. For our heat map we want low values to fade away. In the above case the lowest values are dropped completely (0×00000000) and the remaining values are set to 50% opacity.

Kernel Density

Here’s a kernel density service that we’ll be using:

We start out grabbing our bounding box (xmin, ymin, xmax, ymax), the width and height, size of our kernel and the kernel’s spread (standard deviation in meters). “Context” is a global object in “demo.azavea” that we use to store some static stuff, like our feature and server.

GeoTrellis provides a few operations for converting strings to relevant objects:

• ParseRasterExtent
• ParseInt
• ParseDouble

More can be found in the “geotrellis.rest.op.string” package

A common kernel to use for Kernel Density operations is the Gaussian, which we create using CreateGaussianRaster. In our example we allow the standard deviation to be set by the service and set the amplitude to 100.

The main show is the KernelDensity operation. We’re using the feature set that we loaded from the shapefile. The second argument is a function to convert our feature value in a number. In our case each feature is a Point[Int] so we use the identity function.

This kernel density server represents everything in meters and kilometers so we need to scale the values appropriately before we dive in to the rendering (spreadOp and sizeOp).

The last 10 lines render the PNG and and return it. Since we’re generating tiles on the fly we’re not going to use quantile breaks because the tile edges won’t line up. Instead, we create our own break array and use RenderPng directly.

Resources

For more information checkout http://geotrellis.github.com/ or chat with us at #geotrellis on freenode.

The code can be found on github at https://github.com/geotrellis/labs1-demo

Unit Testing / TDD

As it turns out, there aren’t a whole lot of great tools in the .NET ecosystem that will unit test JavaScript source files.  Sure, you could install nodejs on a windows box, or even run your tests in the browser through QUnit.  It really comes down to a matter of taste, and my preference is to have fewer moving pieces in the build chain if possible.  It turns out that JSTest.NET fits into our .NET ecosystem nicely, so I wanted to give it a shot (it also helps that it’s a package available in NuGet).

One of the interesting parts about JSTest.NET is that it runs in the Windows Scripting Host, which is a neat way to run JavaScript programs from the command line in Windows. If you’ve ever double-clicked on a JavaScript file, and received a cryptic error, you probably ran the file through the Windows Scripting Host without realizing it. Since the executing process is this scripting host, you don’t have access to normal browser stuff, like “window” or “document”. This turns out to be important, especially in web applications.

JSTest.NET + require.js

Require.js is a great way to write modular JavaScript code in client and server environments. Part of the design of require.js is to enforce an Asynchonous Module Definition (AMD) API style in your JavaScript source.  This helps developers in many ways; I’m not here to sing its praises, but it has helped our teams build reliable apps.

Part of AMD’s architectural design means that the browser will asynchronously load JavaScript modules when they are needed, based on the name of the module and an optional URL mapping. It’s up to the engineer’s discretion, but these modules could be quite atomic and have no dependencies (including the global ‘window’ object in every browser). Unit testing libraries that operate outside the browser (and therefore don’t have any real URLs associated with them), seems to throw a wrench in the whole AMD process.

As it turns out, it’s no wrench at all. Require.js loads modules only once, and keeps defined modules around until they are explicitly discarded. With this knowledge, it’s trivial to create a set of tests by manually including module dependencies. It’s best to see this by example. Let’s get started.

Processor Example

Assume I have a JavaScript library module that processes data in some fashion:

This module defines itself as ‘processor’. This is not strictly necessary when using require.js, but it is quite important in the require.js loading mechanism (it is the only way to retrieve a module from require.js after it has been loaded). This module also includes the special module id ‘module’ as the only dependency.

To test this module with require, we need a JavaScript “test suite” file, and a C# test file which will be executed by our test runner. This puts all of our code and tests in JavaScript, and the C# source file merely manages the suite setup for all tests in the JavaScript “test suite”. Our “test suite” is pretty simple right now; let’s test that the loading of the module went swimmingly:

Wait, what? Why are there two “require” calls in there?  The first invocation of require (with an array argument) is the form that registers a module that has previously been declared with “define”.  The second invocation of require (with a module name argument) is the form that retrieves the cached module by that same name. You can test this out in any web page that uses require: if you use open up a JavaScript console in the browser, you can use the form “require(‘modulename’)” to get the currently loaded module (that is, if you know the module names that the application developer is using). The two of these calls together effectively 1) register the module, then 2) return the loaded url from require’s cache. Now that we’ve convinced require not to load any modules via XMLHttpRequests, we can start testing our modules.

Next, we need something to pilot the tests: a C# class loads up the scripts and starts the tests:

I’ll walk you through what’s happening:

1. JSTest loads all the global function calls in the suite on line 12 into a set of NUnit DataPoints.
2. NUnit runs the Test method (since it’s a Theory, and not a Test) against all the discovered DataPoint types in the class — in our case, it’s an array of TestCase objects, where each one is the name of a global function to execute.
3. JSTest starts composing a TestScript, and includes the JsAssert library on line 20 (this provides the ‘assert’ object)
4. We added our own lines below that add the require.js and processor.js JavaScript sources to the TestScript on line 23 and 24
5. JSTest then adds the test suite to the TestScript on line 25
6. JSTest runs this TestScript.

This process makes it clear that:

• JsAssert is the first JavaScript source loaded.
• Require.js is loaded prior to our modules.
• Our test suite is the last JavaScript source loaded.
• The test method will be run once for each global function found in the test suite.

Processor/Renderer Example

This is all well and good, but what happens when we start adding module dependencies?  Let’s say we have a second module, which depends on our processor:

This module loads the ‘processor’ module as a dependency. To test this in JSTest, simply create the “test suite” in JavaScript:

… and create the C# test pilot, and notice that we are including processor.js manually:

It is a little annoying that you need to list all dependencies of an individual module, but we’re lucky to have relatively small numbers of modules dependent on one another — a page may load many modules, but the unit tests are discrete enough that those dependencies aren’t a big pain (yet).

Now that we can start unit testing our JavaScript from NUnit, we’re going to get crazy!  But what happens when your module uses something like jQuery?  Remember how I mentioned at the beginning of this article that there is no “window” or “document” object?  Well, there’s a way to do that, too.  Tune in for the next installment, and I’ll demonstrate how you can use a headless browser to test!

(All these code examples are available in gist form, and you are free to start using them to run your own tests.)

Intro

This brief article documents some early exploration into using WMS (Web Map Service) with the new Google Maps V2 API. It is intended as a reference to help someone trying to get WMS tiles (IE from GeoServer) onto an Android map.

WMS is used to serve map tiles over HTTP by back end frameworks like GeoServer.  Some set of geo-referenced data, typically shape files or data stored in a PostGIS database, are returned as raster map tiles.  In the past, this data has been consumed by web applications using a client library such as Leaflet or OpenLayers.  With Google’s v2 mapping API for android, it is now relatively straightforward to build Android apps that combine WMS tiles with Google’s base maps and other data such as vector shapes and map markers.

For basic getting started info for v2 Maps, see the Google Developer’s site for the v2 API. This article assumes a working v2 setup with the sample code running without error. After downloading the google play SDK and setting up the library make sure you can view the TileOverlayDemo ($ANDROID_SDK_ROOT/extras/google/google_play_services/samples/maps)

PhillyTreeMap Android App (Proj. Release Spring 2013) showing WMS technique described in the article.

 Extending the UrlTileProvider class.

The v2 API provides the UrlTileProvider class, a partial implementation of the TileProvider class which allows developers to pull in map tiles by composing a URL string.

The API to UrlTileProvider is its getTileUrl method. To request a WMS tile,
we override this method to compose the right URL, and the Android mapping SDK does the rest for us. It seems simple enough, but the problem is that the signature of getTileUrl which is getTileUrl(int x, int y, int zoom) provides tile indexes (x and y) and a zoom level, but WMS requires that we provide a bounding box (xmin, ymin, xmax, ymax) in the request URL. The x, y, zoom parameters provide us enough information to figure this out, but we have to do a little bit of math.

Calculating the map bounds

We know the bounds of the entire map which is square. (roughly -20037508m to 20037508m in both directions using Web Mercator. See the graphic below or the map tiler site for exact values.) We don’t use Latitude/Longitude though, because it is unprojected, and this will cause map distortions.

From the google api docs for TileOverlay:

Note that the world is projected using the Mercator projection (see Wikipedia) with the left (west) side of the map corresponding to -180 degrees of longitude and the right (east) side of the map corresponding to 180 degrees of longitude. To make the map square, the top (north) side of the map corresponds to 85.0511 degrees of latitude and the bottom (south) side of the map corresponds to -85.0511 degrees of latitude. Areas outside this latitude range are not rendered.

Dividing by the number of tiles for a given zoom level.

The number of tiles in either x or z at any zoom level is n = 2^z. With this, and the bounds of the map, we can figure out the size of the tile. Using this information combined with the maps origin (see graphic) and the x, y, zoom data for a given tile, we can find out its bounding box.

Again from the google api docs for TileOverlay:

At each zoom level, the map is divided into tiles and only the tiles that overlap the screen are downloaded and rendered. Each tile is square and the map is divided into tiles as follows:

• At zoom level 0, one tile represents the entire world. The coordinates of that tile are (x, y) = (0, 0).
• At zoom level 1, the world is divided into 4 tiles arranged in a 2 x 2 grid.
• …
• At zoom level N, the world is divided into 4N tiles arranged in a 2^N x 2^N grid.”

Summary

• zoom level: z = [0..21]
  (See GoogleMap.get[Min|Max]ZoomLevel)
• map size: S = 20037508.34789244 * 2
  This constant comes from converting the lat/long values above to EPSG:900913, Web Mercator. Again, see this page on maptiler for a fantastic visual explanation.
• tile size = S / Math.pow(2, z)
  So @ zoom level 0, S is the full map, at zoom level 1 there are 2×2 tiles, and at zoom 3 there are 8×8 tiles and so forth.
• tile origin = (-20037508.34789244, 20037508.34789244)
• minX of the tiles bbox (for tile index x,y) = origin.x + x * S
  Where x is the tile index in the east-west direction passed to the getTileUrl function discussed above.
• maxX of the tiles bbox (for tile index x,y) = origin.x + (x+1) * S
  x+1 because we are looking for the right edge of the tile.
• minY of the tiles bbox (for tile index x,y) = origin.y + y * S
• maxY of the tiles bbox (for tile index x,y)= origin.y + (y+1) * S

Demo Code

In addition to the following code snippets, I’ve put a sample project on github.

Here is a WMSTileProvider class, which inherits from UrlTileProvider. It supports the above bounding box calculation.

import com.google.android.gms.maps.model.UrlTileProvider;

public abstract class WMSTileProvider extends UrlTileProvider {

    // Web Mercator n/w corner of the map.
    private static final double[] TILE_ORIGIN = {-20037508.34789244, 20037508.34789244};
    //array indexes for that data
    private static final int ORIG_X = 0; 
    private static final int ORIG_Y = 1; // "

    // Size of square world map in meters, using WebMerc projection.
    private static final double MAP_SIZE = 20037508.34789244 * 2;

    // array indexes for array to hold bounding boxes.
    protected static final int MINX = 0;
    protected static final int MAXX = 1;
    protected static final int MINY = 2;
    protected static final int MAXY = 3;

    // Construct with tile size in pixels, normally 256, see parent class.
    public WMSTileProvider(int x, int y) {
    	super(x, y);
    }

    // Return a web Mercator bounding box given tile x/y indexes and a zoom
    // level.
    protected double[] getBoundingBox(int x, int y, int zoom) {
    	double tileSize = MAP_SIZE / Math.pow(2, zoom);
    	double minx = TILE_ORIGIN[ORIG_X] + x * tileSize;
    	double maxx = TILE_ORIGIN[ORIG_X] + (x+1) * tileSize;
    	double miny = TILE_ORIGIN[ORIG_Y] - (y+1) * tileSize;
    	double maxy = TILE_ORIGIN[ORIG_Y] - y * tileSize;

    	double[] bbox = new double[4];
    	bbox[MINX] = minx;
    	bbox[MINY] = miny;
    	bbox[MAXX] = maxx;
    	bbox[MAXY] = maxy;

    	return bbox;
    }

}

You might use this class in a factory class as follows:

import java.net.MalformedURLException;
import java.net.URL;
import java.util.Locale;
import android.util.Log;
import com.google.android.gms.maps.model.TileProvider;

public class TileProviderFactory {

	private static final String GEOSERVER_FORMAT =
    		"http://yourApp.org/geoserver/wms" +
    		"?service=WMS" +
    		"&version=1.1.1" +  			
    		"&request=GetMap" +
    		"&layers=yourLayer" +
    		"&bbox=%f,%f,%f,%f" +
    		"&width=256" +
    		"&height=256" +
    		"&srs=EPSG:900913" +
    		"&format=image/png" +				
    		"&transparent=true";	

	// return a geoserver wms tile layer
	private static TileProvider getTileProvider() {
		TileProvider tileProvider = new WMSTileProvider(256,256) {

	        @Override
	        public synchronized URL getTileUrl(int x, int y, int zoom) {
	        	double[] bbox = getBoundingBox(x, y, zoom);
	            String s = String.format(Locale.US, GEOSERVER_FORMAT, bbox[MINX], 
	            		bbox[MINY], bbox[MAXX], bbox[MAXY]);
	            URL url = null;
	            try {
	                url = new URL(s);
	            } catch (MalformedURLException e) {
	                throw new AssertionError(e);
	            }
	            return url;
	        }
		};
		return tileProvider;
	}

}

So your Activity code (again see the sample Google maps code referenced above) would have the following calls to add the overlay:

public class MapDisplay extends android.support.v4.app.FragmentActivity {

	//...

	private void setUpMap() {
	    TileProvider tileProvider = TileProviderFactory.getTileProvider();
	    mMap.addTileOverlay(new TileOverlayOptions().tileProvider(tileProvider));    
	}

Conclusion

This article presented a demonstration of a simple WMS client using Google’s Android v2 Maps API. It covered the math involved with converting from tile index/zoom level to Web Mercator bounding box, and showed how to compose a URL using these values and an instance of Google’s UrlTileProvider class.

The Early Learning portal has information about early childhood learning centers in and around Chicago.  It used to be difficult for parents and families to collect, explore, and share this information. Now that all the early learning centers in the city are mapped and listed in one place, it’s easier for parents and families to discover what services are available.

The portal addresses a simple data problem once the information has been collected. There is very little geographic processing that occurs in the application, and the portal’s main function is  to display location data.  It’s not that often that a school will move, so most of the data is relatively static.

Working on a web application with only a small amount of geographic processing was a little unusual for us. While unusual, we were pretty excited about it because we focused heavily on the user experience of the portal, in order to make it easier to use.  We chose technologies that were open and made the portal usable on desktops, tablets, and smartphones. The Google Maps API (which includes Geocoding and Directions), along with Twitter Bootstrap and Twilio enables the application to play friendly with many different devices and capabilities.

Not only that, but we’re excited about what’s coming next with the City of Chicago. The SmartChicago Collaborative is also working with the City of Chicago on Windy Grid, a massive, real-time data infrastructure. Building that infrastructure will enable much deeper analysis and hopefully connect many different agencies and their data tools together.

Deep analytics and big data at the city level would be really exciting, and we’re really happy to see the City of Chicago embracing these goals.  It means that web applications like the Chicago Early Learning portal are just the tip of the iceberg, and we’re anticipating great things in the future, both from the SmartChicago Collaborative, and the City of Chicago.

With the Cicero API’s map call, you can get a customizable image of a district’s boundaries, and easily embed one or many of these into an interactive map on your website with just a bit of JavaScript.

In this blog post, I’ll explain how to:

1. secure your Cicero account info, set up PHP cURL, and get a token from Cicero
2. get information on a legislative district that represents a particular address
3. get a boundary image of that district
4. and display that image in a Google Maps widget.

I’ll be using a mix of PHP, HTML, and JavaScript. Really though, you could do this using any language, and any web mapping framework – such as the excellent open-source Leaflet.

This blog post is written for the absolute beginner. I expect you’ve done some programming and seen some PHP, HTML, and JavaScript code before, and have a handle on at least the concept of a ReST API. You will need a webserver running PHP, even if that’s only a local test server like XAMPP. Of course, you could follow these steps on a production webserver as well. I will do my best to make no other assumptions, but if you get stuck you can always ask me for help. More advanced readers may want to save time and skip ahead to step 3 or 4, where we start actually using Cicero. Many people, beginners and advanced users alike, may want to skim over significant portions of this post. It may be long, but I assure you, it’s not actually that hard!

All the example files I refer to in this post are available in the cicero-examples GitHub repository.

Finally, in a few weeks, I’ll be writing a second blog post that expands on the simple example code I use in this one, by employing JQuery and extra features of the Google Maps API to make your district maps even more interactive. Stay tuned!

Step 1: Secure your Cicero Account Details

To get a token from Cicero, you’ll need to pass your username and password to the API via an HTTP POST request. We’re going to use an extension PHP has called cURL to do that in step 2. First we need to secure your account info. While visitors can’t see PHP, in some circumstances (if logging is on by mistake, etc) they could find out your Cicero details if you don’t put some protection in place. I recommend making a separate PHP file with just your Cicero username and password set to constants. Put that file in a folder outside of your website’s main web-accessible directory and then use PHP’s include() or require() statements to bring that separate file into your main PHP code. You can direct PHP to look for include files in other directories either through your php.ini file’s include path or simply specifying a relative path in your include statement. If you don’t have access to a protected directory outside your main web-accessible one, if you’re running the Apache webserver, you can protect include files via clever use of the .htaccess file (hint: it’s the second post in that article).

Here (github) is my include.php file:

Penny for Your Address?

Notice the third variable I put into this file, $search_loc. It doesn’t actually have anything to do with your Cicero account details. This will be the address we will pull the elected official and map info from in this example. We’ll use Azavea’s Philadelphia office for this post, but you can experiment later and set this to a different address. More data (state, country and zip code, as well as city and street) will ensure a greater chance of successful geocoding and could make response time faster. Obviously, in a more complete web application, you might be getting this address from a user-submitted form rather than hardcoding it like we are for our example.

Step 2: Setup PHP cURL

Moving along! We said earlier we’d be using PHP’s cURL extension to make POST and GET requests to Cicero. There are other ways to do this, but cURL is widely used and fairly easy – depending on your server. Sometimes, it’s not enabled by default in certain PHP installations. You can find out if you have it installed by making and navigating to a PHP page with the phpinfo() function:

<?php
phpinfo();
?>

Search the page (Ctrl+F works well) for “curl” info. If it’s there, congrats! If not, and you have access to your php.ini file, search for something like the following line:

;extension=php_curl.dll

If your line has a semicolon in front of it like above (that’s PHP’s line comment symbol), go ahead and delete just the ; to enable the extension. If you can’t find the line, just add it to the file without the semicolon. If you’re using Linux or Mac OS X, the extension will be named php_curl.so instead of .dll – .dll is for Windows.

Next, we’ll need to write a function to use cURL to send data up to Cicero and process the responses we get back. I’ve shamlessly stolen the function in my get_cicero_info.php file (on github) from my colleague Joe Tricarico’s excellent Hello World – PHP example in the Cicero documentation:

All get_response() does is take any URL, and optionally any data to be used in a POST request ($postfields), initializes cURL, makes the appropriate POST or GET request to the url specified, reads the resulting response into the variable named $json, and then runs $json through a PHP library function called json_decode to give us a variable with data we can manipulate and use in PHP.

JSON stands for JavaScript Object Notation, and if you do anything with APIs like Cicero, it should be your best friend because it is just So. Much. Easier. to read as a human and use in programs than it’s older cousin, XML (eXtensible Markup Language). Cicero supports both JSON and XML responses, but we’ll use JSON in this tutorial. If you want to learn more about JSON and how to use it with PHP, Michael Nitschinger has a fantastic blog post on the subject.

Step 3: Get a Cicero Token

OK, we finally get to do something with the API in this step! Also in my get_cicero_info.php file, I’ve written another function, get_cicero_token(), to do exactly that:

This function I also based off parts of Joe’s Hello World – PHP example, with a few modifications. We take the $username and $password from our previously-included separate file, and use PHP’s urlencode() function to properly encode them before we send them over HTTP to Cicero. (Some characters like commas, apostrophes, and the like can cause problems if they were sent without first URL encoding them.) Then, we use our get_response() function written earlier, give it the url to get a new token, and the username and password as “postfields”, joined by an ampersand (&) just like you might see in a URL. (PHP string interpolation works easily enough we can just put $username and $password into the postfields string – great!). Now that we (hopefully) have a response, we check that the “success” attribute of the response object that Cicero gave us is True. If it’s not, we’ll exit the PHP program because we can’t do anything without a token. Your code may have a bug, or your Internet connection may be down, or Cicero may be temporarily unavailable. If we did get a successful response, this function makes a $token_info array with both our token value and Cicero user ID. We’ll use this array for other API calls.

Step 4: Query Cicero for Officials

(i.e, Where are You and What do You Want?)

This is where it gets exciting. Now we’ll write a PHP function to query Cicero for elected official data, and get what we need to later request a district boundary image from Cicero in step 5 and build a Google Maps widget in step 6.

Let’s take a look at my get_cicero_info.php file’s get_map_info_by_location_and_district_type() function below (boy, that’s a mouthful):

This function takes the $token_info array our last function made, as well as the $search_loc that we hardcoded in our include.php file, and $district_type (new) – which I’ve set in this example to STATE_LOWER, one of the many district types available in Cicero. If you wanted something else (like your US House district), just use the appropriate Cicero district type name in place of what I’ve used. Some addresses may belong to multiple districts of a certain district type (the LOCAL type is a good example, when a city might have several at-large seats like a Mayor and Controller, etc), so even when filtering this way you may need to put in extra program logic to find exactly the right one you want. When it returns more than one, Cicero orders officials by last name alphabetically.

Once again, we URL encode the data we’ll pass to Cicero (useful in particular for the $search_loc address, as it converts spaces to +’s and other appropriate URL formatting changes). Our function then builds a $query_string from our information, and specifies that we want JSON output (with “format=json”). Note that since $token_info is an array, we have to add curly braces to our string interpolation syntax here to get at our token and user ID elements:

We then append our $query_string to the main URL for Cicero’s official call, and get a response:

With Cicero calls that use geocoding, like official, a list of “candidates” objects will be part of the response. Sometimes a given address will geocode to multiple lat/long points – the official information for each of these points will be a candidate object in the list. Usually, only one candidate object is returned though, so we assume that in this example. In our code, there’s a check to make sure that Cicero was able to find at least one geocoded candidate location and get the officials from the address you gave it:

Since $candidates is a list, even if it only contains one object like our example right now, we have to reference the 0th element in the rest of our code.

We want to get the geocoded point Cicero produced for our address, because we’ll use this later in the Google Maps widget as the center of the map. To do this, we’ll get the x and y values of the 0th candidate Cicero gave us, and make a new array ($lat_long_of_search_loc) that contains them:

This function also gets our crucial $unique_district_id for the district we want to display on the map, too. Each district in Cicero has a unique district ID (also called a “primary key”), and while you can request a district map with other information, after you’ve gotten an official object this is the easiest and most fool-proof:

Finally, we’ll stuff our address’s lat/long and the unique district ID into a new PHP array, $map_info, so we can use them in our final PHP function:

Step 5: Grab a District Map and Get Ready for JavaScript

So we have a latitude and longitude to center a map on, and we have a unique district ID from Cicero. We’re almost done with our PHP server-side programming.

Let’s take a look at the get_cicero_info.php file‘s get_district_map() function, and the last few lines in the file:

What’s going on here? This function takes our $token_info array again, along with our new $map_info array. We build a new $query_string and $map_url for the map call this time. There are a few parameters in this one that we haven’t seen until now:

• include_image_data – This tells Cicero to send along the base64 image data as well as a link to a PNG image hosted on our servers. This is optional, but has advantages over using the hosted image URL which is why we use it in this example. If you take a look at the JSON response, you’ll see the base64 data as a huge stream of seemingly senseless characters – it’s how we can represent an image in text form. You can actually put this base64 stream in the src= part of the HTML image tag, and browsers will render it as an image. Pretty cool, huh? If you use the URL given to the district map image given in Cicero’s response instead, be warned that image is only guaranteed to stay on our servers for 7 days. The advantage to using base64 data is that we will already have it handy when it comes time to display the map,  so we won’t need to download the PNG then which saves us time and bandwidth.
• srs=4326 – If you’re not a geographer, you might not know what a Spatial Reference System (SRS) is, nor would you know what EPSG codes or other SRIDs are. But you don’t really need to! Just know for now that Google Maps takes latitude and longitude data, which is EPSG code 4326, and the one we’ll be using. If you don’t set this, Cicero defaults to a map projection system called “Web Mercator,” which might play nice with other mapping frameworks, but not Google Maps. (Even though Google Maps is based on Web Mercator, the Image Overlay function we’ll be using assumes regular latitude and longitude.)
• width=500&height=500 – These specify the dimensions of the map image Cicero will return, and are only two of a slew of other map style options – including colors! The max is 3300 pixels, but 500 or 1000 loads faster and the visual difference is negligible except when a user zooms in closely. In my upcoming second blog post, we’ll discuss dynamically getting district overlay images of different sizes depending on the user’s browser window size and as a user zooms into the map, which will maintain image quality much better. For now though, we’ll keep it simple and hardcode 500 pixels, and limit the maximum zoom level in Google Maps in step 6.

Our function gets a response from Cicero, verifies that a map actually was returned in the response (exits the script if not), and then gets the data for the map object (including the image URL, and the lat/long data to properly position it on our map widget) and sets that to the $district_map_image_data array.

Since we’ll need to use this data in JavaScript code next, we return a new json_encoded PHP array of the lat/long of our center point, and the $district_map_image_data. Finally, the last three lines execute all our PHP functions and create a final $map_data_for_javascript variable with the info we’ll need as we move to client-side JavaScript in step 6!

Step 6: To the Browser!

At last! Let’s write a few lines of HTML and JavaScript, and we’ll finally get to see a district map from Cicero.

I’ve used Google’s Ground Overlay tutorial and code sample file as a base for my display_gmaps_overlay.php file. I’ll highlight important parts here, but you should really take a look yourself and follow along.

NOTE: My demo file uses the Google Maps JavaScript API without an API key. Google will let you do this for development, but if you want to put this code on a live site, you’ll need to register with them and get a key. If you’re using my file, you will need to add your key in the “googleapis.com” URL.

Since PHP is executed on the server (server-side), and JavaScript is executed in the browser (client-side), there’s a small kludge to get them to work together. (I’ll improve this using JQuery in my next blog.) Even though it is mostly HTML and JavaScript, display_gmaps_overlay.php is still a PHP file. We need to include get_cicero_info.php:

to give us access to the output of the functions we just made, and then also set our PHP variable $map_data_for_javascript to a JavaScript variable. We can do this, oddly enough, with a “php echo” statement:

The PHP in this file and the included one is processed on the server, and the new JavaScript var map_data_from_php is set to the output of PHP’s echo of $map_data_for_javascript (which, as you should remember, we made as a JSON-formatted string with json_encode), and the client is none the wiser. We’ve taken a PHP object and made it a JavaScript object!

Following Google’s code sample, we next need to create a map object centered at our address’s geocoded latitude/longitude. Because map_data_from_php is JSON, we can navigate the fields using simple dot syntax:

Next comes a tricky part: defining the bounds of our district image – the lat/long of where it starts and ends. If we didn’t define these, Google Maps would have no idea where to put our image. If we define them incorrectly, Google Maps either won’t show our district image at all or it will be horribly contorted:

Let’s step through this, because the ordering of your arguments is very important here. There are two Google Maps API calls used – google.maps.LatLngBounds(), and google.maps.LatLng(). The first takes two (lat,long) pairs as arguments and makes what we geographers call a “bounding box”. Assuming North is “up”, Google specifies these pairs must be given in the order of southwest (“bottom left”) first, northeast (“top right”) second. IE, LatLngBounds(southwest point, northeast point). For example, if you wanted to draw a bounding box over most of the state of New Jersey, you’d put Philadelphia’s lat/long point first, and New York City’s lat/long point second. The second API call, LatLng(), makes *one* (lat,long) pair given two decimal numbers.

We’re using values from the “extent” object of the “district_map_image_data” object of the “map_data_from_php” variable to make all of these points. In JavaScript’s dot syntax, that’s map_data_from_php.district_map_image_data.extent.whateverValue. y_min, x_min, y_max, and x_max are all values in the extent object, that we got from the Cicero API with their default names – even though they took a trip through PHP, they’re essentially unchanged.

A note about Latitude and Longitude

Let’s look at the extent object (note, this is not in the example files):

"extent":{"x_min":-75.167614,"srid":4326,"y_max":40.002104,"y_min":39.927964,"x_max":-75.093474}

Like every location in the lower 48 contiguous United States, Philadelphia has a positive, northern latitude and a negative, western longitude. The decimal degree numbers in the extent object above may be confusing if you’re more familiar with the degrees/minutes/seconds way of measuring latitude and longitude (ie, 39°N 59′ 59″), but they represent the same thing and are easily converted. Computation is much easier with decimal degrees, which is why Google Maps and Cicero use that format.

The following can be counter-intuitive to many people: Latitude is Y, Longitude is X. That’s not the way the lines seem to appear on a globe, but it’s true if you think about which axis each is measuring.

So, Google Maps is looking for LatLngBounds(southwest point, northeast point), in that order. Use the extent values Cicero gave you to construct those points: the Southwesterly vertex of the map image will use the pairing (y_min, x_min), and the Northeasterly vertex will use the point (y_max, x_max). Doesn’t make sense? Remember: since the X/longitude values are negative in the Western hemisphere, x_min=-75.16… is actually further West than x_max=-75.09….

Near the finish line…

Next we need to set some map options and initialize the map, which I’ve kept mostly the same as in Google’s code sample. Set our “azavea” var to the center, and set the maxZoom at 16. Zooming in any higher would make our simple 500 pixel image overlay fuzzy. Note that we’ll improve this later in my next blog:

If you want a marker at your address, create it:

Finally, we’ll make our “districtoverlay” variable using the image URL given to us by Cicero, the imageBounds we defined above (and I explained at length), and add the overlay to the map:

That just about covers it! If you put these github files on a server with PHP, and navigate to display_gmaps_overlay.php using a browser, you should see an interactive Google map widget with your district image load! Congratulations! If you’re integrating this into a larger web application, perhaps this page could pop up when a representative’s name is clicked, or the map size could be shrunk down and otherwise embedded into the page. Either way, now you’ll be able to give your politically-minded website viewers some interactive, geographic context as to who their elected officials are.

Intro

One of the things that differentiates Backbone from other frameworks is that it is extremely lightweight. It specifies the framework for doing complex things, including making complex interactive user interfaces, but it doesn’t come with any usable UI widgets. The purpose of this article is to talk about some Backbone and JavaScript techniques that might make it possible to build reusable interface libraries to use with custom Backbone apps.

A typical pattern, tight view and model binding:

In a recent application, I added UI to modify the opacity and color ramp palette in a map raster layer. It essentially provides a user interface so that when the user slides the opacity slider, the data displayed becomes transparent, revealing the underlying base map. The application took advantage of Backbone Views, Models and event bindings to structure the components and interaction.

I am wary of trivial Backbone examples, but a simplified example is an opacity control in some sort of image editing app:

When the user slides the slider in the opacity view, it sets the opacity attribute of the image model. This fires a “change” event on the image model, which causes the image display view to update to the right opacity.

The problem with this structure is that what is basically a generic control, used for controlling the level of any numerical attribute, has been tightly couple to an image model, where it only controls opacity. What we’d really like to do is build a generic UI component called slider control, that we could then use for the example above. If we can do this, then we can create plug and play controls that conform to specifications. IE you might replace a slider control with a list-based control that still sets a numerical value.

So as a first shot at generalizing this interface, perhaps we could -parameterize- the Opacity Controller View. Perhaps its constructor takes not only a reference to its model, but also to an attribute parameter to change on that model. So for example, you might have the following instances:

var imageOpacityControlView = new LevelControlView({
    model: imageView,
    attribute: opacity,
});

var imageBrightnessControlView = new LevelControlView({
    model: imageView,
    attribute: brightness,
});

var image ContrastControlView = new LevelControlView({
    model: imageView,
    attribute: contrast,
})

The implementation of this LevelControlView could look like this:

views.LevelControlView = Backbone.View.extend({
    initialize: function initLevelControlview() {
        this.template = this.options.template;
        this.model = this.options.model;
        this.attribute = this.options.attribute;
    },

    events: {
        "change .slider" : "setLevel"
    },

    render: function {
        this.$el.html(this.template({}));
    },

    setLevel: function(evt) {
        // in a more advanced version we could map to a parameterized range 
        // of values.  For now just set the parameterized attribute 
        //(whether // thats opacity, contrast or brightness or something else). 
        this.model.set(this.attribute, this.$el.find(".slider").val());
    }
});

This is a pretty good first shot, but it’s slightly inflexible. If the view controls more than one model, or a model and a collection, they all have to be added explicitly. When our app changes, and the views must change to accommodate. This is true because there is too tight of a coupling between the views and their models. It also means that we overload people who are good at UI design and engineering with system integration and application architecture tasks. What you really want is to be able to write the view and view model once, in a generic, reusable way, and then forget about it, regardless if other aspects of your app changes.

The Solution

The solution is in two parts, first: lets create a model for our UI view that is dumb and doesn’t know anything about the application logic. For the level control above, the model has exactly 1 attribute, level. We then need to wire this up to our image model, so that it can update its own view.

It’s tempting to bind the models together directly, but this is probably not a good idea. The problem with this is that now we need to make either the Level Model, or the Image Model smart enough to respond to a change event in another model. There’s really no good place for this binding code with this architecture. It’s also not good for maintainability and managing complexity.

View to view model coupling is a pattern that we use a lot, and is well understood. On the other hand it is not good for our models to start listening to other models for change events. You end up with an application graph that looks like a total complex mess because there are no rules for how objects can communicate.

The second part of the solution is to bring the notion of a Controller back into Backbone application development. Let’s introduce an Image Controller. Its instance has references to the level model, and to the image model, and it wires these two models together.

controllers.ImageController = {
    initialize: function(options) {

        this.opacityControl = options.opacityControlModel;
        this.image = options.imageDisplayModel;

        // this binding is what wires the 2 models together through
        // the setOpacity method.
        this.opacityControl.on("change", this.onOpacityChange, this);
    },

    onOpacityChange: function() {
        this.image.set("opacity", this.opacityControl.get("level");)
    }

    /* We can do exactly the same thing for an arbitrary number of UI
       controls like rotation, brightness, contrast, etc.  (Of course
       it is beyond the scope of this exercise, and probably quite difficult
       to programmatically change some of these parameters because they are not
       exposed through CSS the way opacity and rotation are.
    */

};

So what you end up with is

It’s pretty clear how you could use this to wire up lots of disparate generic UI’s to their targets. You could have a levelControlModel, a brightnessControlModel and a contrastControlModel all wired up in the ImageController in similar fashion. (It would be up to your excellent, focused UI developer to figure out how you actually create a parameterized image view where you can set contrast and brightness.. no mean feat!)

Conclusion

This does a couple of things for us. First off, when it comes time to debug this application, we know that View -> Model bindings are where they always used to be, but that if there is a more complex binding, we probably need to look to the controller for the feature. The other advantage is that are user interfaces are now entirely decoupled from our application, and can therefor be extracted out to reusable UI libraries, and developed by people who focus on HCI engineering exclusively.

Going a step further, it would be entirely possible to create libraries of reusable UI components that other people could plug into their own apps. Everyone structures their backbone apps a little bit differently, some using Require.js, some using global namespaces. I think if you had a library that played as nicely as possible with all of these different approaches, you could produce a product that would be useful to a lot of different developers. Also it would be produced in a way that the UI was beautiful and highly decoupled from application logic by necessity.

Until recently any client who wanted to purchase Cicero API credits would contact us directly and request that the credits be added to their account. We then issued the client an invoice to be paid by check.  In order to enable customers to purchase credits immediately (at any time of day), eliminate the need for us to process orders by phone, and deposit paper checks, we decided to implement online payment processing accessible via the user account pages.

We considered PayPal, Google Payments and several other online payment processing services, but after extensive research we decided to go with Stripe, a secure online payment system. Despite being the “new kid on the block”, Stripe has quickly established itself as a super secure and easy-to-use alternative to some of the more well-known payment processing services available.

Processing online payments with Stripe is simple and secure. Great documentation, easy-to-use wrappers, and an extensive testing framework make developing applications a pleasure. The intuitive online dashboard makes it easy to manage and visualize customers, payments, and refunds.

How it works

After creating an account and obtaining an API key, you create a form on your website to collect your user’s payment information. Stripe’s JavaScript API is used to securely submit this data to Stripe’s servers. If the card info is valid, Stripe returns a single-use token that can be passed onto your backend to charge the customer. With this process, none of the user’s card information is sent to your servers, and the customer is never redirected to a third-party site to pay.

Stripe API libraries exist for over a dozen programming applications and frameworks. We use Stripe’s Python library to create and update customer records with contact and card information. As part of this step, Stripe checks the billing address against the information on the card (although not all banks support this).

If any of the fraud checks fail, we do not attempt to charge the customer. Otherwise, the card is charged and API credits are added to the user’s account.

For added redundancy, we also take advantage of Stripe’s webhooks. They send a web request to our Cicero servers, and we mark the payment confirmed. In the unlikely event that, say, the plug got pulled on our server right as a charge was being made, we would be able to add the credits upon receiving notification from Stripe that the payment went through successfully. (If our servers were to be down, Stripe would keep retrying the request until they come back up and an “OK” response is received.)

Testing

There is a way to test just about anything without anyone being charged real money. Developers can use a test account dashboard, API keys, and credit card numbers. Writing unit tests is easy because there are special card numbers to test each type of error, such as a declined card or mismatched ZIP code.

Pricing

The fee is 30 cents plus 2.9%. The customer’s country and type of card used do not matter. Volume discounts are offered for merchants that charge over $1 million per year.

Stripe Caveats

• Only a customer’s last card used is stored in Stripe. This makes it easy to allow the user to use their previous card instead of reentering all of the information. However, if he or she wanted to use the a card used for an order before the last, it would have to be reentered.
• Payments are disbursed with a seven-day float. This is not a major issue for us, but could be for a reseller taking a high-volume special order with a short lead time, for example. Waiting a week to get paid is certainly better than other providers that can sometimes freeze your account without warning.
• Stripe is not yet available outside the United States.

Some fun big picture statistics: between 0.6 and 0.7, there have been 301 files changed, 10,566 lines of code added, and 7,385 lines of code deleted.

The rainbow bridge to GeoTrellis Asgard

Here is a summary of some important changes:

• Powerful new support for extremely large rasters through tiled raster data classes
• New map/reduce-style definition of operations for data parallelism on large raster data
• Support for a range of new raster data types for efficient memory utilization and performance improvements
• Significant speed improvements through lazy evaluation
• Improved geoprocessing operation hierarchy and operation naming scheme
• Modernized benchmark structure with many new performance benchmarks, including tiled raster tests
• First class support for floating point raster data values, with floating point raster interface, support in core operations for floating point values, and implementation of floating point ARG formats
• The getting started documentation has been significantly expanded and improved (http://azavea.github.com/geotrellis/getting_started/GeoTrellis.html)
• Executable geotrellis tools can now be installed via conscript
• The geotrellis template has moved to the github repository geotrellis.g8, and can now be installed using giter8

Another exciting development since the 0.6 release: support for the ARG raster format was contributed to GDAL by David Zwarg. Using a recent build of GDAL, users can convert from other raster formats into ARG and also use GDAL’s full suite of raster conversion tools on existing ARG files. Future releases of GDAL will contain this ARG-support, as well as tools like QGIS which use GDAL internally.

Best,
GeoTrellis Team

Release Notes: http://notes.implicit.ly/post/30455035622/geotrellis-0-7-0
Github: http://github.com/azavea/geotrellis
Maven repository: https://oss.sonatype.org/content/repositories/releases/
Documentation: http://azavea.github.com/geotrellis/getting_started/GeoTrellis.html
API Scaladocs: http://azavea.github.com/geotrellis/0.7/api
Issue tracker: https://github.com/azavea/geotrellis/issues?milestone=3&state=open
Mailing list: https://groups.google.com/group/geotrellis-user
IRC: #geotrellis on freenode