Fast forward a few years and bring OpenLayers into the mix. We rewrote the asset editing portion of the software to allow data managers to move asset coordinates using an OpenLayers map. These coordinates were saved with still considerably more precision than we needed, but remember, we’re storing whatever precision we get. So far so good.

Now back to the present and we’re working on a comparison tool for our data managers. Suddenly values in the database are not matching the values coming out of our OpenLayers map. Almost, but not quite. In fact, only the last degrees of precision are different. After a bit of digging, we discovered that OpenLayers was returning numbers with between 1 and 3 fewer decimal places than our stored coordinates. Remember that we’re talking about distance differences smaller than a crack in the sidewalk here, but programing languages don’t know anything about “close enough”. Either two numbers are the same or they aren’t and -39.6827663878 is not the same as -39.682766387 no matter how small the physical difference is. So we started digging for the reason.

OpenLayers has a value tucked away in its utility files that sets the default precision of a floating point number to 14 characters. This limit was added when a user noticed that the edges of certain coordinate systems were not behaving correctly due to some floating-point math precision errors. While the OpenLayers community recognizes that most systems allow floats to have 16 digits,  “14 significant digits are sufficient to represent sub-millimeter accuracy in any coordinate system that anyone is likely to use with OpenLayers“. So OpenLayers’ answer to the accuracy question is to save everything that will fit in a standard float, with a few decimal places pared off just in case.

So the next question is: “So what?” The difference between 14 and 16 decimal places in a meter-based coordinate system is microscopic, and in a degrees-based one it’s not much bigger. So far as storing a saved coordinate in Sajara, we didn’t really care if we had 16 digits or 14 digits; the result wouldn’t look any different to our audience. However, since our initial coordinates had 16 digits and OpenLayers only preserved 14 of them, any programmatic comparison fails! No one likes to deal with false positives, but a 100% false positive rate was unacceptable.

We had a few choices here. First we could reset the default precision value in OpenLayers to zero, which would tell the library to never truncate anything. That’s a fairly simple change but we weren’t sure it wouldn’t have unforeseen data effects. Also, there’s a somewhat vague warning about problems with the Web Mercator projection when this value is zero, which is one of the projections Sajara can use. So that option was out.

Second, we could have told SQLServer to alter the precision of coordinate values to 14, which is a fairly major change. This option was ruled out because of a difference in the definition of “precision” between SQLServer and OpenLayers. I mentioned earlier that SQLServer will store a maximum of 16 numeric digits plus a decimal and a negative sign, so a total of 18 characters. OpenLayers, however, considers the default precision of 14 to mean 14 characters instead of 14 numeric digits. So if a number has a decimal and a negative sign, we’re down to 12 numeric digits.  This little difference reintroduces the possibility of false positives, so it isn’t really a change for the better.

The solution we finally decided to use was to change the OpenLayers default precision value to 18. Why 18? That’s the maximum amount of characters that SQLServer will store for a float, so OpenLayers will always be able to deal with any stored coordinates without having to truncate. Now, if we compare our stored coordinates with OpenLayer coordinates, we only get a change notice when an asset has actually been moved. Which is exactly what we wanted.

Here are some technical details for those interested:

The full variable name is OpenLayers.Util.DEFAULT_PRECISION and can be found in the Util.js file. There are a few good comments preceding the variable in the code, but more background can be found in the OpenLayers ticket #1951. SQLServer information can be found in mdsn. Note that if you wind up changing the OpenLayers precision value, you should do it as soon as possible after loading the library, so you don’t have the possibility of code using different precision values.

Tune Your Cache

Make sure you spend time planning the cache. Not only will the cache look better in the final application, but it will also load to S3 faster and cost less in the long run.

• Set the extents in the MXD or MSD before publishing to a map service. The overhead of transferring the 254 byte empty tiles caused a lot of unnecessary burden on the upload process as well as the fact that you are paying for them to be stored in the cloud. If it doesn’t need to be there, don’t build it.
• Choose the correct image format for the cache. If you are caching a base map and do not need to support transparency, make it a JPEG. If it needs to support background transparency, use PNG. ESRI’s suggestions for planning a map cache can be found here.

Get a Good Tool to Transfer the Files

I started using the free version of Cloudberry Labs S3 explorer. But I had to move over 90 Gbs worth of data to my S3 bucket. The CloudBerry S3 Explorer – Pro supported multithreading which allowed for up to 5 threads to either enumerate through the folders, copy the files or apply the ACL. It is a low cost application that more than pays for itself when moving a lot of files up to a bucket.

When transferring the files up, I was working in blocks of directories, not the whole scale level. It was quicker for me to work in 20 to 30 subdirectories than grabbing a whole scale level. It did require a little bit more management on my end, but more steady progress was made.

Accessing the Tiles

ArcGIS Server does not support cloud hosted caches at the 9.3.1 release. The ESRI Javascript API and Flex API can be extended to use caches hosted in the cloud (Flex example from Mansour Raad), so you’ll have to roll your own. For the Philly Storm Water project, we were using the Open Layers and someone has rolled one for us. There is a patch that can be used to access the cache without communicating through ArcGIS Server straight from the client-side library. The one thing to note is that the Tile Origin is pretty touchy, we had to make some adjustments to the origin values to make sure everything lined up correctly.

Summary

Now that the site is up there and we are starting to get some traffic hitting it, putting the tiles in S3 was the right decision. There is no reason for ArcGIS Server to waste any cycles moving tiles around, let it do the heavy lifting with the vector layers and queries. Hopefully the rumors are true, and the ArcGIS Server 10 release will be more aligned with cloud computing. Until then, there are still plenty of ways to take advantage of the benefits.

The Layer object can understand what projection you’re working in, if you happen to give it one. If the displayed map area happens to wrap around the IDL (and your layers are set up correctly), the Layer object gracefully acknowledges the IDL and produces correct coordinates.

In contrast, the Bounds object is pure math. It’s meant to be as projection-agnostic as possible without actually messing up coordinates in objects that do know about projections and their quirks. This can be a very good thing, since doing math without worrying about projections is a lot faster and works just fine in most cases.

The problem I ran into comes from one little Layer object configuration option that is uncharacteristically poorly documented: wrapDateLine. The issue it tells you to reference (#487) talks about using modular arithmatic to fix how layers handle the IDL… I’m not a math person, but handling the IDL is exactly what I want it to do! Too bad they couldn’t have mentioned this in the documentation.    What the Map winds up doing if you don’t have this config option set is using the Bounds object’s straight-up math to get the center of the extent. Unfortunately, using straight math when the IDL is in play puts the center of the map at the longitude exactly opposite of where logic dictates it should go. If the wrapDateLine configuration item is set, then the Layer tells the Map “Whoa, there! The Bounds object doesn’t know about the IDL, but I do. Use my math first.” So the Map uses the Layer’s math to turn the extent into coordinates that the Bounds object’s math will work correctly on. Viola! The North Pacific!

So the simple fix is to have wrapDateLine: true in your layer configuration object.

To see this bit of oddness ‘in action’, go to the OpenLayers Google Map example and pan the map so that the IDL is in view. Using FireBug, enter the commands map.getCenter() and map.getExtent().getCenterLonLat(). The resulting coordinate longitudes will be on opposite sides of the planet.

Understand JavaScript

OpenLayers is a JavaScript library.  If you do not have a sound understanding of JavaScript, do not pass Go and do not collect $200.  Instead, pick up Doug Crockford’s JavaScript: The Good Parts or Stoyan Stefanov’s Object-Oriented JavaScript.  Come back here when you’re done.  We’ll wait.

Check Out the Source

The OpenLayers source code is hosted in Subversion.  Check out the source to your development directory from:

http://svn.openlayers.org/trunk/openlayers/

Get Familiar with the Source

The most important thing you can do is invest some time familiarizing yourself with the JavaScript source code including the code structure, object inheritance, and components similar to the one you want to write.

Code Structure

OpenLayers is written to be object-oriented.  As such, the code is (mostly) structured such that the base objects are located in the root directory and any children of that object are in a sub-folder of the base object’s name. You can browse the source here.

Object Inheritance

OpenLayers object inheritance could be an article in and of itself, so we’ll just stick to the highlights.

All inheritance is done via the OpenLayers.Class function and supports multiple inheritance by augmenting any number of existing object definitions with your own object literal definition.  In real life, it looks like this:

OpenLayers.MyNewObj = OpenLayers.Class(OpenLayers.MyBaseObj, {
    prop1: 1,
    prop2: 2,
    doThis: function() {
        /* doing this */
    }
});

Check out the source code of OpenLayers.Class to really understand how OpenLayers implements object inheritance.

Dig Into a Component Similar to the One You Want to Write

Odds are that you are not setting out to write a completely unique component, but rather wish to build a specific implemention of something that already exists.  Most of my plugins have been tile-based layer plugins that get content from HTTP requests.  By looking through the examples, we know that OpenLayers already supports this type of functionality in OpenLayers.Layer.WMS and others.

We can discover what functionality OpenLayers already supports by digging into the WMS source code. Start by looking at the object definition and find its base object.  For example, the WMS layer extends OpenLayers.Layer.Grid.  The source for the base object will tell you which properties and functions you will have access to by default and if there are any functions you are required to override.  But don’t stop there!  Keep investigating base objects of the base objects until you get all the way up the chain.  For WMS, you will find OpenLayers.Layer.Grid, OpenLayers.Layer.HTTPRequest, and OpenLayers.Layer.  Understanding these objects will be invaluable when debugging.

Once you have an understanding of the base objects, and assuming they have the functionality you need, take another look at the WMS layer object.  Look at each property and try to understand why the author added it.  Also look at the functions.  Is it a new function?  Why was it added?  Is it an override? How is it different than the base functionality and why?  Answering these questions will enable you to pinpoint the functionality you will need to implement for your plugin.

Code!

You now know the way of the OpenLayers plugin, grasshopper.  Go and code.  But don’t forget the code conventions!

I won’t get into what sourcemap is (that’s already been done), but I thought it would be cool to mention some of the technical challenges that the project was/is facing, and what we’re doing about it.

First, the project started out with maps from a free, proprietary mapping API. This was a great stepping off point, but as the maps and networks became more complex, that API started to break.  I advocated that the project migrate to an Open Source JavaScript mapping library, by the name of OpenLayers.  This helped us overcome that barrier to complex maps.

Next, we wanted to change the cartography from simple lines between points to something more ’stylish’.  The problem that we experienced was that it was difficult to discriminate the paths of colinear parts of an object. To solve this, we added a slight arc to the lines.  This technique makes it easier to discriminate the connections between colinear parts and an object.

Time to bust out the trigonometry again!  I doodled the following sketches on a mousepad weekly planner.  I wanted an offset of the line in the vertical direction for most horizontal lines.  This calculation got me started, but then I realized that it would be weird if the arcs suddenly swapped sides if you passed the Y axis (e.g. from 359 degrees to 1 degree).

So I added another step, which attenuated the arcing, starting at 45 degrees from either side of the Y axis, and gradually decreasing to no deflection in the Y direction.

The result?  A cool looking network of lines that are clearly distinguishable! You can see it in action here.

We’ve got even more improvements on the way for the map, and I think it will make it a really compelling, and intuitive visualization of global supply chains.  Watch this space for the next installment.

]]> http://www.azavea.com/blogs/labs/2009/05/azavea_rd_sourcemaporg/feed/ 1 http://www.azavea.com/blogs/labs/2009/04/openlayers-with-arcgis-or-arcims/ http://www.azavea.com/blogs/labs/2009/04/openlayers-with-arcgis-or-arcims/#comments Wed, 01 Apr 2009 13:16:50 +0000 David Zwarg http://www.azavea.com/blog_labs/?p=6 OpenLayers, an open source web mapping library for browsers, is 99% web mapping goodness.  We took a look at the library, and found that we could almost use it in our projects to provide a robust mapping interface.  The extra 1% that we really needed was support for ArcIMS and ArcGIS Server.

So we fixed it.

Jeff added support for ArcGIS Server, and I added support for ArcIMS (in addition to a subset of ArcXML).  These are tickets in the OpenLayers tracking system, under the monickers #1749 and #213, respectively.  ArcGIS Server has made it into the trunk, and we are anxiously awaiting the incorporation of ArcIMS/ArcXML support.  It looks like they are both on track to make it into OpenLayers v2.8, so we are excited.

We’ve been running production applications with the patches we’ve provided to the OpenLayers community for months now, so it’ll be wonderful to make it ‘official’.