Azavea Labs

Where software engineering meets GIS.

Getting an ArcGIS Server Map Cache in S3

When deciding how to best handle the air photos in the new Philadelphia Water Department Stormwater Map Viewer, we kicked around a few ideas. We decided to put the cache in Amazon’s Simple Storage Service to offload some of the local disk requirements and leverage their fast data storage and delivery infrastructure. In moving the process, we learned a few things:

Tune Your Cache

Make sure you spend time planning the cache. Not only will the cache look better in the final application, but it will also load to S3 faster and cost less in the long run.

  • Set the extents in the MXD or MSD before publishing to a map service. The overhead of transferring the 254 byte empty tiles caused a lot of unnecessary burden on the upload process as well as the fact that you are paying for them to be stored in the cloud. If it doesn’t need to be there, don’t build it.
  • Choose the correct image format for the cache. If you are caching a base map and do not need to support transparency, make it a JPEG. If it needs to support background transparency, use PNG. ESRI’s suggestions for planning a map cache can be found here.

Get a Good Tool to Transfer the Files

I started using the free version of Cloudberry Labs S3 explorer. But I had to move over 90 Gbs worth of data to my S3 bucket. The CloudBerry S3 Explorer – Pro supported multithreading which allowed for up to 5 threads to either enumerate through the folders, copy the files or apply the ACL. It is a low cost application that more than pays for itself when moving a lot of files up to a bucket.

When transferring the files up, I was working in blocks of directories, not the whole scale level. It was quicker for me to work in 20 to 30 subdirectories than grabbing a whole scale level. It did require a little bit more management on my end, but more steady progress was made.

Accessing the Tiles

ArcGIS Server does not support cloud hosted caches at the 9.3.1 release. The ESRI Javascript API and Flex API can be extended to use caches hosted in the cloud (Flex example from Mansour Raad), so you’ll have to roll your own. For the Philly Storm Water project, we were using the Open Layers and someone has rolled one for us. There is a patch that can be used to access the cache without communicating through ArcGIS Server straight from the client-side library. The one thing to note is that the Tile Origin is pretty touchy, we had to make some adjustments to the origin values to make sure everything lined up correctly.

Summary

Now that the site is up there and we are starting to get some traffic hitting it, putting the tiles in S3 was the right decision. There is no reason for ArcGIS Server to waste any cycles moving tiles around, let it do the heavy lifting with the vector layers and queries. Hopefully the rumors are true, and the ArcGIS Server 10 release will be more aligned with cloud computing. Until then, there are still plenty of ways to take advantage of the benefits.

Wireframing w/ Balsamiq – 5 Tips

I have been spending a lot of time pulling together new wireframes for HunchLab using Balsamiq over the past couple of weeks. In doing so I have learned a few tricks along the way.

5. Get Additional Controls

Looking for particular interfaces or additional controls, check out Mockups to Go. I was able to find a ribbon toolbar, iPhone interface and a series of pointers that I could bring right into the wireframes.

4. Change to System Fonts

While I understand the goal of the application to make things look sketchy so the reviewers can focus on functionality not on the particulars of the specifics of the UI. But I definitely believe there is a reason why Comic Sans has been identified as the worst font ever (link to Google search for some supporting details).

There are a couple of ways to do this, the first is the most straight-forward. Select ‘View’ from the toolbar at the top and then choose ‘Use System Fonts’.
SystemFont

Another (slightly more complicated method) is to edit the Balsamiq Mockups config file. Here are the detailed instructions from Balsamiq’s website. Going this path gives a little more flexibility to the font used in the design process.

3. Clone Current Mockup & Duplicate Feature

I found these two functions to be great time savers. With the duplicate feature tool, no matter how many features are selected, it is a single-click copy and paste function.

The cloning of mockups lets you easily duplicate the interface and either create the next step in a series of wireframes or rearrange the controls to look at a different configuration.

2. Internal Links to Replicate Interaction

I have been using the links on controls to allow users to interact with the wireframes. Before any code is written users can interact with the UI and give feedback to workflows and interacting with the pages.

By default when working in the ‘Full Screen Presentation’ mode, there is a large exaggerated cursor that was confusing to some of the people I worked with.
BidgPointer

In the bottom right-hand corner of the presentation mode, there is tool to set the cursor to be more like a normal user experience when interacting in a browser or desktop environment.
NormalMode SmallPointer

1. Link to APIs for Images

Creating graphics to represent graphs or maps can be a time consuming task. There are a number of API’s out there that can be used to get a good representation of what will be presented in the interface. A few of my personal favorites are:

Google Static Maps API

Sparklines from Joe @ BitWorking


Google Charts API