Azavea Labs

Where software engineering meets GIS.

Bracing for (Potential) Catastrophic Success — Amazon’s Cloudfront CDN

Most of the web applications we build are either used internally by our clients or have a steady stream of public user activity.   With our recent Redistricting the Nation launch we wanted to experiment with some optimizations to make our site more resilient to traffic spikes as well as to improve the user experience.

Our strategy is broken down into a few components:

This post covers the Cloudfront CDN.

Previously, we had experimented with Amazon’s Web Services stack to host applications, but we hadn’t experimented with their Cloudfront CDN product.   Pricing for the CDN is quite similar to Amazon S3 and allows organizations to build scalable applications without the upfront cost of most CDNs.  We decided to use the CDN to host some large Javascript assets as well as our image components.

Cloudfront is quite easy to setup.   We simply created an Amazon S3 bucket called s3.azavea.com and pointed a CNAME record for s3.azavea.com to the full bucket domain — s3.azavea.com.s3.amazonaws.com.    Then, we enabled a Cloudfront distribution for the s3.azavea.com bucket using the free tool Cloudberry.   Finally, we setup a CNAME record for cdn.azavea.com to the Cloudfront distribution domain d17ib0dlm1q8qa.cloudfront.net and we were rolling.

Since the CDN is heavily cached, it was easiest to use s3.azavea.com links during development to reduce the amount of file versioning that was necessary.   Once we were settled on our assets, we switched to cdn.azavea.com links and started using the CDN.

The speed of the CDN is quite astounding.  Splitting assets across another domain name also improves the browser’s ability to request more files at once improving the user experience.  We were quite pleased with how easily we could offload assets to Cloudfront and realize gains with limited time investment.

A few notes to keep in mind when you are working with a CDN for the first time:

  • Since there is no way to flush assets out of Cloudfront’s edge nodes, be sure to use file name versioning.   This was a bit alien to us, but is easy to incorporate once you think it through.   For instance, we decided not to set a far-future expiration header on our PDF assets as they are often directly linked to and we wanted to be able to update them regularly.
  • Speaking of PDFs, it seems that while Cloudfront supports byte-range requests for assets, it doesn’t assert the “Accept-Ranges: bytes” HTTP header. This makes our large PDFs fully download before Adobe displays them within the browser.  Unfortunately there is no way to add this header at the moment.
  • Cloudberry is great to add HTTP headers to S3 assets.   We decided that most of our assets would have a six month cache lifespan by asserting the “Cache-Control: max-age=15552000″ header.

Nesting Comments Using Ext.XTemplate

When considering a nested comment implementation, we really only have to deal with two types of comment: root comments and child or nest comments. Root comments are easy to describe. They are not a child of any other comment. They’re what you get when someone has a brilliant new insight, hits the “New Comment” button and dazzles us all. Nest comments, on the other hand, seem like they should be more complicated. If you’re allowing replies to comments, then a nest comment could have its own nests. Wouldn’t that make a comment both a nest and a root? Not in this post. For the purposes of this example: roots don’t have parents; everything else is a nest. Simple.

Let’s address a few data prerequisites before diving into the actual how-to of template nesting. No matter how your comment data gets to your page, it should be in order. For example: say we have 3 comments :

  • Root A – Oct 1
    • Nest 1 – Oct 15
  • Root B – Oct 4

We seem to be following a chronological paradigm here and everything is in a logical order. In order to do the least processing in your javascript, your data should come to the page in this same order: Root A, Nest 1 then Root B.

Each comment should have some basic information available to the template. Root comments only need some kind of id along with the actual comment information (text, date, all that kind of thing). Nest comments need a bit more information. They should know the id of their parent (whether a root or another nest) and their nesting level or how deeply they’re nested from the top of the conversation. For our example above, a sample data set should look something like this:

{ [id: 100, text: "Root A", date: "Oct 1", nestLevel: 0, parent: 0],
  [id: 350, text: "Nest 1", date: "Oct 15", nestLevel: 1, parent: 100],
  [id: 288, text: "Root B", date: "Oct 4", nestLevel: 0, parent: 0] }

Notice that our root comments have nestLevel and parent values of 0 (zero). This makes sense: Roots can’t have parents by our definition above, and they will always have a nest level of 0. Take a look at the second entry, our nest comment. The parent value is 100 which is the same as the id value for our first root comment, Root A. In effect, we have an internal relationship key for our data.

Now that we have data in the right order and our nests know their parents and how deep they are, we can set up our Ext.XTemplate object to handle the visual component of our nesting.

var tplComments = new Ext.XTemplate(
 '<tpl for=".">',
    '<div id="comment-{id}" style="margin-left:{[values.nestLevel === 0 ? "5" : "15"]}px;
              background-color:{[values.nestLevel % 2 === 0 ? "#fff" : "#f9f9dd"]};
              border:1px #ccc solid;
              padding:5px;">',
        //comment text, date and other info here
    '</div>',
 '</tpl>',
);

Lets pick apart our template and see how it visually nests our comments for us. On the first line, we’re declaring our object, nothing new here. The next line uses a template looping mechanism (for=”.”) to create the internal tag structure for each data element in our comment set.  Now things get interesting.

The id of our eventual div tags will include the id of the comment being templated. This is done using some token replacement where you see {id}. So the html for our first comment will be in the div with the id “comment-100″.

In the style attribute of our comment div are two blocks of javascript surrounded by {[    ]} that the template will treat as javascript instead of just text. Inside those blocks, the “values” object refers to the particular comment object we’re dealing with inside our loop. The attributes on this object are the same as the property names in our data set: id, text, date, nestLevel and parent. When English-a-fied, the first block reads: If the nestLevel is 0 (meaning a root comment) then write 5 here, for any other value write 15. This will give our roots a left margin of 5px and our nests a left margin of 15. While this doesn’t quite get us to an infinitely-nested set of comments, it gets us close and provides a visual cue to the audience. We’ll get closer to a truly nested solution in the javascript utilizing this template later.

The second block of javascript in the style attribute switches the background from white to cream on alternating comments. Its English sentence might read: If the nestLevel is an even number, use a white background, otherwise use cream.

Now for some javascript to bring it all together and make our nesting work properly. After your comment data is available in the page, you can use the XTemplate‘s append function and some basic Ext DOM querying to stack your comments indefinitely.

if(comment.get("nestLevel") === 0)
{
   tplComments.append(commentLocation, comment.data);
}
else
{
    tplComments.append(Ext.get("comment-" + comment.get("parent")), comment.data);
}

This block of javascript splits our comments set into the two definitions from the beginning: roots and nests. Roots just get appended to the end of the commentLocation element on your page. Nothing fancy there. Nests, however, take a bit more thought. A nest knows the id of it’s parent comment. In our template object, each comment is contained in a div with the same id as the comment. In order to nest a comment under its proper parent, use Ext’s built-in DOM searching capabilities to locate the parent, then append the nest.

Each nest will wind up nested 15px in from the left edge of its parent; alternating comments have subtle color changes and each wrapping div is associated to a comment by id for future tinkering. And we’re done!