Tag Archives: OpenStreetMap

State of the Map: A Weekend with OpenStreetMap Folks

I spent this past weekend at the annual OpenStreetMap conference, State of the Map, held in Denver this year.  [While I think that pairing it with the FOSS4G conference was a terrific idea, I was not able to stay this week for the latter event, but I'm excited to hear from my Azavea colleagues, David, Justin and Matt about that event].

I’m going to walk through some highlights of the event from my own perspective.  I’ve written a lot about OpenStreetMap in this blog, so it’s no secret that I’m a big fan of this project that sets out to create a shared and open map of the planet.  A lot has happened in the past year, but here are some of the things I saw as important or just plain cool:

  • MapQuest and OSMMQ has been a big contributor and user of OSM. They are working on new quality assessment software tools and hope to release them to the community soon.  This will help to highlight the major swaths of the US, in particular, where the data needed for routing and geocoding needs to be fixed.
  • Bing and OSMSteve Coast did an overview of Bing contributions to OSM since he joined Microsoft:
    • Bing aerial imagery – Bing has agreed to share its high quality aerial imagery with the OSM community. This is a big deal, as it will enable more mapping to be done without physically traveling to a site and Bing’s imagery is much higher quality than was previously available to the community
    • DetectRoad API – deriving street vector data from Bing aerial imagery
    • Windows Phone 7 OSM Editor – the MS Bing folks are working on this
    • Frontdoor Addressing – a free and open app to move points that were geocoded to the rooftops of residences to the front door.  150 million records.  Microsoft has agreed to share all results with OpenStreetMap under the OSM Open Data license.
  • ArcGIS and OSM – Esri continues to invest in tools for editing and using OSM data.  The ArcGIS OSM Editor was upgraded in the spring to version 1.1 and Esri is working on version 2, which was just released in beta.  Some new directions Esri is taking include:
    • Overall objective is to add support for OSM throughout Esri stack.
    • Initial objective is to add support for publishing OSM through ArcGIS Server, including support of editing via the Javascript API.
  • OSM in Japan - two presentations by Daniel Kastl on the history of OSM in Japan; challenges specific to the Japanese language, addressing systems and urban structure; and use of OSM for earthquake/tsunami response and recovery.  Nostalgia for me as I recognized all of the unique and wonderful things about living in Japan.
  • Mapnik – David Zwarg told me about a lightning talk about Mapnik2, which he says “has TONS of awesome features”
  • Gameification – Peter Batty pointed to the need for a more game-ified OSM editing experience that might bring in a larger audience of editors.  I think there is a lot to be said for this.  There was an ad hoc session on Sunday focused on this question and the potential for a game-like user experience to potentially expand the OpenStreetMap community but also to potentially negatively affect data quality.
  • Walking PapersMichel Migurski (Stamen Design) summarized recent work on his Walking Papers project to a standing-room-only crowd, including interesting applications of Astrometry algorithms to geolocating map images taken from phones.  Other highlights include support for multiple languages, atlas (cutting up a scene into 4-up, 16-up, etc.) and tweaks that support other audiences (crisis mappers, educators and museums).
  • Migurski followed up this performance on Sunday with a rousing plea to make creating and using OSM data a lot easier.  I have to admit, if you are new to the OSM community figuring out how to either use or contribute is daunting.
  • Cool Tools I Didn’t Know About
    • TopOSM - OSM maps overlaid on topographic elevation maps
    • Cartagr.am
    • Acetate - attractive stylesheets for data visualization
    • TileStache - renders vector and raster tiles from contemporary map sources – think of it as a next generation TileCache
    • ImpOSM – next generation importer for OSM data
    • Cascadenik – cascading stylesheets for working with Mapnik
This was my first time at State of the Map, and I was impressed by the cohesive and enthusiastic community that is gathering around this important resource.

Create Neighborhood Maps with OSM and MapOSMatic

As regular readers of my articles may have noted, I’m a big fan of OpenStreetMap. I recently discovered a pretty cool service, MapOSMatic, that enables you to generate customized maps of neighborhoods and cities using the OSM database.  Each map generates two files:

  • the map with a label and border, nicely organized with a lettered and numbered grid
  • an index with the street names

You can generate maps in PNG, PDF or SVG formats, and the PDF versions are generated with vector graphics and text objects, so they can be printed at any resolution.   Further, since the data is available under the open OSM data license, you can re-use and distribute as you see fit.

Based on an idea articulated by Gilles Lamiral, an OSM contributor in France, the application was developed by a small team during a one-week “hackfest” in August 2009.  The initial version was limited to French and English and was based on a static database. A second hackfest in December 2009 added daily OSM data updates, global coverage and redesigned UI.  To the original French and English versions, translations have been added  for a growing list of languages including: Spanish, Catalan, German, Italian, Russian, Arabic, Danish, Dutch, Portuguese, Croatian and Polish.  Features planned for the near term include adding legends, paper size selection, configurable styling, options for displaying different amenity layers and support for multi-page maps.

In a couple of minutes I was able generate a map and street index of my neighborhood in Philadelphia.

Spring Garden Neighborhood Map

Spring Garden Street Name Index

And because OSM is global, it works the same way everywhere in the world.  Check out the neighborhood where my father grew up in Loughborough, England – the OSM map is sufficiently awesome in Europe that the building footprints are there for the entire downtown area.

MapOSMatic rendering process

map of loughborough neighborhood

ArcSquirrel, ZigGIS, OSM and Alternative ArcGIS Editors

I recently listened to a DirectionsMag podcast regarding a new product by a Welsh company, exeGesIS, called ArcSquirrel.  Apart from an awesome company name and humorous product name [since we changed our company name, I think a lot about names.  How cool is "exegesis" for a GIS company?], it’s a plugin for ArcGIS desktop that enables direct editing and data management of SQL Server 2008 spatial data layers.

While I think it is really great that Microsoft implemented a spatial data type as part of their flagship SQL Server database product, the initial release was a somewhat crippled product.  You could query spatial data stored in SQL Server using a wonderful series of extensions to the SQL language, but MS did not package any tools to actually load data.  Further, the ADO.Net and LINQ database access frameworks didn’t really support the new spatial data types very well either.  Some open source spatial data tools were posted on CodePlex and that was useful, but there weren’t really great tools for editing the data directly.

ArcSquirrel logoEnter, ArcSquirrel.  This extension for the Esri ArcGIS desktop tools will enable you to edit the SQL Server spatial data columns using your favorite desktop GIS tools.  ArcSquirrel adds a new toolbar to the ArcMap application as well as tools for loading GIS data to SQL Server, support for multi-user editing, metadata integration with ArcCatalog and support for joins and spatial functions.  At $240/seat, it’s pretty affordable.

OpenStreetMap logoThis is not the first such specialized GIS data editor that extends the ArcGIS desktop product.  Obtuse Software has created ZigGis, an extension to ArcMap for editing PostGIS data.  More recently, Esri has developed and released an open source extension to ArcMap that supports editing the OpenStreetMap database.  I’m particularly impressed that Esri has not only created an extension for OSM, but has elected to release it under an open source license.  The beta version was released in July and the 1.0 release was out last week.  Software like this is going to enable the 100,000′s of ArcGIS desktop users to contribute to the global OpenStreetMap database and thereby make it more useful for everyone.  Based on the Esri demo at the US State of the Map event, Randal Hale has written up a nice review of the extension.  Kudos to Marten Hogeweg and his colleagues at Esri.

Resources

UPDATE:

10/13/2010: ArcSquirrel has released an API that enables programmatic control over the data management and editing process.

OpenStreetMap on ArcGIS.com

I’m confident that my recent post asking that ESRI add some support for OpenStreetMap had nothing to do with it, but I’m still happy to report that ESRI rolled out its new ArcGIS.com web site and one of the new basemaps is OSM.  Pretty cool.

OSM in ArcGIS.com

I could quibble. For example, there are no tiles for the highest zoom levels, and that just seems like a sad omission. Nonetheless, there’s global coverage and it’s a major vote of support for OSM.  BTW, I also think the design of the new ArcGIS.com web site represents a vast improvement overt the ArcGIS Online system.  It’s got a straightforward user interface and muted style that’s easy on the eye.
OSM BaseMap Limits

When will ESRI Support OpenStreetMap?

OpenStreetMap: the free wiki world map

ESRI has a perception problem. It is similar to the one that Microsoft and other commercial software firms have developed vis-a-vis open source software projects. ESRI is perceived by many in the open source world as being opposed to open source software. While I think ESRI has fed this perception to some extent, the open source community has also cultivated a “David vs. Goliath” approach that encourages an adversarial relationship with the larger software companies that I don’t think it terribly helpful either.

But as Paul Ramsey recently pointed out in his address at the FOSS4G 2009 conference in Sydney, most of the mainstream commercial software firms now support open source software platforms, melding commercial and open source business models. Commercial software firms contribute to open source projects for a myriad of reasons including:

  • As a critical component of their platform
  • Low cost R&D
  • Build a broad constituency for a standard
  • Increase the number of developers focused on a particular platform
  • Retire a platform while still enabling customers to receive support

ESRI has pursued at least three of these approaches in its work with open source projects, and while projecting a competitive attitude about some open source projects (and justifiably so), they also deserve some credit for supporting open source projects in a variety of areas including:

ESRI also gave us an open specification on the now venerable shapefile and looks set to do the same (after some years of delay) for the File Geodatabase. And ESRI has contributed resources to development as well as platform support for many of the OGC standards. I would also argue that many of the most successful open source projects could not exist without substantial support from commercial software companies.  PostGIS would not have got off the ground without early and ongoing support from Refractions. Apache and many Java projects gained from substantial investments by IBM.  In other words, I don’t think we gain by having open source software seen as being in opposition to commercial software.  It’s simply part of a complex software development ecosystem.

But I opened by saying that ESRI has a perception problem. In addition to continuing to support select open source projects when it makes strategic sense, I’d like to make a pitch for ESRI supporting the OpenStreetMap project. OpenStreetMap is really multiple projects. It does include open source (GPL) software that would probably be of limited interest to ESRI, but it’s primary output is an open map of the planet. Just as ESRI has helped to encourage the broad use of free government data sets like the Census TIGER and USGS data sets, it should help promote the OpenStreetMap effort.

Why support it?

  • More data means more use of GIS: In the same way that free distribution of TIGER, USGS, Dept of Defense and other data sets catalyzed GIS development in the 1980′s and 1990′s, more data in more parts of the world will encourage more sophisticated uses of GIS, where ESRI really shines.
  • PR value: Support for the OpenStreetMap project will give ESRI some of the street cred that companies like AutoDesk have gained by contributing software projects to the open source community.
  • Free data for ArcGIS Online: The OpenStreetMap data set offers a free, global data set with distinctive cartography that covers some parts of the world even better than the commercial providers. Providing an OSM map service to ArcGIS Online will only make it more attractive for ESRI’s customers.

How should it be supported?

  • ArcGIS Desktop: Enable ArcMap to both display data from OSM and be an editor. The ESRI desktop GIS community are some of the most skilled and knowledgeable people engaged with map production. By enabling them to use the software tools with which they are already familiar (rather than the capable, but clunkier tools like Mercartor and JOSM), they will be able to make valuable contributions to OSM that will make the map better in every part of the world.
  • Toolboxes: Create tools that convert OSM data formats to feature classes.
  • ArcGIS Online: Provide an OSM-based map tile set.

The OSM project is not public domain, so there are important license considerations, but even with the currently proposed revisions, it’s a pretty open license with only attribution and share-alike provisions limiting its use.  But as the Haiti earthquake response demonstrated, OSM is an important and evolving piece of infrastructure that will only be better with ESRI’s support.

——–

UPDATE: 3/22/2010

I should have also cited a recent ESRI blog on some techniques for incorporating OpenStreetMap into ArcGIS Server that are possible now.  These include:

  • Use the WMS extension
  • Use the Data Interoperability Extension (a nice package from Safe Software that is an extension for both the ArcGIS desktop and ArcGIS Server) which now support the OSM XML file directly.  Export the data from OSM to shapefiles or a geodatabase and serve it up.
  • You can also use an extension Azavea (that’s us) created for the ArcGIS Flex API that supports direct integration of the OSM tile structure for Flex apps.

OSM Maps Port au Prince in Haiti Response

The OpenStreetMap community has really stepped up to the plate and delivered some amazing vector data using a mix of Yahoo! imagery, old CIA maps and new GeoEye imagery.  Some people were digitizing, while others were making sure updated shapefiles were generated every 5 minutes.  Hundreds of sessions were generated in a few days.  The images below, swiped from the Mikel’s post at the OpenGeoData blog, demonstrate the dramatic progress:

OSM at the time of the quake

OSM at the time of the quake

OSM after a couple of days

OSM after a couple of days

OSM, after quake, zoomed in

OSM, after quake, zoomed in

Sean Wohltman made some interesting observations, however, that Google’s similar MapMaker effort was working at cross-purposes to the OSM efforts, leaving users of the maps needing to make a decision about which version they should use.  A common effort would benefit more people, but the legal terms and conditions prevent a straightforward resolution.  Geospatial data developers and users have made great contributions to the Haiti relief efforts, but while the geo-geeks are playing a leadership role in one respect, they are also exposing some tough contradictions in our legal infrastructure.

Update 1/18/2010:

Some additional OSM Resources related to the Haiti quake:

OSM Haiti with Mapnik rendering and earthquake related locations

OSM Haiti with Mapnik rendering and earthquake related locations

OpenStreetMap License is Changing

OpenStreetMap: the free wiki world map

Whether for commercial software or open source projects, the crafting of a license is one of the most important decisions a company or team can make.  The license determines who can use the software, how it can be used as well as how it can be shared.  Open data projects, while different from open source software, face the same types of questions.

OpenStreetMap is probably the single largest and most significant open data project in the geospatial realm.  The project was started because “most maps you think of as free actually have legal or technical restrictions on their use, holding back people from using them in creative, productive, or unexpected ways.” Up until now, OSM has been using a license called Creative Commons Attribution Share Alike (CC-BY-SA).  However, OpenStreetMap is more like a database than it is like a text document or photograph and database projects have run into some specific problems with the CC family of licenses.  The OpenStreetMap project is proposing a move to the Open Database License (ODbL).  Like many collaborative projects, the move is being made by submitting the change and the justification for it to the community for review, comment and vote.

Why make this move?  What’s wrong with the CCBYSA license? A lot of people use the CC licenses to publish their articles, photos, paintings and other creative work.  But the various forms of the Creative Commons licenses are designed to work within the legal infrastructure the surrounds the concept of copyright.  Structured databases are collections of facts.  When factual data (like streets drawn on a map) are arranged the way you’d expect it to be, it’s not necessarily protected by copyright law, particularly under U.S. copyright law, which only protects works that arise from creativity.  If copyright doesn’t apply to factual data and the CC licenses are based on copyright law, we have a problem.  The is the core of the issue.  Even the Creative Commons folks have said that the CCBYSA license should not be applied to databases.

The new proposal, ODbL, resolves the issues by applying copyright where it applies and applying contract law where it does not.  It attempts to take the best of both worlds and create a happy medium that applies to database projects like OSM. As perhaps the largest open database in the world, OSM was one of the touchstone cases that the Open Data Commons and Open Knowledge Foundation used to build the license.

But it’s also interesting to note what it won’t cover.  The ODbL will only apply to distribution of the OSM database.  Contributions to OSM (like GPX tracks and other database edits) are covered by a Contributor Agreement which will refer to the ODbL as the means of distributing their contributions.  It won’t cover image tiles generated based on the OSM database.  It won’t cover the OSM wiki, which, since it is text and therefore considered a creative work, will remain covered by CCBYSA.  And it won’t cover the software source code used to run the entire OSM system – that will be usually, but not always, be covered by the GPL.

There remains some controversy within the OSM community. Many members, including one of the founders advocating for the change, feel that a completely free, Public Domain license (no limits on usage) would be preferable.  The ODbL will retain the “share-alike” concept of the current CCBYSA license (requiring both attribution and that changes be submitted back to the community and distribution carry the same terms). They feel that the spirit of reciprocity codified in this approach is stronger. The new OSM license will include both the concept of attribution and share-alike because many members of the community feel that this limitation benefits the project.  Nonetheless, others feel strongly that a truly public domain situation would be better in the long run, encouraging broad usage without consideration for consequences.  In the best democratic tradition, however, both sides express their positions in Vote Yes and Vote No pages.  Check them out.  And if you are an active member of the OSM Foundation, make sure you cast your vote.

You may be tempted to file this under “boring”, but the nuances of licenses are an important part of the creative economy in which we operate.  They set the terms under which we interact with each others work.