Wednesday, July 29, 2009

Hello World (Map)

I'm resurrecting my blogging effort outside of my former company--in the hopes of sharing useful tips I discover in my new software pursuits.

One particular area of distraction for me has been in GIS and mapping. I'm using the excellent mapnik toolkit to produce mapping visualizations. Mapnik sits on a fairly heavy stack of dependencies including:
  • GDAL/OGR - libraries and utilities for dealing with a multitude of raster and shape (vector) files. These have very good python bindings
  • PostgreSQL/PostGIS - nice, but not required
  • PROJ4 - Cartographic Projections library
  • BOOST - The forever-to-compile C++ libraries that mapnik uses for wrapping C++ codes.
Getting all of this running on OSX is a huge pain in the neck--especially if I want to use my preferred Python Distribution. Much of this pain is alleviated with Framework installs of many of the dependencies from the excellent kyngchaos site (many thanks!). I'm still not convinced I have all my libraries linked properly (who needs ICU_LIBS anyway?!).

My original mapping goal was fairly simple: Develop a composite map of roads, terrain and some custom features for a nice large-format printout for my son's room. Technically, this meant a few things:
  • converting srtm elevation data [1] to nice contour shape files (gdal_contour).
  • generating a hillshade raster image from srtm data
  • generating a color relief from srtm data
  • properly composing the custom features (considering Cartographica for this, but it's a bit expensive for a hobby project and the product seems at bit...nascent. Suggestions?).
  • beautifully rendering everything using a nice scanline approach.
It's important to note that the data and the tools I've used (with the exception of Cartographica) are all freely available (some tools do have copy-left provisions, and the data are not to be 'sold' in many cases). There is a massive amount of data for my region of interest, and a half day of effort gave me something like this:

Well...there was still much work to be done. The above image used the downloadable contour sets from the two Texas counties that this image spans, and the renderer I was using (cartographica) doesn't do very good text-layout "collision detection," nor does it have very fine-grained control over things like max angles and spacing for labels. So, putting together a simple mapnik script, using my own generated contours, applying hill shading, and tweaking the color relief to make it more wife-friendly yields something like this:

I'm getting close. I need to re-append the roads data and add all my custom features (labels, points of interest, etc.) I'm pleased with the progress so far.

This exercise has led me to think about potentially different approaches. While mapnik is extremely nice, they do use boost which, unfortunately, leads to a more lengthy and difficult build/install. (I'll try to document the OS X steps that I used when I set it up on a new machine--I had to deviate from the "official" installation instructions in a couple of places.) Why didn't they use SWIG? Is the mapnik C++ layer heavily templated? Also, I'd like to see the rendering layer use my favorite stack of 2D rendering tools -- Kiva and Enable. This would allow an intermediate layer which would provide more general vector output routines. I'll try to get a handle on the level of effort for this in the coming weeks, as time allows.

[1] The srtm data set is one of the cooler products I've come across--a shuttle mission to map the world and provide free access to global elevation data at 10m or 30 resolution (gladly sponsored by this American taxpayer!).


  1. Great post Travis.

    On the Mapnik end, I think what we need is both a Boost Framework/ICU (to make available easy installers for all Mapnik Dependencies) and a Mapnik Framework.

    I've been working on the latter, so get in touch ( if you are interested. Sure would be great to get your feedback. I have not previously used the Enthought Python distribution, but I'm very interested in trying it out. Potentially we could supply a Framework build of Mapnik against Enthought but support for easy linking to the Enthought distribution should also be added to Mapnik's build scripts if it is not already possible in trunk.

    Mapnik's C++ is heavily templated, and while swig would be nice (to allow for other language bindings), for python bindings we use the boost_python libraries. Their installation and linking is tricky - as you note - but they are a powerful and flexible way to write python bindings since we are already using boost.

    Anyway, really nice work and I look forward to learning more from you about how to improve Mapnik.

  2. I would be interested how you added the gdal libraries to a running Enthought environment on the Mac? If i just copy the .pth file into Enthought's site-package folder, ipython doesn't even start anymore, crashing with a mysterious 'Bus Error'?
    I wish, Enthought would pack gdal as well, it's becoming so important with the increasing value of GIS applications.