Using MySQL lat/lon Columns as a Mapnik Datasource

Ken-ichi — Mon, 04 May 2009 05:07:28 +0000

With the gracious help of Dane Springmeyer, I recently got Mapnik up and running on my Mac, so naturally the first thing I wanted to do was to plot iNaturalist observations. Mapnik will talk with a PostGIS database just fine, but it wont’t pull points from a MySQL database table, at least not directly. What it does support, though, is the all-powerful OGR. Back when I used to use Mapserver, I had used OGR a few times to act as a bridge between simple lat/lon columns in a MySQL table and Mapserver (great description here), so I figured the same would be possible in Mapnik.

First Hurdle: Getting Mapnik to Show Anything At All

First I made an OVF file so OGR would know how to talk to my database:

>
   name="observations">
    >MYSQL:inaturalist_development,user=root,tables=observations> 
    >SELECT id, latitude, longitude, iconic_taxon_id FROM observations> 
    >wkbPoint>
     encoding="PointFromColumns" x="longitude" y="latitude"/> 
  >
>

and made sure it was working:

mapnik-inat > ogrinfo inat.ovf
ERROR 4: Update access not supported for VRT datasources.
Had to open data source read-only.
INFO: Open of `inat.ovf'
      using driver `VRT' successful.
1: observations (Point)

Then I basically modified the Mapnik tutorial to use the OGR virtual data source:

import mapnik
 
# Setup the datasource and layer
ds = mapnik.Ogr(file="inat.ovf", layer="observations")
lyr = mapnik.Layer('test lyr')
lyr.datasource = ds
 
# Style the points using a simple PNG
m = mapnik.Map(600, 300, "+proj=latlong +datum=WGS84")
s = mapnik.Style()
r = mapnik.Rule()
symbolizer = mapnik.PointSymbolizer('mm_8_stemless_iNatGreen.png', 'png', 8, 8)
symbolizer.allow_overlap = True
r.symbols.append(symbolizer)
s.rules.append(r)
m.append_style('My Style', s)
lyr.styles.append('My Style')
m.layers.append(lyr)
 
# Zoom to the extent of the data and write the map to a PNG
m.zoom_to_box(lyr.envelope())
mapnik.render_to_file(m, 'test.png', 'png')

And then I got a map!

Ok, a kind of crappy map of test data, but actual, real, visible pixels! So much better than arcane error messages!

Second Hurdle: Google

Of course, it’s just a map of some test points, but still, it worked. Yay. Next thing I wanted to do was to see how easy it would be to lay these points over a Google map, since one of the things I’d like to do with Mapnik is have a layer showing ALL observations on the iNat observation browser. Turns out this was a piece of cake using tilelite.

First I put my map configuration in an XML file, adding a bit of complexity using SQL filters to show different markers for different kinds of points (here corresponding to the IDs for different iconic taxa, e.g. plants, animals, fungi, etc.).

 srs="+proj=latlong +datum=WGS84">
   name="observationsStyle">
    >
      >[iconic_taxon_id] = 47126>
       file="mm_8_stemless_iNatGreen.png" type="png" width="8" height="8" />
    >
    >
      >[iconic_taxon_id] = 1 or [iconic_taxon_id] = 47178 or [iconic_taxon_id] = 3 or [iconic_taxon_id] = 20978 or [iconic_taxon_id] = 26036 or [iconic_taxon_id] = 40151>
       file="mm_8_stemless_DodgerBlue.png" type="png" width="8" height="8" />
    >
    >
      >[iconic_taxon_id] = 47115 or [iconic_taxon_id] = 47119 or [iconic_taxon_id] = 47158>
       file="mm_8_stemless_OrangeRed.png" type="png" width="8" height="8" />
    >
    >
      >[iconic_taxon_id] = 47170>
       file="mm_8_stemless_DeepPink.png" type="png" width="8" height="8" />
    >
    >
      />
       file="mm_8_stemless_unknown.png" type="png" width="8" height="8" />
    >
  >
 
   name="observationsLayer" srs="+proj=latlong +datum=WGS84">
    >observationsStyle>
    >
       name="type">ogr>
       name="file">inat.ovf>
       name="layer">observations>
    >
  >
>

Then I fired up tilelite:

tilelite > python liteserv.py ~/projects/mapnik-inat/inatmap.xml
Using mapfile: '/Users/kueda/projects/mapnik-inat/inatmap.xml'
Listening on port 8000...

and added a new GTileLayer to my map:

var myCopyright = new GCopyrightCollection();
var allObsLyr = new GTileLayer(myCopyright, 0, 12, {
  isPNG: true,
  tileUrlTemplate: 'http://localhost:8000/{Z}/{X}/{Y}.png'
});
var allObsOverlay = new GTileLayerOverlay(allObsLyr);
map.addOverlay(window.allObsOverlay);

et voila:

I have no idea how scalable this kind of solution is, but I just wanted to show that it works. It was my first real experiment with Mapnik, and I’m pretty happy with how simple things were after the decidedly unsimple installation process.

Extending Existing Django ModelAdmin Classes

Ken-ichi — Fri, 25 Jul 2008 19:04:39 +0000

Django 1.0 is coming, and it’s bringing needed but backwards-incompatible changes to the admin site, ostensibly the best thing about Django. One problem I’ve run into while migrating to the new version is inline editing of related classes in existing classes. For example, it’s pretty common (and recommended) to “extend” Django’s built-in User model by creating your own, related Profile model. For the old admin site, you specified the ability to edit a relate Profile in the User form in the Profile model:

class Profile(models.Model):
    user = models.ForeignKey(User, unique=True, edit_inline=True)

That was simple, but kinda quirky. Changing the way the User form looks by changing the related Profile model? Weird. BUT, it did let you make changes to the User model w/o mucking with the built in Django code.

The new way of doing things is to specify all your admin site options in a separate file, and inline editing is now a little more reasonable: if you want to edit related objects in a model’s form, you specify that in that model’s admin options. But this means you need to alter Django’s built-in User model’s admin options, which could mean muckiness. So here’s what I did in my myapp/admin.py:

from django.contrib import admin
from django.contrib.auth.models import User
from myapp.models import *
 
class ProfileAdmin(admin.ModelAdmin):
    # just some stuff for my profile forms
    filter_horizontal = ('teams',)
    radio_fields = {'appointment': admin.HORIZONTAL}
    list_display = ('user', 'picture', 'appointment', 'affiliation')
admin.site.register(Profile, ProfileAdmin)
 
# need to make an inline class...  ick
class ProfileInline(admin.StackedInline):
    model = Profile
    max_num = 1
 
# the meat: grab the existing UserAdmin class, modify it, unregister User,
# and re-register User with the modified UserAdmin
UserAdmin = admin.site._registry[User].__class__
UserAdmin.inlines = [ProfileInline,]
admin.site.unregister(User)
admin.site.register(User, UserAdmin)

Took me a while to realize admin.site._registry held instances of the ModelAdmin classes, not the classes themselves. This technique also assumes that User gets registered with the admin site before you run this code. I’m guessing the order in which admin.autodiscover() registers models is the same as the INSTALLED_APPS setting in settings.py.

kueda.net » python

Using MySQL lat/lon Columns as a Mapnik Datasource

First Hurdle: Getting Mapnik to Show Anything At All

Second Hurdle: Google

Extending Existing Django ModelAdmin Classes