simplest way to benchmark a URL

Since I will forget this:

ab -c 10 -n 100

ab is a simple benchmarking tool from Apache. -c 10 means 10 concurrent connections, -n 100 means 100 requests. Note that it gave me Benchmarking localhost (be patient)...apr_socket_recv: Connection refused (61) when I tried it on http://localhost:3000/, but that seems to be an IPv6 thing and replacing that with seemed to work.

You can also do this with POST requests if you have HTTP Basic auth:

echo "observation[latitude]=38&observation[longitude]=-121" > post.txt && \
ab -c 10 -n 100 \
  -A username:password \
  -T application/x-www-form-urlencoded \
  -p post.txt \
  "" && \
rm post.txt

Haven’t found a way to just give it POST params without creating a file, but whatever, this works.

windshaft on ubuntu

Just set up Windshaft on Ubuntu for the first time, and since there were a few hiccups I figured I’d give a rough outline of my installation process:

# Use mapnik 2.2 (
# I was getting some errors with interactivity requests on an older version of
# mapnik
sudo add-apt-repository ppa:mapnik/v2.2.0
sudo apt-get update
sudo apt-get install libmapnik libmapnik-dev mapnik-utils python-mapnik
# Uninstall system gyp, it was causing problems for me when compiling mapnik
# node extensions, e.g. gyp 'module' object has no attribute 'script_main'
sudo aptitude remove gyp
# Use nvm ( instead of the package nodejs,
# which didn't seem to work for me, though that may have been tangled up with
# the mapnik 2.2 issue. Either way, nvm should install a working version of
# nodejs w/ npm.
curl | sh
nvm install 0.10.26 # or whatever works
# install windshaft with npm
npm install windshaft

I’m also using nginx and Passenger to serve the web app. I’ll assume you know how to install those two things and skip to my nginx conf:

http {
    passenger_root /path/to/passenger;
    passenger_ruby /path/to/ruby;
    passenger_nodejs /home/inaturalist/.nvm/v0.10.26/bin/node;
    server {
      listen 80;
      passenger_enabled on;
      passenger_app_root /path/to/app;
      passenger_document_root /path/to/app/public;
      error_log /var/log/nginx/your-error.log;
      access_log /var/log/nginx/your-access.log;

One thing that threw me for a while is that Passenger won’t work with older versions of node (see, so make sure you’re using 1.0 or higher.

Also note that console.log in Node will write to /var/log/nginx/error.log, not to one of your server’s custom log files.

I was running into the error described here,

ERROR: index 'model_name_core': sql_range_query: ERROR: integer out of range

so I re-compiled Postgres with 64 bit support and upgraded to a version of Thinking Sphinx that incorporated the patch (1.4.6), but still no dice. After manually re-running the SQL query Sphinx was using many times, I finally narrowed the problem down to a row with a date set in the year 8, as in 0008. TS generates a SQL statement that extracts the epoch from date column and casts them as INTs, but this should have been cast as something bigger. This was fixed in d9f3df980a6f8441a17b with the use_64_bit config option, but that’s fix isn’t available in the 1.x line of TS, and the 2.x line doesn’t work with Rails 2.x, so I lose.

The problematic row was erroneous, so I fixed it and that repaired indexing, but I’m still running a potentially buggy version of TS. Ugh.

However, I did learn that \x enables vertical output in psql, much like \G does in mysql. Yay.

Just installed Mapnik on the iNat slice at Slicehost, which runs Ubuntu Hardy (8.04). Since the process was not exactly trivial, here’s what I ended up doing:

# Get up to date and install deps
# Docs:
aptitude update
aptitude install binutils cpp-3.3 g++-3.3 gcc-3.3 gcc-3.3-base libboost-dev libboost-filesystem-dev libboost-filesystem1.34.1 libboost-iostreams-dev libboost-iostreams1.34.1 libboost-program-options-dev libboost-program-options1.34.1 libboost-python-dev libboost-python1.34.1 libboost-regex-dev libboost-regex1.34.1 libboost-serialization-dev libboost-serialization1.34.1 libboost-thread-dev libboost-thread1.34.1 libicu-dev libicu38 libstdc++5 libstdc++5-3.3-dev python2.5-dev
aptitude install libfreetype6 libfreetype6-dev libjpeg62 libjpeg62-dev libltdl3 libltdl3-dev libpng12-0 libpng12-dev libtiff4 libtiff4-dev libtiffxx0c2 python-imaging python-imaging-dbg proj
aptitude install libcairo2 libcairo2-dev python-cairo python-cairo-dev libcairomm-1.0-1 libcairomm-1.0-dev libglib2.0-0 libpixman-1-0 libpixman-1-dev libpthread-stubs0 libpthread-stubs0-dev ttf-dejavu ttf-dejavu-core ttf-dejavu-extra
aptitude install libgdal-dev python2.5-gdal
aptitude install libxslt1.1 libxslt1-dev libxml2-dev libxml2
easy_install jonpy
easy_install lxml
# Configure and instlall mapnik from source
# Docs:
python scons/ configure INPUT_PLUGINS=shape,gdal,ogr,sqlite
python scons/
python scons/ install
# Make sure ld knows where to find the new libs
# Thanks to
echo "/usr/local/lib64" >> /etc/

My main mistake was thinking that the python-mapnik package in Aptitude was going to cut mustard. Didn’t even include OGR support! Luckily the most recent Mapnik release (0.6.1) built just fine. Still easier that installing under OS X.

I just put on Ruby Enterprise Edition. Unfortunately, I’m really not seeing any savings in memory consumption, aside from the Passenger ApplicationSpawner taking up a few less MB. Kind of dissapointed.

However, installing REE on my Mac was slightly less trivial than I was led to believe. Suffice it to say the REE installer assumes you have dependent libs in /usr or /usr/local, and if you don’t, there aren’t any flags you can pass in to change that. However, you can set ENV vars, so I was able to get the installer to work like this:

export CC=gcc && export CXX=g++ && ./installer

Basically, the REE installer checks deps by using Ruby to write and compile a C file, but it’s default path to gcc was not the same as the gcc I normally use.

After that, I had to make sure all my gems were in place. I ended up installing rails, Hpricot, and ruby-debug (the latter 2 require compiled C) using REE’s own copy of rubygems, and I made sure the rest of our gems were vendorized. REE’s rubygems is just like normal:

/path/to/ree/bin/gem install rails ruby-debug hpricot

If you run into problems, make sure you’re tailing Apache’s error log and not just the Rails log, b/c that’s where Passenger will moan about its problems.

As I said, memory gains seem non-existent on our Ubuntu production machine, which is weird because I saw a 25% reduction in memory usage on my Macbook.

My Ubuntu Cheat Sheet

Looks like I’m maintaining an Ubuntu server these days…

Package Management

# Finding Installed Dependent Packages
apt-cache rdepends --installed packageX
# Installing Individual .deb Packages
sudo dpkg -i package_file.deb
# Ubuntu version info
cat /etc/lsb-release

Trust not to aptitude…

Sometimes aptitude does silly things, like install X11 when all you wanted was ImageMagick. Try apt-get as well to see if it doesn’t have a different set of deps (this is true of ImageMagick as of December 2008).


Old-style service scripts are in /etc/init.d but up-to-date methods of managing them are described at

# make a service start at startup
sudo update-rc.d servicename defaults
# remove from startup
sudo update-rc.d -f servicename remove