Tutorial: Building your own map server with OpenStreetMap on Ubuntu 18.04 – Part 1

So you want to host your own map server? It’s reasonably easy with OpenStreetMap and Ubuntu 18.04. This tutorial will walk you through the necessary steps.

1. Introduction
2. Hardware Requirements
3. Installing Required Packages
4. Setting up
5. Finishing Touches

1. Introduction

OpenStreetMap is a free project founded in 2004 that aims to gather free data in order to create a free world map. That means that this map data is not under control by any particular company like Google Maps or Apple Maps.

There are various ways to work with that data, but the most common way is to import them into a Postgresql database with the postgis plugin and then publish them through a render backend. One of the most popular render backends are renderd/mod_tile, which will allow us to serve tiles through a webserver. Finally, for displaying the map in the browser, we’ll be using OpenLayers, a Javascript library.

2. Hardware Requirements

Map Size	Hardware Requirements
Small Country (e.g. Germany)	8 GB RAM, HDD possible
Big Country (e.g. USA)	16 GB RAM, HDD possible
Planet	32 GB RAM, SSD required ~ 900 GB of data

3. Installing Required Packages

We’ll be starting by installing the required packages. For simplicity, we’re assuming you have acquired superuser privileges through one way or another.

# apt-get update && apt-get upgrade
# apt-get install postgresql postgresql-contrib postgis postgresql-10-postgis-2.4 postgresql-10-postgis-2.4-scripts osm2pgsql git

As previously mentioned, we’re using postgresql as our database. Postgis is the plugin for spatial data. Osm2pgsql is the tool to import the data into the data base. Git is required for cloning a few repositories we need later on. If you get the message that apt can’t find some or any of these packages, take a look at /etc/apt/sources.list. It should look something like this:

deb http://de.archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
#deb-src http://de.archive.ubuntu.com/ubuntu bionic main restricted universe multiverse

deb http://de.archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
#deb-src http://de.archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse

deb http://de.archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse
#deb-src http://de.archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse

deb http://de.archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse
#deb-src http://de.archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse

# deb http://archive.canonical.com/ubuntu bionic partner
# deb-src http://archive.canonical.com/ubuntu bionic partner

Furthermore, we need the mapnik packages, carto and nodejs:

# apt-get install curl unzip gdal-bin mapnik-utils libmapnik-dev nodejs npm 
# npm install -g carto

To render and display local tiles, we need mod_tile and renderd:

# sudo add-apt-repository ppa:osmadmins/ppa
# apt install libapache2-mod-tile renderd

We also need some fonts for displaying our map later:

# apt-get install ttf-dejavu fonts-noto-cjk fonts-noto-hinted fonts-noto-unhinted ttf-unifont

These should be all the packages that are required.

4. Setting up

After we have installed the required packages, we can start to set up our map server.

First, we’ll switch over to our postgres user that was automatically created by installing the postgres database.

# sudo -u postgres -i

Then, we’ll create a new database user:

# createuser osm

Next, we’ll create a new UTF-8 encoded data base named gis that belongs to our new user .

# createdb -E UTF8 -O osm gis

We’ll set osm as the owner of this new database:

# psql -c "ALTER TABLE spatial_ref_sys OWNER TO osm;" -d gis

Now, we need to create and enable two extensions four our newly created database:

# psql -c "CREATE EXTENSION postgis;" -d gis
# psql -c "CREATE EXTENSION hstore;" -d gis

We’re done with the new osm user for now:

# exit

Now, we’re adding another system user called osm:

# adduser osm

Now, we need to clone the https://github.com/gravitystorm/openstreetmap-carto repository, as that contains styles and transformation scripts we need for the database import.

# git clone https://github.com/gravitystorm/openstreetmap-carto.git

Next, we need to download a *.pbf file for import. They’re available at http://download.geofabrik.de/ .

Example for the whole planet:

# wget -c http://planet.openstreetmap.org/pbf/planet-latest.osm.pbf

Now, we can actually import our data:

# osm2pgsql --slim -d gis --hstore --multi-geometry --number-processes 2 --tag-transform-script /home/osm/openstreetmap-carto/openstreetmap-carto.lua --style /home/osm/openstreetmap-carto/openstreetmap-carto.style -C 12000 planet-latest.osm.pbf 

The -C option should match the amount of RAM on your PC/virtual machine, while the –number-processes option should match the amount of cores you have available. Depending on the size of your pbf file, the import can take a very long time. On my virtual machine with 32 GB RAM, 1 core and a 1 TB SSD, the import took around 47 hours. Your mileage might vary.

Since this takes so long, it’s highly recommended to run osm2pgsql in a tmux session if you’re connected to your future map server remotely. Then you can detach from your session and ensure that your import process won’t stop when your ssh session is disrupted for whatever reason.

We now need to switch back to the osm user and cd into the directory where we cloned openstreetmap-carto to. There we need to run

# scripts/get-shapefiles.py

This script will load shapefiles from the internet. Shapefiles are another format in which we can store geodata.

Then, we’ll use carto that we installed earlier through npm to convert our project.mml file to a style file for mapnik.

# carto project.mml > style.xml

This usually yields a lot of warnings, but they can safely be ignored. Now, it’s time to configure renderd.

# vim /etc/renderd.conf

num_threads should match the number of cores on your mapserver. plugins_dir should be /usr/lib/mapnik/3.0/input XML should match the style file we generated earlier through carto.

Save and close the file. Now edit the init script /etc/init.d/renderd

# vim /etc/init.d/renderd

RUNASUSER should be set to osm.

Then, change the permissions of the mod_tile folder:

# sudo chown osm:osm /var/lib/mod_tile/ -R

Reload the daemon on disk and restart renderd:

# systemctl daemon-reload
# systemctl start renderd

If you browse to http:/yourhost/osm/slippymap.html and select local tiles, you should be able to see a map of your imported *.pbf.

Congrats! This concludes parts 1 of this tutorial. In part 2, we’ll deal with prerendering tiles and displaying everything nice and clean through Openlayers, a powerful Javascript library.