Build your own_map_by_yourself

..

Instruction in Building TileServer and Providing MapEditor for Crowdsourcing

.

Build Your Own Map by Yourself!

.

by

.

Marc Huang

.

[email protected]

. September 16, 2015

..

References

.

1. https://switch2osm.org/This website teaches you how to build tile servers

2. https://github.com/openstreetmap/openstreetmap-websiteA project to build OpenStreetMap website and API which used Ruby onRails

3. http://wiki.openstreetmap.org/OpenStreetMap Wikipedia

...Introduction

.1/46

..

Introduction

..

This slide is for...

.Introduction

.2/46

1. Who has map data and wants to provide tile servers2. Who wants to edit map by using editors such as iD,

Potlatch3. Who is interested in building a local OpenStreetMap

..

OSM Component Overview

.Introduction

.3/46

..

What will we do

.Introduction

.4/46

..

Contents

.Introduction

.5/46

1. Build a tile server2. Build a website containing map editors, API, and UI3. Connect two services together

..

Environment

.Introduction

.6/46

. VirtualBox VM

. Ubuntu 12.04 (32-bit)

. Memory: 2GB

. Storage: 120GB,

. Networks: NAT and Host-only Adapter

..

My settings

.Introduction

.7/46

. User: map

. Password: 000000

. ssh installed (sudo apt-get install openssh-server)

. Basically, you should know how to get the “ip” of yourVM (ifconfig, and it looks like “192.168.x.x”) and how touse “ssh” (ssh [email protected])

. If you get confused, you probably have to google“virtualbox ubuntu install, virtualbox ssh”

. If not, good! Let’s move on!

...Build a Tile Server

.8/46

..

Build a Tile Server

..

Preparing & Software Install

.Build a Tile Server

.9/46

. sudo apt-get update

. sudo apt-get upgrade

. sudo apt-get install subversion git-core tar unzip wget bzip2build-essential autoconf libtool libxml2-dev libgeos-dev libpq-devlibbz2-dev proj munin-node munin libprotobuf-c0-devprotobuf-c-compiler libfreetype6-dev libpng12-dev libtiff4-devlibicu-dev libboost-all-dev libgdal-dev libcairo-devlibcairomm-1.0-dev apache2 apache2-dev libagg-dev liblua5.2-devttf-unifont vim

. mkdir src

. cd src

. wgethttps://www.dropbox.com/s/a1awl6ua037oy1y/osm2pgsql-0.81.0.zip

. wgethttps://www.dropbox.com/s/vqghx8fbbwq966u/osmosis-bin-0.35.zip

..

Install Postgresql/PostGIS


.10/46

. sudo apt-get install postgresql-9.1-postgispostgresql-contrib-9.1 postgresql-server-dev-9.1

Note: you must indicate a version of postgresql or you will geterrors (as following block showed) after rebooted your VMErrorpsql: could not connect to server: No such file or directoryIs the server running locally and accepting connections on Unixdomain socket ”/var/pgsql_socket/.s.PGSQL.5432”?

..

Set up Postgresql/PostGIS


.11/46

. sudo -u postgres -i

. createuser <username>(e.g., my user name is “map” then $ createuser map)

. createdb -E UTF8 -O <username> <database>(e.g., $ createdb -E UTF-8 -O map map)

. psql -f/usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql-d <database>(you will see “CREATE FUNCTION... DROP FUNCTION”)

. psql -d <database> -c "ALTER TABLEgeometry_columns OWNER TO <username>; ALTERTABLE spatial_ref_sys OWNER TO <username>;"(you will see “ALTER TABLE”)

..

Install & Set up osm2pgsql


.12/46

. cd ~/src

. unzip osm2pgsql-0.81.0.zip

. cd osm2pgsql-0.81.0

. ./autogen.sh

. ./configure

. make

. sudo make install

. psql -f /usr/local/share/osm2pgsql/900913.sql -d<database>

Note: You can check it by using “osm2pgsql -v”

..

Install Mapnik


.13/46

. cd ~/src

. git clone git://github.com/mapnik/mapnik

. cd mapnik

. git branch 2.0 origin/2.0.x

. git checkout 2.0

. python scons/scons.py configure INPUT_PLUGINS=allOPTIMIZATION=3SYSTEM_FONTS=/usr/share/fonts/truetype/

. python scons/scons.py

. sudo python scons/scons.py install

. sudo ldconfigNote: You can check it by using “python»» import mapnik»» exit()”

..

Install mod_tile


.14/46

. cd ~/src

. git clone git://github.com/openstreetmap/mod_tile.git

. cd mod_tile

. ./autogen.sh

. ./configure

. make

. sudo make install

. sudo make install-mod_tile

. sudo ldconfig

..

Install and set up mapnik style-sheet


.15/46

. cd ~/src

. svn cohttp://svn.openstreetmap.org/applications/rendering/mapnikmapnik-style

. cd mapnik-style

. sudo ./get-coastlines.sh /usr/local/share

. cd inc

. cp fontset-settings.xml.inc.templatefontset-settings.xml.inc

. cp datasource-settings.xml.inc.templatedatasource-settings.xml.inc

. cp settings.xml.inc.template settings.xml.inc

..

Configuration: setting.xml.inc


.16/46

. sudo vim setting.xml.inc

These lines should change as following<!ENTITY symbols "symbols"><!ENTITY osm2pgsql_projection "&srs900913;"><!ENTITY dwithin_node_way "&dwithin_900913;"><!ENTITY world_boundaries "/usr/local/share/world_boundaries"><!ENTITY prefix "planet_osm">

..

Configuration: datasource settings


.17/46

. sudo vim datasource-settings.xml.inc

These lines should change as following:<Parameter name="type">postgis</Parameter><Parameter name="dbname">map</Parameter><Parameter name="estimate_extent">false</Parameter><Parametername="extent">-20037508,-19929239,20037508,19929239</Parameter>

Note: My dbname is “map” and you have to change it to your dbname

..

Configuration: fontset settings


.18/46

. sudo vim fontset-settings.xml.inc

These lines should change as following:<FontSet name="book-fonts"><Font face-name="unifont Medium" /></FontSet><FontSet name="bold-fonts"><Font face-name="unifont Medium" /></FontSet><FontSet name="oblique-fonts"><Font face-name="unifont Medium" /></FontSet>

..

renderd configuration


.19/46

. sudo vim /usr/local/etc/renderd.conf

This file should looks like as following[renderd]socketname=/var/run/renderd/renderd.socknum_threads=4tile_dir=/var/lib/mod_tilestats_file=/var/run/renderd/renderd.stats[mapnik]plugins_dir=/usr/local/lib/mapnik/inputfont_dir=/usr/share/fonts/truetype/unifontfont_dir_recurse=1[default]URI=/osm_tiles/TILEDIR=/var/lib/mod_tileXML=/home/map/src/mapnik-style/osm.xmlHOST=localhostTILESIZE=256

Note: You have to delete all other unused lines in this file.


.20/46

. sudo mkdir /var/run/renderd

. sudo chown map /var/run/renderd

. sudo mkdir /var/lib/mod_tile

. sudo chown map /var/lib/mod_tile

. sudo vim /etc/apache2/conf.d/mod_tile

add this line into file “mod_tile”LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so

..

Apache configuration


.21/46

. sudo vim /etc/apache2/sites-available/default

add these lines after line “webmaster@localhost”LoadTileConfigFile /usr/local/etc/renderd.confModTileRenderdSocketName /var/run/renderd/renderd.sockModTileRequestTimeout 0ModTileMissingRequestTimeout 30

..

Tuning Postgresql and System


.22/46

. sudo vim /etc/postgresql/9.1/main/postgresql.conf

change these linesshared_buffers = 128MBcheckpoint_segments = 20maintenance_work_mem = 256MBautovacuum = off

. sudo vim /etc/sysctl.conf

add this line at the end of file “sysctl.conf”kernel.shmmax=268435456

. sudo rebootTake a break, make a cup of tea :)

..

Download map data


.23/46

. mkdir ~/osm

. cd osm

. wgethttp://download.geofabrik.de/asia/taiwan-latest.osm.pbf

Download a small portion of world map for testing. I live in Taiwan so Idownload the map of Taiwan)

..

Import map data into Postgresql


.24/46

. osm2pgsql --slim -d map -C 1600 --number-processes 2taiwan-latest.osm.pbf

If your osm2pgsql is not version 0.81, you might face errors.pthread_join() returned exception: Throw location unknown (consider usingBOOST_THROW_EXCEPTION)Dynamic exception type:boost::exception_detail::clone_impl<boost::exception_detail::current_exception_std_exception_wrapper<std::runtime_error>>std::exception::what: CREATE TABLE planet_osm_point_tmp AS SELECT *FROM planet_osm_point ORDER BY CASE WHEN ST_IsEmpty(way) THENNULL ELSE ST_GeoHash(ST_Transform(ST_Envelope(way),4326),10) ENDfailed: ERROR: GetProj4StringSPI: Cannot find SRID (4326) in spatial_ref_sy

..

Start your tile server


.25/46

. sudo mkdir /var/run/renderd

. sudo chown map /var/run/renderd

. renderd -f -c /usr/local/etc/renderd.confIf your used ssh you might face a problem”locale::facet::_S_create_c_locale name not valid”, just execute thefollowing command before you start tile serverexport LC_ALL="en_US.UTF-8"Now, open a new terminal and execute

. sudo /etc/init.d/apache2 restart

..

Check your tile server


.26/46

Open your browser and go to:http://localhost/osm_tiles/0/0/0.pngIf you see a picture of world map then the tile server is working.You can stop renderd process by press Ctrl+c (the originalterminal)Otherwise, if any error occured, you have to double check all theconfigurations and install steps. For example, all the config files infolder “src/mapnik-style/inc”, renderd.conf, and others.

..

Create a startup daemon for renderd


.27/46

If everything works well then we are going to create a starupdaemon for renderd.

. sudo cp ~/src/mod_tile/debian/renderd.init/etc/init.d/renderd

. sudo chmod u+x /etc/init.d/renderd

. sudo vim /etc/init.d/renderd

change these lines and replace “map” to your usernameDAEMON=/usr/local/bin/$NAMEDAEMON_ARGS="-c /usr/local/etc/renderd.conf"RUNASUSER=map

. sudo /etc/init.d/renderd start

. sudo ln -s /etc/init.d/renderd /etc/rc2.d/S20renderd

..

Using your tile server


.28/46

. sudo mkdir /var/www/osm

. cd /var/www/osm

. sudo wgethttp://cdn.leafletjs.com/downloads/leaflet-0.7.5.zip

This url may change, you can go to http://leafletjs.com/download.html todownload the latest version

. sudo unzip leaflet-0.7.5.zip

. sudo vim index.html


.29/46

Copy and paste this script to index.html<html><head><title>My first tile server</title><link rel="stylesheet" type="text/css" href="leaflet.css"/><script type="text/javascript" src="leaflet.js"></script><style>#map{width:100%;height:100vh}</style></head><body><div id="map"></div><script>var map = L.map('map').setView([25.555,121.899],7);L.tileLayer('http://192.168.x.x/osm_tiles/{z}/{x}/{y}.png',{maxZoom:18}).addTo(map);</script></body></html>

..

Check your map


.30/46

Go to:http://192.168.x.x/osmor if you visit from localhttp://localhost/osmIf you were not used map of Taiwan, you should change the initiallocation (setView([Lag,Lng],7)).Next, we are going to build a map website and provide an editorfor users to edit map data by API

...Build an OpenStreetMap Website

.31/46

..

Build an OpenStreetMapWebsite

..

Preparing & Software Install

.Build an OpenStreetMap Website

.32/46

. cd

. sudo apt-get install ruby1.9.1 libruby1.9.1 ruby1.9.1-devri1.9.1 libmagickwand-dev libxslt1-dev nodejsapache2-threaded-dev build-essential libpq-devlibsasl2-dev default-jre

. sudo gem1.9.1 install bundler

..

Install openstreetmap-website


.33/46

. git clonehttps://github.com/openstreetmap/openstreetmap-website.git

. cd openstreetmap-website

. bundle install

. cp config/example.application.yml config/application.yml

. cp config/example.database.yml config/database.yml

. sudo vim config/image_optim.yml

add this line at the end of this filesvgo: false

..

Create & Set up Database


.34/46

. bundle exec rake db:create

. psql -d openstreetmap -c "CREATE EXTENSIONbtree_gist"you will see ”CREATE EXTENSION”

. cd db/functions

. make libpgosm.soyou will see some compiling commands start by ”cc”

. cd ../..

..

Database Configuration


.35/46

. psql -d openstreetmap -c "CREATE FUNCTION maptile_for_point(int8,int8, int4) RETURNS int4 AS '`pwd`/db/functions/libpgosm','maptile_for_point' LANGUAGE C STRICT"

. psql -d openstreetmap -c "CREATE FUNCTION tile_for_point(int4,int4) RETURNS int8 AS '`pwd`/db/functions/libpgosm','tile_for_point' LANGUAGE C STRICT"

. psql -d openstreetmap -c "CREATE FUNCTION xid_to_int4(xid)RETURNS int4 AS '`pwd`/db/functions/libpgosm', 'xid_to_int4'LANGUAGE C STRICT"

. bundle exec rake db:migrate

. bundle exec rake testto ensure that everything is set up properly

..

Start Your Webiste


.36/46

. bundle exec rails server -b 192.168.x.xif you will browse from local then just execute “bundle exec railsserver”

Go to http://192.168.x.x:3000, if this website just looks likeOpenStreetMap’s website, bravo!Sign up an account and note down your “Display Name”Open a new terminal after you are in the “Check your email!” page

..

Managing Users


.37/46

. cd openstreetmap-website

. bundle exec rails console

. » user = User.find_by_display_name(”Your Display Name”)=> #[...]

. » user.status = ”active”=> ”active”

. » user.save!=> true

. » quitGo back to your browser and refresh the page, check if you are logged in.Next, on the top-right corner, select “My settings” and click “oauthsettings” then “Register your application”Fill “iD” into “Name”, fill “http://192.168.x.x:3000” into “MainApplication URL”, and checked “modify the map” in belowClick “Register” and copy the “Consumer key”

..

Editor Configuration


.38/46

. sudo vim config/application.yml

find the “id_key” line, uncomment it and fill the key’id_key: "VGWM4FlewbJSjLvWbxml5xr45zus8nH0DWKN6yBw"Restart the rail server and go to your website to click “Edit”button (default editor is iD)You will see the iD editor is working.

..

Install & Set up Osmosis


.39/46

. cd ~/src

. wgethttp://bretth.dev.openstreetmap.org/osmosis-build/osmosis-latest.zip

. mkdir osmosis-latest

. unzip osmosis-latest.zip -d osmosis-latest

. chmod u+x osmosis-latest/bin/osmosis

. sudo -u postgres psql

. # \password

. # Enter password:...set a password for “posstgres”

. # \q

..

Import data by API


.40/46

. ./osmosis-latest/bin/osmosis --read-pbf../osm/taiwan-latest.osm.pbf --write-apidbhost="localhost" user="postgres" password="YourPassword" database="openstreetmap"validateSchemaVersion="no"

Note: If your map data version is not 0.6 then you have to use osmosis-0.35 toconvert it first.

..

The ”duplicate key” problem


.41/46

. psql -d openstreetmap

execute the following commandsSELECT pg_catalog.SETVAL('acls_id_seq', (SELECT MAX(id) FROM acls))SELECT pg_catalog.SETVAL('changesets_id_seq', (SELECT MAX(id) FROM changesets ))SELECT pg_catalog.SETVAL('client_applications_id_seq', (SELECT MAX(id) FROM client_applications))SELECT pg_catalog.SETVAL('current_nodes_id_seq', (SELECT MAX(id) FROM current_nodes))SELECT pg_catalog.SETVAL('current_relations_id_seq', (SELECT MAX(id) FROM current_relations))SELECT pg_catalog.SETVAL('current_ways_id_seq', (SELECT MAX(id) FROM current_ways))SELECT pg_catalog.SETVAL('diary_comments_id_seq', (SELECT MAX(id) FROM diary_comments))SELECT pg_catalog.SETVAL('diary_entries_id_seq', (SELECT MAX(id) FROM diary_entries))SELECT pg_catalog.SETVAL('friends_id_seq', (SELECT MAX(id) FROM friends))SELECT pg_catalog.SETVAL('gpx_file_tags_id_seq', (SELECT MAX(id) FROM gpx_file_tags))SELECT pg_catalog.SETVAL('gpx_files_id_seq', (SELECT MAX(id) FROM gpx_files))SELECT pg_catalog.SETVAL('messages_id_seq', (SELECT MAX(id) FROM messages))SELECT pg_catalog.SETVAL('note_comments_id_seq', (SELECT MAX(id) FROM note_comments))SELECT pg_catalog.SETVAL('notes_id_seq', (SELECT MAX(id) FROM notes))SELECT pg_catalog.SETVAL('oauth_nonces_id_seq', (SELECT MAX(id) FROM oauth_nonces))SELECT pg_catalog.SETVAL('oauth_tokens_id_seq', (SELECT MAX(id) FROM oauth_tokens))SELECT pg_catalog.SETVAL('redactions_id_seq', (SELECT MAX(id) FROM redactions))SELECT pg_catalog.SETVAL('user_blocks_id_seq', (SELECT MAX(id) FROM user_blocks))SELECT pg_catalog.SETVAL('user_roles_id_seq', (SELECT MAX(id) FROM user_roles))SELECT pg_catalog.SETVAL('user_tokens_id_seq', (SELECT MAX(id) FROM user_tokens))SELECT pg_catalog.SETVAL('users_id_seq', (SELECT MAX(id) FROM users))\q

Now you can use iD to edit your own map.

...Connect Together

.42/46

..

Connect Together

..

Change Standard Map to Your Map

.Connect Together

.43/46

. sudo vimopenstreetmap-website/vendor/assets/leaflet/leaflet.osm.js

find all matches of the following textsurl: document.location.protocol === 'https:' ?'https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png' :'http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png',

replace all tourl: 'http://192.168.x.x/osm_tiles/{z}/{x}/{y}.png',

..

Change Initial Location of Map

.Connect Together

.44/46

. sudo vim openstreetmap-website/app/assets/javascripts/osm.js.erb

set your initial Location#mapParams.lon = -0.1;mapParams.lon = 120.1204;#mapParams.lat = 51.5;mapParams.lat = 23.0260;#mapParams.zoom = parseInt(params.zoom || 5);mapParams.zoom = parseInt(params.zoom || 10);

..

Update Tile Server

.Connect Together

.45/46

. cd

. sudo vim update-tile.sh

add these commands into it./src/osmosis-latest/bin/osmosis --read-apidb host="localhost"database="openstreetmap" user="postgres" password="YourPassword" validateSchemaVersion="no" --write-xmlfile=".osm/update.osm"osm2pgsql --slim -d map -C 1600 ./osm/update.osmsudo rm -R /var/lib/mod_tile/default/sudo /etc/init.d/renderd restartNote: You can execute these commnads separated instead of using shell script.

. sudo chmod u+x update-tile.sh

. ./update-tile.sh

..

Finally

.Connect Together

.46/46

If you see the changes of your map on your tile server theneverything is working, congratulations! You can download the draftof all install commands here:https://www.dropbox.com/s/oglhsw57slcsybt/installOr, email me: Marc Huang<[email protected]> if youhave any questions.Thank you, happy coding.

Date post:	08-Jan-2017
Category:	Technology
Upload:	marc-huang
View:	763 times
Download:	2 times

Build your own_map_by_yourself

Technology