Docker @ FOSS4G 2016, Bonn

transcript

An overview of Docker images for geospatial

applicationsFOSS4G conference 2016, Bonn, Germany

Daniel NüstInstitute for Geoinformatics

University of Münster

Talk abstract

http://2016.foss4g.org/talks.html#146

Docker is a growing open-source platform for building and shipping applications as cloud services in so called containers. But containers can be more than that! Following the idea of DevOps, Dockerfiles are a complete scripted definition of an application with all it's dependencies, which can be build and published as ready to use images. As each container is only running "one thing" (e.g. one application, one database, a worker instance), multiple containers can be configured with the help of docker-compose.

More and more geospatial open source projects or third parties provide Dockerfiles. In this talk, we try to give an overview of the existing Docker images and docker-compose configurations for FOSS4G projects. We report on test runs that we conducted with them, informing about the evaluation results, target purposes, licenses, commonly used base images, and more. We will also give a short introduction into Docker and present the purposes that Docker images can be used for, such as easy evaluation for new users, education, testing, or common development environments.

This talk integrates and summarizes information from previous talks at FOSS4G and FOSSGIS conferences, so I'd like to thank Sophia Parafina, Jonathan Meyer, and Björn Schilberg for their contributions.

Agenda

What is Docker? Why?

What can it be used for?

Overview of geo-related containers

Select containers Live Demo

Why containerization?Why Docker?

Motivation

http://www.slideshare.net/gmccance/cern-data-centre-evolution

Pets vs. Cattle

Motivationhttps://www.docker.com/use-cases

Science

Reproducibility is at the of

Slide by Docker inventor & Docker, Inc. CTO Solomon Hykes, DockerCon 2014

https://www.docker.com/what-dockerhttps://en.wikipedia.org/wiki/Operating-system-level_virtualizationhttps://youtu.be/ki8CZkutoxQ

Containerization using

kernel featuresnamespaceslibcontainer, LXCcgroupsresources

Houses vs. Appartments

Docker basics

Dockerfile

Docker Image

Docker Container

build run

pausestop/killstartlogscpexecrmstats

docker-compose configuration

Docker Container

updown

Docker CLI

Docker Engine

Docker Registry

“Official Docker registry”: Docker Hub

https://hub.docker.com/u/osgeo/

$ docker run -it --rm osgeo/proj.4root@f1141552cee9:/# projRel. 4.9.3, dd Month yyyyusage: proj [ -beEfiIlormsStTvVwW [args] ] [ +opts[=arg] ] [ files ]root@f1141552cee9:/#

Process and evaluation

GoogleSearch Docker HubSearch FOSS4G talksOSGeo project list

Criteria and data points

Goals / target users / applications (e.g. testing, dev env,

new users, demos, …)Maintainer (3rd party, original devs?)LicenseBase image

“geo” on Docker Hub

“gis” on Docker Hub

Degree

tfr42/deegreetfr42/teamenginemdillon/postgis

FROM tomcat:8-jre7MAINTAINER deegree TMC <tmc@deegree.org>

# set deegree versionENV DEEGREE_VERSION 3.3.18

# download deegreeRUN wget http://repo.deegree.org/content/repositories/public/org/deegree/deegree-webservices/${DEEGREE_VERSION}/deegree-webservices-${DEEGREE_VERSION}.war -O /usr/local/tomcat/webapps/deegree-webservices.war

# run tomcatCMD ["catalina.sh", "run"]

gdal - Live Demo

docker run --rm geodata/gdal

Play around with gdal, easily use different versions (of library combinations)

https://github.com/geo-data/gdal-docker

Use it as a “binary executable” in scripts, e.g. https://gist.github.com/spara/73972116fe751bf374e29618a10d5bed

https://hub.docker.com/r/spara/gdal_ef/

QGIS - Live Demo

xhost +# --rm will remove the container as soon as it endsdocker run --rm --name="qgis-desktop-2.14ltr" \ -i -t \ -v ${HOME}:/home/${USER} \ -v /tmp/.X11-unix:/tmp/.X11-unix \ -e DISPLAY=unix$DISPLAY \ kartoza/qgis-desktop:2.14xhost -

docker run --name "qgis-server-webclient" -p 8081:80 -d -t opengisch/qgis-server-webclient

PostGIS

PostGIS - Live Demo

docker run --name some-postgis -e POSTGRES_PASSWORD=mysecretpassword -d mdillon/postgis

docker run -it --link some-postgis:postgres --rm postgres \ sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres'

# \list# \q

tileserver-mapnik - Live Demo

wget -c https://osm2vectortiles-downloads.os.zhdk.cloud.switch.ch/v1.0/extracts/zurich.mbtiles

git clone https://github.com/mapbox/mapbox-studio-osm-bright.tm2.git

docker run -v $(pwd):/data -p 8085:80 klokantech/tileserver-mapnik

52°North WPS - Live Demo

docker run -p 80:8080 52n-wps-base

http://localhost/wps

pycsw Live Demo

docker run --rm -it -p 8000:8000 geocontainers/pycsw

http://localhost:8000/

GeoNode Live Demo

docker run -p 8000:8000 -p 8080:8080 openmarble/arch-geonode

http://localhost:8080/

GeoMOOSE Live Demo

docker run -p 80:80 --rm=true --name geomoose_demo geomoosehttp://localhost/geomoose/geomoose_dev.html

Great example: Instructions for demo, testing, and development usage of the image

Osmosis Live Demo

docker run --rm -t -i -v $(pwd)/tmp:/data farberg/openstreetmap-osmosis-docker --read-pbf /data/monaco.osm.pbf --write-xml /data/monaco.xml

docker run --rm -t -i -v $(pwd)/tmp:/data ferimer/osmosis:20160525 --read-pbf /data/monaco.osm.pbf --write-xml /data/monaco2.xml

diff tmp/monaco2.xml tmp/monaco.xml

http://www.pdal.io/workshop/docker.html

PDAL stands on the shoulders of giants. It uses GDAL, GEOS, and many other dependencies. Because of this, it is very challenging to build it yourself. We could easily burn an entire workshop learning the esoteric build miseries of PDAL and all of its dependencies. Fortunately, Docker will allow us to use a fully-featured known configuration to run our examples and exercises without having to suffer so much.

docker run -t pdal/pdal pdal --help

CartoDB

docker run -d -p 3000:3000 -p 8080:8080 -p 8181:8181 --name carto sverhoeven/cartodb

sudo sh -c 'echo 127.0.1.1 cartodb.localhost >> /etc/hosts'

docker run -p 80:80 --link carto:cartodb.localhost -d spawnthink/cartodb-nginx

http://cartodb.localhost

Join on the OSGeo wiki

https://wiki.osgeo.org/wiki/DockerImages

https://wiki.osgeo.org/wiki/DockerImagesMeta

Core arguments for FOSS4G community

(all the Docker advantages… write once, biz ops, cloud, etc.)

Lower entrance barrier for users AND devs(e.g. sandboxed stack per developer, J. Meyer)

Easier transition from testing to production

Tasks & Challenges for the community

Put your project on Docker Hub as an automated build and add a license

Use base images, extend them to provide useful variants

Collaborate with existing images!

Make services and apps configurable using ENVironment parameters and files

Projects and their developers must build up knowledge

Communicate intention of Dockerfile because there is not one Dockerfile fitting all problems

UbuntuGIS easy, but huge images (try alpine!)

Packaging UI / Desktop applications still in flow

One more thing...

http://geocontainers.org

One image per week

Geocontainers is a RIP-OFF! http://biodocker.org/

https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=1&pullCount=0&q=foss4g&starCount=0 as of Aug 23 2016

Thanks for your attention!

What are your questions?

http://o2r.info @o2r_projectdaniel.nuest@wwu.de

Thanks to…Björn Schilberg (Intevation) - https://www.fossgis.de/konferenz/2015/programm/events/847.de.html and https://www.youtube.com/watch?v=CeI8S_tnCZ0Jonathan Meyer - https://2016.foss4g-na.org/session/image-mosaics-automation Sophia Parafina - https://2016.foss4g-na.org/session/spatial-data-processing-docker

Docker @ FOSS4G 2016, Bonn

Technology