Intro to PostGis
For beginners
Alban de Lavenne
17 décembre 2012
How do you organise your data ?
The problem :
lots of di�erent �les, formats : csv, txt, xls, shp, asc ...
in di�erent folders more or less well organised and easy to �nd
lack of descriptions (source, stations, projection, last version ...)
reproducible science ?
easy sharing ?
easy backup ?
Do you need a database ?
not so di�cult to built
enforce you to organize you data
save you a lot of time in the long run
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 2 / 25
How do you organise your data ?
The problem :
lots of di�erent �les, formats : csv, txt, xls, shp, asc ...
in di�erent folders more or less well organised and easy to �nd
lack of descriptions (source, stations, projection, last version ...)
reproducible science ?
easy sharing ?
easy backup ?
Do you need a database ?
not so di�cult to built
enforce you to organize you data
save you a lot of time in the long run
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 2 / 25
How do you organise your data ?
The problem :
lots of di�erent �les, formats : csv, txt, xls, shp, asc ...
in di�erent folders more or less well organised and easy to �nd
lack of descriptions (source, stations, projection, last version ...)
reproducible science ?
easy sharing ?
easy backup ?
Do you need a database ?
not so di�cult to built
enforce you to organize you data
save you a lot of time in the long run
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 2 / 25
How do you organise your data ?
The problem :
lots of di�erent �les, formats : csv, txt, xls, shp, asc ...
in di�erent folders more or less well organised and easy to �nd
lack of descriptions (source, stations, projection, last version ...)
reproducible science ?
easy sharing ?
easy backup ?
Do you need a database ?
not so di�cult to built
enforce you to organize you data
save you a lot of time in the long run
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 2 / 25
How do you organise your data ?
The problem :
lots of di�erent �les, formats : csv, txt, xls, shp, asc ...
in di�erent folders more or less well organised and easy to �nd
lack of descriptions (source, stations, projection, last version ...)
reproducible science ?
easy sharing ?
easy backup ?
Do you need a database ?
not so di�cult to built
enforce you to organize you data
save you a lot of time in the long run
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 2 / 25
How do you organise your data ?
The problem :
lots of di�erent �les, formats : csv, txt, xls, shp, asc ...
in di�erent folders more or less well organised and easy to �nd
lack of descriptions (source, stations, projection, last version ...)
reproducible science ?
easy sharing ?
easy backup ?
Do you need a database ?
not so di�cult to built
enforce you to organize you data
save you a lot of time in the long run
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 2 / 25
How do you organise your data ?
The problem :
lots of di�erent �les, formats : csv, txt, xls, shp, asc ...
in di�erent folders more or less well organised and easy to �nd
lack of descriptions (source, stations, projection, last version ...)
reproducible science ?
easy sharing ?
easy backup ?
Do you need a database ?
not so di�cult to built
enforce you to organize you data
save you a lot of time in the long run
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 2 / 25
What Is PostGIS ?
Open source object-relational database management system (ORDBMS)Widely considered as the most full-featured open-source database system.
PostGIS = database extender for the PostgreSQL Database Management System
open source, freely available, fairly OGC compliant
In a nutshell it adds spatial functions such as :
distance, area, union, intersection
specialty geometry data types to the database
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 3 / 25
Installing PostgreSQL with PostGIS Functionality
Install PostgreSQL �rst, then PostGIS extension (using Stack Builder)http://www.postgresql.org/download/
http://postgis.refractions.net/download/
Windows
http://www.postgresql.org/download/windows
Mac
http://www.kyngchaos.com/software:postgres
Ubuntu
sudo apt-get install postgresql postgis
psql -d template1 -c "alter user postgres with password 'postgres'"
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 4 / 25
Creating a database
Using pgAdminIII interface
Right clic on database
or using SQL query
CREATE DATABASE blavetWITH ENCODING='UTF8'
OWNER=postgresLC_COLLATE='fr_FR.UTF -8'LC_CTYPE='fr_FR.UTF -8'CONNECTION LIMIT=−1;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 5 / 25
Adding spatial extension
Using pgAdminIII interface
Right clic on extensions
or using SQL query
CREATE EXTENSION postgisVERSION "2.0.1"
Check the table "spatial_ref_sys"Check also that you have lots of new functions
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 6 / 25
Organising a database
How would you organise a database ? Having :
Runo� time series of di�erent basins
Rainfall time series of di�erent raingauges
Spatial description of those measurements
Other spatial informations (like soil type, geology...)
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 7 / 25
Adding tables
Table of basins
CREATE TABLE basins(
id_basin serial NOT NULL ,name text ,city text ,altitude numeric ,area numeric ,x_lambert2e numeric ,y_lambert2e numeric ,CONSTRAINT basins_pkey PRIMARY KEY ( id_basin )
)WITH (
OIDS=TRUE) ;ALTER TABLE basins
OWNER TO postgres ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 8 / 25
Adding tables
Table of measures of runo�
CREATE TABLE measures_runoff(
id_runoff serial NOT NULL ,id_basin integer NOT NULL ,datehourmeasure timestamp with time zone NOT NULL ,runoff numeric ,timestep text ,unit text ,source text ,CONSTRAINT measures_runoff_pkey PRIMARY KEY ( id_runoff ,
id_basin , datehourmeasure ))WITH (
OIDS=FALSE) ;ALTER TABLE measures_runoff
OWNER TO postgres ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 9 / 25
Adding tables
Table of raingauges
CREATE TABLE raingauges(
id_raingauge serial ,id_meteofrance integer ,city text ,name_raingauge text ,altitude numeric ,type_raingauge text ,producer text ,latitude_dec numeric ,longitude_dec numeric ,CONSTRAINT raingauge_pkey PRIMARY KEY ( id_raingauge )
)WITH (
OIDS=TRUE) ;ALTER TABLE raingauges
OWNER TO postgres ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 10 / 25
Adding tables
Table of measures of rainfall
CREATE TABLE measures_rainfall(
id_rainfall serial ,id_raingauge integer ,datehourmeasure timestamp with time zone ,rainfall numeric ,timestep text ,unit text ,CONSTRAINT rainfall_pkey PRIMARY KEY ( id_rainfall ,
id_raingauge , datehourmeasure ))WITH (
OIDS=FALSE) ;ALTER TABLE measures_rainfall
OWNER TO postgres ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 11 / 25
Adding tables
Table of antilope points
CREATE TABLE antilope_points(
id_point numeric NOT NULL ,x_lambert2e numeric ,y_lambert2e numeric ,CONSTRAINT antilope_points_pkey PRIMARY KEY ( id_point )
)WITH (
OIDS=TRUE) ;ALTER TABLE antilope_points
OWNER TO postgres ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 12 / 25
Adding tables
Table of measures of antilope points
CREATE TABLE measures_rainfall_antilope(
id_measure serial ,id_point integer ,datehourmeasure timestamp with time zone ,rainfall numeric ,units text ,CONSTRAINT measures_rainfall_antilope_pkey PRIMARY KEY (
id_measure ))WITH (
OIDS=FALSE) ;ALTER TABLE measures_rainfall_antilope
OWNER TO postgres ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 13 / 25
Filling tables
Import �les to database
COPY basins ( id_basin , name , city , altitude , x_lambert2e , y_lambert2e )FROM '.../ Intro_to_PostGIS/basins.csv' with delimiter ';' ;
COPY measures_runoff ( id_basin , datehourmeasure , runoff , timestep ,unit ) FROM '.../ Intro_to_PostGIS/runoff.csv' with delimiter ';' ;
COPY raingauges ( id_raingauge , id_meteofrance , city , name_raingauge ,altitude , latitude_dms , longitude_dms , type_raingauge , producer ,latitude_dec , longitude_dec ) FROM '.../ Intro_to_PostGIS/raingauges.csv' with delimiter ';' ;
COPY measures_rainfall ( id_raingauge , datehourmeasure , rainfall ,timestep , unit ) FROM '.../ Intro_to_PostGIS/rainfall.csv' withdelimiter ';' ;
COPY antilope_points ( id_point , x_lambert2e , y_lambert2e ) FROM '.../Intro_to_PostGIS/antilope_points.csv' with delimiter ';' ;
COPY measures_rainfall_antilope ( id_point , datehourmeasure , rainfall, units ) FROM '.../ Intro_to_PostGIS/antilope_rainfall.csv'with delimiter ';' ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 14 / 25
Adding internal spatial information
AddGeometryColumn
AddGeometryColumn ( schema_name , table_name , column_name , srid ,type , dimension )
(Source : JUMP, Technical Report)
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 15 / 25
Adding internal spatial information
(Source : OpenGIS Simple Features Speci�cation for SQL)
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 16 / 25
Geometry Constructors and Geometry Accessors
ST_MakePoint
ST_MakePoint ( x , y , z )
Full list here : Geometry Constructors
ST_X, ST_Y and ST_Z
ST_X ( geometry )ST_Y ( geometry )ST_Z ( geometry )
Full list here : Geometry Accessors
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 17 / 25
Geometry Editors
ST_SetSRID
ST_SetSRID ( geom , srid )
ST_Transform
ST_Transform ( geom , srid )
Full list here : Geometry Editors
A Spatial Reference System Identi�er (SRID) is a unique value used to unambiguouslyidentify projected, unprojected, and local spatial coordinate system de�nitions.
Large database of SRID created by the European Petroleum Survey Group (EPSG) iswidely used.
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 18 / 25
Spatial Relationships and Measurements
Spatial Relationships and Measurements
ST_Area ( geometry )ST_Centroid ( geometry )ST_Contains ( geometry , geometry )ST_Distance ( geometry , geometry )
Full list here : Spatial Relationships and Measurements
Geometry Processing
ST_Buffer ( geometry , radius_of_buffer )ST_Union ( geometry set g1field )ST_Union ( geometry , geometry )ST_Intersection ( geometry , geometry )
Full list here : Geometry Processing
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 19 / 25
Adding internal spatial information
Adding geometry columns
−−Adding geometry columnsSELECT AddGeometryColumn ( 'basins ' , 'outlet_lambert2e ' , 27572 , '
POINT' , 2 ) ;SELECT AddGeometryColumn ( 'basins ' , 'outlet_lambert93 ' , 2154 , '
POINT' , 2 ) ;SELECT AddGeometryColumn ( 'basins ' , 'limits_lambert2e ' , 27572 , '
POLYGON ' , 2 ) ;SELECT AddGeometryColumn ( 'basins ' , 'limits_lambert93 ' , 2154 , '
POLYGON ' , 2 ) ;SELECT AddGeometryColumn ( 'raingauges ' , 'raingauges_lambert2e ' ,
27572 , 'POINT' , 2 ) ;SELECT AddGeometryColumn ( 'raingauges ' , 'raingauges_lambert93 ' ,
2154 , 'POINT' , 2 ) ;SELECT AddGeometryColumn ( 'antilope_points ' , '
antilope_points_lambert2e ' , 27572 , 'POINT' , 2 ) ;SELECT AddGeometryColumn ( 'antilope_points ' , '
antilope_points_lambert93 ' , 2154 , 'POINT' , 2 ) ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 20 / 25
Adding internal spatial information
Filling geometry columns
−−F i l l geometry columnsUPDATE basins SET outlet_lambert2e=st_setsrid ( st_makepoint (
x_lambert2e , y_lambert2e ) , 27572) ;UPDATE basins SET outlet_lambert93=st_transform ( outlet_lambert2e
, 2154 ) ;UPDATE raingauges SET raingauges_lambert2e=st_transform (
st_setsrid ( st_makepoint ( longitude_dec , latitude_dec ) , 4326),27572) ;
UPDATE raingauges SET raingauges_lambert93=st_transform (st_setsrid ( st_makepoint ( longitude_dec , latitude_dec ) , 4326),2154) ;
UPDATE antilope_points SET antilope_points_lambert2e=st_setsrid (st_makepoint ( x_lambert2e , y_lambert2e ) , 27572) ;
UPDATE antilope_points SET antilope_points_lambert93=st_transform( st_setsrid ( st_makepoint ( x_lambert2e , y_lambert2e ) , 27572),2154) ;
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 21 / 25
Adding external spatial information
Using graphical interface
Using Quantum-GIS extension (like QGIS spit)pgShapeLoader...
Using command line with shp2pgsql
shp2pgsql -iIDd -s 2154 -W utf8 basins_lambert93.shp basins_polygon > import.sql
psql -U postgres -W -h localhost -p 5432 -d blavet -f import.sql
OR
shp2pgsql -iIDd -s 2154 -W utf8 basins_lambert93.shp basins_polygon |
psql -U postgres -W -h localhost -p 5432 -d blavet
Filling geometry columns
UPDATE basins SET limits_lambert93=st_geometryn ( geom , 1 )FROM basins_polygonWHERE basins_polygon . id_basin=basins . id_basin ;
and limits_lambert2e ...
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 22 / 25
Exemple SQL of query
Select a chronique of runo�Calculate the area of one basinSelect raingauges inside one basinSelect average antilope rainfall time serie for each basinsImport shape�le geology and get a table of statistics of geology for each basinCheck if there is double measures in runo� and rainfall
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 23 / 25
Visualizing your data
Using Qgis to vizualize your dataPlugin in Qgis for query and shape�le creationPlugin in Qgis for importing data
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 24 / 25
Connect to your database with R
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 25 / 25
Connect to your database with R
A. de Lavenne (INRA, Rennes) University of Trento 17 décembre 2012 25 / 25