A look at the Elephants TrunkA look at the Elephants Trunk

PostgreSQL 11PostgreSQL 11PostgresOpen SV 2018

San Francisco, USA

Magnus Hagander magnus@hagander.net

Magnus HaganderMagnus HaganderRedpill Linpro

Principal database consultantPostgreSQL

Core Team memberCommitterPostgreSQL Europe

PostgreSQL 11PostgreSQL 11

PostgreSQL 11PostgreSQL 11Not done yet

Almost!Some things exist already

But may be removed

Development scheduleDevelopment scheduleAugust 2017 - branch 10September 2017 - CF1November 2017 - CF2January 2018 - CF3March 2018 - CF4August 2018 - Beta3Target: Oct 2018 - release

New featuresNew featuresDBA and administrationSQL and developerBackup and replicationPerformance

WAL segment sizeWAL segment sizeconfigurableconfigurable

Change from 16MB without recompileUseful in some high-WAL environmentsOr very constrained ones

$ initdb -D /pgdata --wal-segsize=32

pg_stat_statementspg_stat_statementsqueryid is now 64-bitMuch less risk of collission(25% risk a�er 3bn instead of 50k)Possible breaking change!

Expression index statsExpression index statsSET STATISTICS can be done for expression indexDefined on columns by ordinal

CREATE INDEX coord_idx ON measured (x, y, (z + t)); ALTER INDEX coord_idx ALTER COLUMN 3 SET STATISTICS 1000;

INCLUDE indexesINCLUDE indexesAdd extra columns to indexNot in keyOnly used for index only scans

CREATE UNIQUE INDEX myidx ON mytable USING btree (id) INCLUDE (secondfield);

Automatic prewarmAutomatic prewarmpg_prewarm

Already existsAutomatically dump list

Regular intervalsDefault: 5 minutes

Automatically load on start

More default rolesMore default rolespg_read_server_filespg_write_server_filespg_execute_server_program

ALTER TABLE ADDALTER TABLE ADDCOLUMNCOLUMN

With NOT NULL DEFAULT valuesNow fast!Avoids rewriteNew rows gets materialized valueMust be non-volatile

New featuresNew featuresDBA and administrationSQL and developerBackup and replicationPerformance

websearch_to_tsquerywebsearch_to_tsqueryLike phraseto_tsquery()But less pickyMore like typical search enginesQuotes, AND/OR, and negation

Domain enhancementsDomain enhancementsARRAYs over domainsDomains over composite types

Window frame clausesWindow frame clausesNow full SQL:2011 supportRANGE BETWEEN

Previously, just ROWSNow handles values

Exclusion clausesExclude current rowExclude ties

Window frame clausesWindow frame clausespostgres=# SELECT i, SUM(i) OVER (ORDER BY i ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING), SUM(i) OVER (ORDER BY i RANGE BETWEEN 2 PRECEDING AND 2 FOLLOWING) FROM numb i | sum | sum ----+-----+----- 1 | 9 | 4 3 | 16 | 9 5 | 25 | 15 7 | 35 | 21

Window frame clausesWindow frame clausespostgres=# SELECT i, SUM(i) OVER (ORDER BY i ROWS BETWEEN 2 PRECEDING AND 2 EXCLUDE CURRENT ROW), SUM(i) OVER (ORDER BY i RANGE BETWEEN 2 PRECEDING AND 2 EXCLUDE CURRENT ROW) FROM numbers; i | sum | sum ----+-----+----- 1 | 8 | 3 3 | 13 | 6 5 | 20 | 10 7 | 28 | 14

Stored proceduresStored proceduresNot just void-returning functionsSQL standard syntax

Uses CALLTransaction control

Not just savepoints

Stored proceduresStored procedurespostgres=# CREATE PROCEDURE myproc() LANGUAGE plpgsql AS $$ BEGIN INSERT INTO mytable VALUES (1); COMMIT; INSERT INTO mytable VALUES (2); ROLLBACK; END; $$

Stored proceduresStored procedurespostgres=# CALL myproc(); CALL postgres=# SELECT * FROM mytable; a --- 1 (1 row)

New featuresNew featuresDBA and administrationSQL and developerBackup and replicationPerformance

Advance replicationAdvance replicationslotsslots

Without consumingKeep slots in sync across nodesMainly for cluster management

SELECT * FROM pg_replication_slot_advance('test_slot', '0/1678BC8')

Logical replication ofLogical replication ofTRUNCATETRUNCATE

Separately enabled in publicationPublished by default

Exclude unloggedExclude unloggedtablestables

Unlogged tables excluded from base backupsDeleted on restore anyway...(temp tables also excluded)

Validate checksumsValidate checksumsBase backups validate checksums by defaultCheap since I/O is already paid

New featuresNew featuresDBA and administrationSQL and developerBackup and replicationPerformance

ParallelismParallelism9.6 added parallelism10 made it useful11 makes it even better!

ParallelismParallelismGeneral enhancementsParallel append plan nodesParallel aware hash joins

Parallell CREATE INDEXParallell CREATE INDEXbtree indexes onlyO�en CPU bound

Much faster now!max_parallel_maintenance_workers=2

PartitioningPartitioningDeclarative partitioning in 10Syntax and basic functionality11 makeas it much more powerful!

Default partitionsDefault partitionsWhere to put rows that match no other partition

postgres=# CREATE TABLE p_def PARTITION OF p DEFAULT; CREATE TABLE

Allow UPDATE to moveAllow UPDATE to moverowsrows

Change the value in partition keyPreviously only within partitionIncluding in and out of defaultSome concurrency issues

Local partitionedLocal partitionedindexesindexes

Indexes can be created on master tableAutomatically added to partitionsBoth existing and newCan still do individual indexes too

Cross partition UNIQUECross partition UNIQUEUNIQUE indexes on parent

PRIMARY KEYMust include all partition keys(foreign keys only one way)

INSERT ON CONFLICTINSERT ON CONFLICTNow on partitioned tables

Better partitionBetter partitionpruningpruningDone in executorOnce at start

For parametersOnce at runtime

For subqueries etc

Hash partitioningHash partitioningPartition by automatic hash value

postgres=# CREATE TABLE p2(i int, t text) postgres-# PARTITION BY HASH (i); CREATE TABLE postgres=# CREATE table p2_1 PARTITION OF p2 postgres-# FOR VALUES WITH (MODULUS 4, REMAINDER 0); CREATE TABLE

Partition wise joinPartition wise joinJoin of tables on partition keyIdentical partition keyJoining on complete partition keyDefault: off

Partition wisePartition wiseaggregatesaggregatesPartition key part of GROUP BYRun aggregates per partitionSummarize at the end

Other performanceOther performance

JIT compilationJIT compilationLLVM based JIT compilation

Availability depends on packagingOptimized expresison processing

Big speedup for some analyticalE.g. large computational aggregatese

Automatically enabled for expensive queries

That's a lot!That's a lot!

There's always moreThere's always moreLots of smaller fixesPerformance improvementsetc, etcCan't mention them all!

Please help!Please help!

Please help!Please help!Download and test!

apt packages availablerpm/yum packagesavailable

Thank you!Thank you!Magnus Hagander

magnus@hagander.net @magnushagander

https://www.hagander.net/talks/

This material is licensed