+ All Categories
Home > Documents > Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since...

Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since...

Date post: 24-Jul-2020
Category:
Upload: others
View: 18 times
Download: 0 times
Share this document with a friend
195
Neues in Open-Source-SQL-Datenbanken @MarkusWinand • @ModernSQL http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf
Transcript
Page 1: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Neues inOpen-Source-SQL-Datenbanken

@MarkusWinand • @ModernSQL

http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

Page 2: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Neues inOpen-Source-SQL-Datenbanken

@MarkusWinand • @ModernSQL

http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

Page 3: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM
Page 4: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SQL-92

Page 5: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CHECKConstraints

Page 6: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CHECK Constraints Since SQL-92CREATETABLEorder_lines(…qtyINTEGERNOTNULLCHECK(qty>0),…)

Page 7: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CHECK Constraints Since SQL-92CREATETABLEorder_lines(…qtyINTEGERNOTNULLCHECK(qty>0),…)

INSERT…(…,qty,…)VALUES(…,1,…)

Page 8: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CHECK Constraints Since SQL-92CREATETABLEorder_lines(…qtyINTEGERNOTNULLCHECK(qty>0),…)

INSERT…(…,qty,…)VALUES(…,1,…)INSERT…(…,qty,…)VALUES(…,3,…)

Page 9: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CHECK Constraints Since SQL-92CREATETABLEorder_lines(…qtyINTEGERNOTNULLCHECK(qty>0),…)

INSERT…(…,qty,…)VALUES(…,1,…)INSERT…(…,qty,…)VALUES(…,3,…)INSERT…(…,qty,…)VALUES(…,0,…)

Page 10: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CHECK Constraints Since SQL-92CREATETABLEorder_lines(…qtyINTEGERNOTNULLCHECK(qty>0),…)

INSERT…(…,qty,…)VALUES(…,1,…)INSERT…(…,qty,…)VALUES(…,3,…)INSERT…(…,qty,…)VALUES(…,0,…)

Before MySQL 8.0.16and MariaDB 10.2:

Syntax accepted,Constraint ignored

Page 11: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CHECK Constraints Since SQL-921999

2001

2003

2005

2007

2009

2011

2013

2015

2017

10.2 MariaDB8.0.16 MySQL

8.3 PostgreSQL3.5.7 SQLite

9.7 DB2 LUW11gR1 Oracle

2008R2 SQL Server

Page 12: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

INTERSECT and EXCEPT

Page 13: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Since SQL-92INTERSECT & EXCEPT

UNION[ALL]

Concatenatestwo results

Page 14: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Since SQL-92INTERSECT & EXCEPT

UNION[ALL]

Concatenatestwo results

INTERSECT[ALL]

Common rows from two results

Page 15: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Since SQL-92INTERSECT & EXCEPT

UNION[ALL]

Concatenatestwo results

INTERSECT[ALL]

Common rows from two results

EXCEPT[ALL]

Remove rowsfrom first result

Page 16: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

INTERSECT & EXCEPT Since SQL-921999

2001

2003

2005

2007

2009

2011

2013

2015

2017

10.3[0] MariaDBMySQL

8.3 PostgreSQL3.5.7[0] SQLite

9.7 DB2 LUW11gR1[0] Oracle

2008R2[0] SQL Server[0]Not [all]

Page 17: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM
Page 18: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SQL:1999

Page 19: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

LATERAL

Page 20: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Select-list sub-queries must be scalar[0]:

LATERAL Before SQL:1999

SELECT…,(SELECTcolumn_1FROMt1WHEREt1.x=t2.y)AScFROMt2…

(an atomic quantity that can hold only one value at a time[1])

[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar

Page 21: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Select-list sub-queries must be scalar[0]:

LATERAL Before SQL:1999

SELECT…,(SELECTcolumn_1FROMt1WHEREt1.x=t2.y)AScFROMt2…

(an atomic quantity that can hold only one value at a time[1])

[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar

✗,column_2

More thanone column? ⇒Syntax error

Page 22: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Select-list sub-queries must be scalar[0]:

LATERAL Before SQL:1999

SELECT…,(SELECTcolumn_1FROMt1WHEREt1.x=t2.y)AScFROMt2…

(an atomic quantity that can hold only one value at a time[1])

[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar

✗,column_2

More thanone column? ⇒Syntax error

}More thanone row?

⇒Runtime error!

Page 23: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMt1JOIN(SELECT*FROMt2WHEREt2.x=t1.x)derived_tableON(derived_table.x=t1.x)

Derived tables (from clause subqueries) cannot look outside:

LATERAL Before SQL:1999

Page 24: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMt1JOIN(SELECT*FROMt2WHEREt2.x=t1.x)derived_tableON(derived_table.x=t1.x)

Derived tables (from clause subqueries) cannot look outside:

LATERAL Before SQL:1999

Invalid

Page 25: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMt1JOIN(SELECT*FROMt2WHEREt2.x=t1.x)derived_tableON(derived_table.x=t1.x)

Derived tables (from clause subqueries) cannot look outside:

LATERAL Before SQL:1999

Belongs

there

Page 26: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMt1CROSSJOINLATERAL(SELECT*FROMt2WHEREt2.x=t1.x)derived_tableON(true)

LATERAL Since SQL:1999Lateral derived queries can see table names defined before:

Page 27: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMt1CROSSJOINLATERAL(SELECT*FROMt2WHEREt2.x=t1.x)derived_tableON(true)

LATERAL Since SQL:1999

Valid due to

LATERAL

keyword

Lateral derived queries can see table names defined before:

Page 28: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMt1CROSSJOINLATERAL(SELECT*FROMt2WHEREt2.x=t1.x)derived_tableON(true)

LATERAL Since SQL:1999

Valid due to

LATERAL

keyword

Useless, but still required

Lateral derived queries can see table names defined before:

Page 29: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMt1CROSSJOINLATERAL(SELECT*FROMt2WHEREt2.x=t1.x)derived_tableON(true)

LATERAL Since SQL:1999

Valid due to

LATERAL

keyword

Lateral derived queries can see table names defined before:

Use CROSS JOIN to omit the ON clause

Page 30: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

But WHY?

Page 31: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Use-CasesLATERAL

Page 32: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

‣ Top-N per group

inside a lateral derived tableFETCHFIRST (or LIMIT, TOP)applies per row from left tables.

Use-CasesLATERAL

Page 33: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

FROMtCROSSJOINLATERAL(SELECT…FROM…WHEREt.c=…ORDERBY…LIMIT10)derived_table

‣ Top-N per group

inside a lateral derived tableFETCHFIRST (or LIMIT, TOP)applies per row from left tables.

Use-CasesLATERAL

Page 34: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

FROMtCROSSJOINLATERAL(SELECT…FROM…WHEREt.c=…ORDERBY…LIMIT10)derived_table

‣ Top-N per group

inside a lateral derived tableFETCHFIRST (or LIMIT, TOP)applies per row from left tables.

Use-CasesLATERAL

Add proper indexfor Top-N query

https://use-the-index-luke.com/sql/partial-results/top-n-queries

Page 35: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

FROMtCROSSJOINLATERAL(SELECT…FROM…WHEREt.c=…ORDERBY…LIMIT10)derived_table

‣ Top-N per group

inside a lateral derived tableFETCHFIRST (or LIMIT, TOP)applies per row from left tables.

‣ Also useful to find most recent news from several subscribed topics (“multi-source top-N”).

Use-CasesLATERAL

Page 36: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

FROMtCROSSJOINLATERAL(SELECT…FROM…WHEREt.c=…ORDERBY…LIMIT10)derived_table

‣ Top-N per group

inside a lateral derived tableFETCHFIRST (or LIMIT, TOP)applies per row from left tables.

‣ Also useful to find most recent news from several subscribed topics (“multi-source top-N”).

‣ Table function arguments

(TABLE often implies LATERAL)

Use-CasesLATERAL

FROMtJOINTABLE(your_func(t.c))

Page 37: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

LATERAL Availability1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 MariaDB8.0.14 MySQL

9.3 PostgreSQLSQLite

9.1 DB2 LUW11gR1[0] 12cR1 Oracle

2005[1] SQL Server[0]Undocumented. Requires setting trace event 22829.[1]LATERAL is not supported as of SQL Server 2016 but [CROSS|OUTER]APPLY can be used for the same effect.

Page 38: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

WITH(Common Table Expressions)

Page 39: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

WITH (non-recursive) The ProblemNested queries are hard to read:

SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)

Page 40: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Understand

this first

WITH (non-recursive) The ProblemNested queries are hard to read:

SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)

Page 41: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Then this...

WITH (non-recursive) The ProblemNested queries are hard to read:

SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)

Page 42: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Then this...

WITH (non-recursive) The ProblemNested queries are hard to read:

SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)

Page 43: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Finally the first line makes sense

WITH (non-recursive) The ProblemNested queries are hard to read:

SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)

Page 44: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)

WITH (non-recursive) Since SQL:1999

Page 45: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)

Keyword

WITH (non-recursive) Since SQL:1999

Page 46: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)

Name of CTE and (here optional) column names

WITH (non-recursive) Since SQL:1999

Page 47: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)

Definition

WITH (non-recursive) Since SQL:1999

Page 48: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)

Introduces another CTE

Don't repeat WITH

WITH (non-recursive) Since SQL:1999

Page 49: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)

WITH (non-recursive) Since SQL:1999

Page 50: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)

May refer toprevious CTEs

WITH (non-recursive) Since SQL:1999

Page 51: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)AS(SELECT…FROM…)

SELECT…FROMbJOINcON(…)

Third CTE

WITH (non-recursive) Since SQL:1999

Page 52: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)AS(SELECT…FROM…)

SELECT…FROMbJOINcON(…)

No comma!

WITH (non-recursive) Since SQL:1999

Page 53: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)AS(SELECT…FROM…)

SELECT…FROMbJOINcON(…)

Main query

WITH (non-recursive) Since SQL:1999

Page 54: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEs are statement-scoped views:

WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),

b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),

c(…)AS(SELECT…FROM…)

SELECT…FROMbJOINcON(…)

Read top down

WITH (non-recursive) Since SQL:1999

Page 55: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

PostgreSQL “issues”WITH (non-recursive)

In pre 12 PostgreSQL WITH queries are “optimizer fences”:

WITHcteAS(SELECT*FROMnews)SELECT*FROMcteWHEREtopic=1

Page 56: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEScanoncte(rows=6370)Filter:topic=1CTEcte->SeqScanonnews(rows=10000001)

PostgreSQL “issues”WITH (non-recursive)

In pre 12 PostgreSQL WITH queries are “optimizer fences”:

WITHcteAS(SELECT*FROMnews)SELECT*FROMcteWHEREtopic=1

Page 57: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEScanoncte(rows=6370)Filter:topic=1CTEcte->SeqScanonnews(rows=10000001)

PostgreSQL “issues”WITH (non-recursive)

In pre 12 PostgreSQL WITH queries are “optimizer fences”:

WITHcteAS(SELECT*FROMnews)SELECT*FROMcteWHEREtopic=1

Page 58: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CTEScanoncte(rows=6370)Filter:topic=1CTEcte->SeqScanonnews(rows=10000001)

PostgreSQL “issues”WITH (non-recursive)

CTE doesn't

know about the outer

filter

In pre 12 PostgreSQL WITH queries are “optimizer fences”:

WITHcteAS(SELECT*FROMnews)SELECT*FROMcteWHEREtopic=1

Page 59: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Views and derived tables support "predicate pushdown":SELECT*FROM(SELECT*FROMnews)nWHEREtopic=1;

PostgreSQL “issues”WITH (non-recursive)

Page 60: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Views and derived tables support "predicate pushdown":SELECT*FROM(SELECT*FROMnews)nWHEREtopic=1;

BitmapHeapScanonnews(rows=6370)->BitmapIndexScanonidx(rows=6370)Cond:topic=1

PostgreSQL “issues”WITH (non-recursive)

Page 61: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Views and derived tables support "predicate pushdown":SELECT*FROM(SELECT*FROMnews)nWHEREtopic=1;

BitmapHeapScanonnews(rows=6370)->BitmapIndexScanonidx(rows=6370)Cond:topic=1

PostgreSQL “issues”WITH (non-recursive)

Page 62: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

PostgreSQL 12 inlines non-recursive CTEs if it isreferred to only once.

Use [NOT]MATERIALIZED to force it (not SQL standard).

WITHcteAS[NOT]MATERIALIZED(SELECT*FROMnews)SELECT*FROMcteWHEREtopic=1

PostgreSQL “issues”WITH (non-recursive)

Page 63: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

AvailabilityWITH (non-recursive)1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.2 MariaDB8.0 MySQL

8.4 PostgreSQL3.8.3[0] SQLite

7.0 DB2 LUW9iR2 Oracle

2005[1] SQL Server[0]Only for top-level SELECT statements[1]Only allowed at the very begin of a statement. E.g. WITH...INSERT...SELECT.

Page 64: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SQL:2003

Page 65: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

BOOLEANTests

Page 66: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Before we start

SQL uses a three-valued logic. Boolean values are either

true, false or unknown(=null).

See: http://modern-sql.com/concept/three-valued-logic

BOOLEANAggregates

Page 67: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

BOOLEANTests

Similar to isnull, there are tests for each Boolean value(of which there are three: true, false, unknown/null)

IS[NOT][TRUE|FALSE|UNKNOWN]

Since SQL:2003

Page 68: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEprices(…valid_fromDATENOTNULL,valid_toDATE,--null:openend…CHECK(valid_from<valid_to),);

WHEREvalid_from<CURRENT_DATEAND(valid_to<=CURRENT_DATE)ISNOTFALSE

BOOLEANTests Since SQL:2003

Page 69: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEprices(…valid_fromDATENOTNULL,valid_toDATE,--null:openend…CHECK(valid_from<valid_to),);

WHEREvalid_from<CURRENT_DATEAND(valid_to<=CURRENT_DATE)ISNOTFALSE

BOOLEANTests Since SQL:2003

Page 70: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEprices(…valid_fromDATENOTNULL,valid_toDATE,--null:openend…CHECK(valid_from<valid_to),);

WHEREvalid_from<CURRENT_DATEAND(valid_to<=CURRENT_DATE)ISNOTFALSE

BOOLEANTests Since SQL:2003

UNKNOWN ifVALID_TO is NULL

Page 71: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEprices(…valid_fromDATENOTNULL,valid_toDATE,--null:openend…CHECK(valid_from<valid_to),);

WHEREvalid_from<CURRENT_DATEAND(valid_to<=CURRENT_DATE)ISNOTFALSE

BOOLEANTests Since SQL:2003

UNKNOWN ifVALID_TO is NULL Takes TRUE and

UNKNOWN

Page 72: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

BOOLEANTests Since SQL:20031999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 MariaDB5.0.51a MySQL8.3 PostgreSQL

3.23.0[0] SQLite

DB2 LUWOracleSQL Server

[0]No IS [NOT] UNKNOWN. Use IS [NOT] NULL instead.

Page 73: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVERand

PARTITIONBY

Page 74: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (PARTITION BY) The ProblemTwo distinct concepts could not be used independently:

Page 75: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (PARTITION BY) The ProblemTwo distinct concepts could not be used independently:

‣Merge rows with the same key properties

‣ GROUPBY to specify key properties

‣ DISTINCT to use full row as key

Page 76: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (PARTITION BY) The ProblemTwo distinct concepts could not be used independently:

‣Merge rows with the same key properties

‣ GROUPBY to specify key properties

‣ DISTINCT to use full row as key

‣ Aggregate data from related rows ‣ Requires GROUPBY to segregate the rows

‣ COUNT, SUM, AVG, MIN, MAX to aggregate grouped rows

Page 77: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (PARTITION BY) The Problem

Page 78: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (PARTITION BY) The Problem

SELECTc1,c2FROMt

SELECTc1,c2FROMt

Page 79: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No SELECTc1

,c2FROMt

SELECTc1,c2FROMt

Page 80: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No SELECTc1

,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMt

Page 81: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMt

Page 82: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMt

SELECTc1,SUM(c2)totFROMtGROUPBYc1

Page 83: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMt

SELECTc1,SUM(c2)totFROMtGROUPBYc1

Page 84: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMtJOIN()taON(t.c1=ta.c1)

SELECTc1,SUM(c2)totFROMtGROUPBYc1

Page 85: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMtJOIN()taON(t.c1=ta.c1)

SELECTc1,SUM(c2)totFROMtGROUPBYc1

Page 86: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMtJOIN()taON(t.c1=ta.c1)

SELECTc1,SUM(c2)totFROMtGROUPBYc1

,tot

Page 87: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) The Problem

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMtJOIN()taON(t.c1=ta.c1)

SELECTc1,SUM(c2)totFROMtGROUPBYc1

,tot

Page 88: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) Since SQL:2003

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMtFROMt

Page 89: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTc1,SUM(c2)totFROMtGROUPBYc1

OVER (PARTITION BY) Since SQL:2003

Yes ⇠

Mer

ge ro

ws ⇢

No

No ⇠ Aggregate ⇢ Yes

SELECTc1,c2FROMt

SELECTDISTINCTc1,c2FROMt

SELECTc1,c2FROMt

FROMt

,SUM(c2)OVER(PARTITIONBYc1)

Page 90: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000

OVER (PARTITION BY) How it works

Page 91: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000

OVER (PARTITION BY) How it works

Look here

Page 92: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000

OVER (PARTITION BY) How it works

Page 93: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000

OVER (PARTITION BY) How it works

Page 94: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000

OVER (PARTITION BY) How it works

Page 95: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000

OVER (PARTITION BY) How it works

Page 96: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

dep salary1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000

SELECTdep,salary,SUM(salary)OVER()FROMemp

OVER (PARTITION BY) How it works

)

Page 97: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 100022 1000 200022 1000 2000333 1000 3000333 1000 3000333 1000 3000

OVER (PARTITION BY) How it works

)

Page 98: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 100022 1000 200022 1000 2000333 1000 3000333 1000 3000333 1000 3000

OVER (PARTITION BY) How it works

)PARTITIONBYdep

Page 99: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 100022 1000 200022 1000 2000333 1000 3000333 1000 3000333 1000 3000

OVER (PARTITION BY) How it works

)PARTITIONBYdep

Page 100: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECTdep,salary,SUM(salary)OVER()FROMemp

dep salary1 1000 100022 1000 200022 1000 2000333 1000 3000333 1000 3000333 1000 3000

OVER (PARTITION BY) How it works

)PARTITIONBYdep

Page 101: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVERand

ORDERBY(Framing & Ranking)

Page 102: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

OVER (ORDER BY) The Problem

SELECTid,value,FROMtransactionst

Page 103: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

OVER (ORDER BY) The Problem

SELECTid,value,FROMtransactionst

Page 104: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

OVER (ORDER BY) The Problem

SELECTid,value,FROMtransactionst

Page 105: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

OVER (ORDER BY) The Problem

SELECTid,value,

(SELECTSUM(value)FROMtransactionst2WHEREt2.id<=t.id)

FROMtransactionst

Page 106: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

OVER (ORDER BY) The Problem

SELECTid,value,

(SELECTSUM(value)FROMtransactionst2WHEREt2.id<=t.id)

FROMtransactionst

Range segregation (<=)not possible with

GROUP BY orPARTITION BY

Page 107: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

(SELECTSUM(value)FROMtransactionst2WHEREt2.id<=t.id)

FROMtransactionst

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

Page 108: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

Page 109: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

Page 110: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYid

Page 111: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDING

Page 112: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 113: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 114: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 115: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 116: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 117: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 118: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 119: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +30

22 3 -10 +20

333 4 +50 +70

333 5 -30 +40

333 6 -20 +20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 120: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10

22 2 +20

22 3 -10

333 4 +50

333 5 -30

333 6 -20

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

Page 121: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003

SELECTid,value,

FROMtransactionst

SUM(value)OVER(

)

acnt id value balance

1 1 +10 +10

22 2 +20 +20

22 3 -10 +10

333 4 +50 +50

333 5 -30 +20

333 6 -20 .0

ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW

PARTITIONBYacnt

Page 122: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (ORDER BY) Since SQL:2003With OVER(ORDERBYn) a new type of functions make sense:

n ROW_NUMBER RANK DENSE_RANK PERCENT_RANK CUME_DISTA 1 1 1 0 0.25B 2 2 2 0.33… 0.75B 3 2 2 0.33… 0.75X 4 4 3 1 1

Page 123: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

‣ Aggregates without GROUPBY

Use CasesOVER (SQL:2003)

Page 124: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

‣ Aggregates without GROUPBY

‣ Running totals, moving averages

Use Cases

AVG(…)OVER(ORDERBY…ROWSBETWEEN3PRECEDINGAND3FOLLOWING)moving_avg

OVER (SQL:2003)

Page 125: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

‣ Aggregates without GROUPBY

‣ Running totals, moving averages

‣ Ranking

Use Cases

AVG(…)OVER(ORDERBY…ROWSBETWEEN3PRECEDINGAND3FOLLOWING)moving_avg

OVER (SQL:2003)

Page 126: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

‣ Aggregates without GROUPBY

‣ Running totals, moving averages

‣ Ranking‣ Top-N per Group

Use Cases

SELECT*FROM(SELECTROW_NUMBER()OVER(PARTITIONBY…ORDERBY…)rn,t.*FROMt)numbered_tWHERErn<=3

AVG(…)OVER(ORDERBY…ROWSBETWEEN3PRECEDINGAND3FOLLOWING)moving_avg

OVER (SQL:2003)

Page 127: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

‣ Aggregates without GROUPBY

‣ Running totals, moving averages

‣ Ranking‣ Top-N per Group

‣ Avoiding self-joins

Use Cases

SELECT*FROM(SELECTROW_NUMBER()OVER(PARTITIONBY…ORDERBY…)rn,t.*FROMt)numbered_tWHERErn<=3

AVG(…)OVER(ORDERBY…ROWSBETWEEN3PRECEDINGAND3FOLLOWING)moving_avg

OVER (SQL:2003)

Page 128: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

‣ Aggregates without GROUPBY

‣ Running totals, moving averages

‣ Ranking‣ Top-N per Group

‣ Avoiding self-joins

[… many more …]

Use Cases

SELECT*FROM(SELECTROW_NUMBER()OVER(PARTITIONBY…ORDERBY…)rn,t.*FROMt)numbered_tWHERErn<=3

AVG(…)OVER(ORDERBY…ROWSBETWEEN3PRECEDINGAND3FOLLOWING)moving_avg

OVER (SQL:2003)

Page 129: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.2 MariaDB8.0 MySQL

8.4 PostgreSQL3.25.0 SQLite

7.0 DB2 LUW8i Oracle

2005[0] 2012 SQL Server[0]No framing

OVER (SQL:2003) Availability

Page 130: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.2 MariaDB8.0 MySQL

8.4 PostgreSQL3.25.0 SQLite

7.0 DB2 LUW8i Oracle

2005[0] 2012 SQL Server[0]No framing

OVER (SQL:2003) AvailabilityImpalaSpark

NuoDB

BigQuery Hive

Page 131: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVERSQL:2003 frame exclusion

Page 132: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Since SQL:2003OVER(ORDERBY…BETWEEN…exclude[noothers|currentrow|group|ties])

OVER (frame exclusion)

Page 133: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Since SQL:2003OVER(ORDERBY…BETWEEN…exclude[noothers|currentrow|group|ties])

default

OVER (frame exclusion)

noothers

Page 134: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Since SQL:2003

x12223

OVER(ORDERBY…BETWEEN…exclude[noothers|currentrow|group|ties])

default

OVER (frame exclusion)

noothers

Page 135: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

currentrow

Since SQL:2003

x12223

OVER(ORDERBY…BETWEEN…exclude[noothers|currentrow|group|ties])

default

OVER (frame exclusion)

currentrow

Page 136: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

groupx=current_row

currentrow

Since SQL:2003

x12223

OVER(ORDERBY…BETWEEN…exclude[noothers|currentrow|group|ties])

default

OVER (frame exclusion)

group

Page 137: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

tiesgroupx=current_row

currentrow

Since SQL:2003

x12223

OVER(ORDERBY…BETWEEN…exclude[noothers|currentrow|group|ties])

default

OVER (frame exclusion)

ties

Page 138: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (frame exclusion) Since SQL:20111999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 MariaDBMySQL

11 PostgreSQL3.28.0 SQLite

DB2 LUWOracleSQL Server

Page 139: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SQL:2011

Page 140: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVERSQL:2011 groups option

Page 141: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (groups option) Since SQL:2011ORDERBYx<frameunit>between1precedingand1following

Page 142: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (groups option) Since SQL:2011ORDERBYx<frameunit>between1precedingand1following

rows,range

Page 143: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (groups option) Since SQL:2011ORDERBYx<frameunit>between1precedingand1following

rows,range

x13

3.53.54

CURRENT ROW

Page 144: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (groups option) Since SQL:2011ORDERBYx<frameunit>between1precedingand1following

rows,range

rowscount(*)

x13

3.53.54

CURRENT ROW

Page 145: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (groups option) Since SQL:2011ORDERBYx<frameunit>between1precedingand1following

rows,range

rangexbetweencurrent_row-1

andcurrent_row+1

rowscount(*)

x13

3.53.54

CURRENT ROW

Page 146: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

OVER (groups option) Since SQL:2011ORDERBYx<frameunit>between1precedingand1following

rows,rangeNew in

SQL:2011 groups

rangexbetweencurrent_row-1

andcurrent_row+1

rowscount(*)

x13

3.53.54

CURRENT ROW

Page 147: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

groupscount(distinctx)

OVER (groups option) Since SQL:2011ORDERBYx<frameunit>between1precedingand1following

rows,rangeNew in

SQL:2011 groups

rangexbetweencurrent_row-1

andcurrent_row+1

rowscount(*)

x13

3.53.54

CURRENT ROW

Page 148: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Since SQL:2011OVER (groups option)1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 MariaDBMySQL

11 PostgreSQL3.28.0 SQLite

DB2 LUWOracleSQL Server

Page 149: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

System Versioning (Time Traveling)

Page 150: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

INSERTUPDATEDELETE

are DESTRUCTIVE

System Versioning The Problem

Page 151: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

System Versioning Since SQL:2011Table can be system versioned, application versioned or both.

Page 152: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,

System Versioning Since SQL:2011Table can be system versioned, application versioned or both.

Page 153: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,start_tsTIMESTAMP(9)GENERATEDALWAYSASROWSTART,

System Versioning Since SQL:2011Table can be system versioned, application versioned or both.

Page 154: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,start_tsTIMESTAMP(9)GENERATEDALWAYSASROWSTART,end_tsTIMESTAMP(9)GENERATEDALWAYSASROWEND,

System Versioning Since SQL:2011Table can be system versioned, application versioned or both.

Page 155: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,start_tsTIMESTAMP(9)GENERATEDALWAYSASROWSTART,end_tsTIMESTAMP(9)GENERATEDALWAYSASROWEND,

PERIODFORSYSTEM_TIME(start_ts,end_ts)

System Versioning Since SQL:2011Table can be system versioned, application versioned or both.

Page 156: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,start_tsTIMESTAMP(9)GENERATEDALWAYSASROWSTART,end_tsTIMESTAMP(9)GENERATEDALWAYSASROWEND,

PERIODFORSYSTEM_TIME(start_ts,end_ts))WITHSYSTEMVERSIONING

System Versioning Since SQL:2011Table can be system versioned, application versioned or both.

Page 157: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

ID Data start_ts end_ts1 X 10:00:00

UPDATE...SETDATA='Y'...

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00

DELETE...WHEREID=1

INSERT...(ID,DATA)VALUES(1,'X')

System Versioning Since SQL:2011

Page 158: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

ID Data start_ts end_ts1 X 10:00:00

UPDATE...SETDATA='Y'...

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00

DELETE...WHEREID=1

INSERT...(ID,DATA)VALUES(1,'X')

System Versioning Since SQL:2011

Page 159: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

ID Data start_ts end_ts1 X 10:00:00

UPDATE...SETDATA='Y'...

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00

DELETE...WHEREID=1

INSERT...(ID,DATA)VALUES(1,'X')

System Versioning Since SQL:2011

Page 160: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

ID Data start_ts end_ts1 X 10:00:00

UPDATE...SETDATA='Y'...

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00

DELETE...WHEREID=1

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00 12:00:00

INSERT...(ID,DATA)VALUES(1,'X')

System Versioning Since SQL:2011

Page 161: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Although multiple versions exist, only the “current” one is visible per default.

After 12:00:00, SELECT*FROMt doesn’t return anything anymore.

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00 12:00:00

System Versioning Since SQL:2011

Page 162: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00 12:00:00

With FOR…ASOF you can query anything you like: SELECT*FROMtFORSYSTEM_TIMEASOFTIMESTAMP'2019-08-1010:30:00'

ID Data start_ts end_ts

1 X 10:00:00 11:00:00

System Versioning Since SQL:2011

Page 163: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

System Versioning Since SQL:20111999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.3 MariaDBMySQLPostgreSQLSQLite

10.1 DB2 LUW10gR1[0] 11gR1[1] Oracle

2016 SQL Server[0]Short term using Flashback.[1]Flashback Archive. Proprietery syntax.

Page 164: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

FORPORTIONOF(UPDATE,DELETE)

Page 165: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,

Since SQL:2011Table can be system versioned, application versioned or both.

FOR PORTION OF

Page 166: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,start_tsTIMESTAMP(9),

Since SQL:2011Table can be system versioned, application versioned or both.

FOR PORTION OF

Page 167: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,start_tsTIMESTAMP(9),end_tsTIMESTAMP(9),

Since SQL:2011Table can be system versioned, application versioned or both.

FOR PORTION OF

Page 168: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

CREATETABLEt(...,start_tsTIMESTAMP(9),end_tsTIMESTAMP(9),

PERIODFORapp(start_ts,end_ts))

Since SQL:2011Table can be system versioned, application versioned or both.

FOR PORTION OF

Page 169: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

ID Data start_ts end_ts1 X 10:00:00 12:00:00

UPDATEtFORPORTIONOFappFROM'10:30:00'TO'11:30:00'SETDATA='Y'

ID Data start_ts end_ts1 X 10:00:00 10:30:001 Y 10:30:00 11:30:001 X 11:30:00 12:00:00

INSERTt(ID,DATA,start_ts,end_ts)VALUES(1,'X','10:00:00','12:00:00')

Since SQL:2011FOR PORTION OF

Page 170: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

ID Data start_ts end_ts1 X 10:00:00 12:00:00

UPDATEtFORPORTIONOFappFROM'10:30:00'TO'11:30:00'SETDATA='Y'

ID Data start_ts end_ts1 X 10:00:00 10:30:001 Y 10:30:00 11:30:001 X 11:30:00 12:00:00

INSERTt(ID,DATA,start_ts,end_ts)VALUES(1,'X','10:00:00','12:00:00')

Since SQL:2011FOR PORTION OF

Page 171: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

ID Data start_ts end_ts1 X 10:00:00 12:00:00

UPDATEtFORPORTIONOFappFROM'10:30:00'TO'11:30:00'SETDATA='Y'

ID Data start_ts end_ts1 X 10:00:00 10:30:001 Y 10:30:00 11:30:001 X 11:30:00 12:00:00

INSERTt(ID,DATA,start_ts,end_ts)VALUES(1,'X','10:00:00','12:00:00')

Since SQL:2011FOR PORTION OF

Page 172: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

5.1 10.4 MariaDBMySQLPostgreSQLSQLite

10.5 DB2 LUWOracleSQL Server

FORPORTIONOF Since SQL:2011

Page 173: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

Since SQL:2011Application Versioning

For a useful application versioning, WITHOUTOVERLAPS

constraints are required.

Apparently this was moved to MariaDB 10.5: https://jira.mariadb.org/browse/MDEV-16978

Page 174: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SQL:2016 (released: 2016-12-15)

Page 175: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

JSON_TABLE

Page 176: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

JSON Since SQL:2016

Page 177: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

[{"id":42,"a1":"foo"},{"id":43,"a1":"bar"}]

JSON Since SQL:2016

Page 178: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

id a1

42 foo

43 bar

[{"id":42,"a1":"foo"},{"id":43,"a1":"bar"}]

JSON Since SQL:2016

Page 179: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMJSON_TABLE(?,'$[*]'COLUMNS(idINTPATH'$.id',a1VARCHAR(…)PATH'$.a1'))r

[{"id":42,"a1":"foo"},{"id":43,"a1":"bar"}]

id a142 foo43 bar

JSON_TABLE Since SQL:2016

Page 180: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMJSON_TABLE(?,'$[*]'COLUMNS(idINTPATH'$.id',a1VARCHAR(…)PATH'$.a1'))r

[{"id":42,"a1":"foo"},{"id":43,"a1":"bar"}]

id a142 foo43 bar

Bind Parameter

JSON_TABLE Since SQL:2016

Page 181: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMJSON_TABLE(?,'$[*]'COLUMNS(idINTPATH'$.id',a1VARCHAR(…)PATH'$.a1'))r

[{"id":42,"a1":"foo"},{"id":43,"a1":"bar"}]

id a142 foo43 bar

SQL/JSON Path ‣ Query language to

select elements from a JSON document ‣Defined in the

SQL standardBind

Parameter

JSON_TABLE Since SQL:2016

Page 182: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMJSON_TABLE(?,'$[*]'COLUMNS(idINTPATH'$.id',a1VARCHAR(…)PATH'$.a1'))r

[{"id":42,"a1":"foo"},{"id":43,"a1":"bar"}]

id a142 foo43 bar

SQL/JSON Path ‣ Query language to

select elements from a JSON document ‣Defined in the

SQL standardBind

Parameter

JSON_TABLE Since SQL:2016

Page 183: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMJSON_TABLE(?,'$[*]'COLUMNS(idINTPATH'$.id',a1VARCHAR(…)PATH'$.a1'))r

[{"id":42,"a1":"foo"},{"id":43,"a1":"bar"}]

id a142 foo43 bar

SQL/JSON Path ‣ Query language to

select elements from a JSON document ‣Defined in the

SQL standardBind

Parameter

JSON_TABLE Since SQL:2016

Page 184: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMJSON_TABLE(?,'$[*]'COLUMNS(idINTPATH'$.id',a1VARCHAR(…)PATH'$.a1'))r

JSON_TABLE Since SQL:2016

Page 185: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

SELECT*FROMJSON_TABLE(?,'$[*]'COLUMNS(idINTPATH'$.id',a1VARCHAR(…)PATH'$.a1'))r

INSERTINTOtarget_table

JSON_TABLE Since SQL:2016

Page 186: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

JSON_TABLE Availability1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

MariaDB8.0 MySQL

PostgreSQLSQLite

11.1.4.4 DB2 LUW12cR1 Oracle

2016[0] SQL Server[0]OPENJSON provides similar functionality

Page 187: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

PostrgreSQL 12 (beta)

Page 188: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

New Standard SQL features in PostgreSQL 12 (likely)

SQL/JSON path language Like XPath for XML and CSS selectors for HTML(the expressive power is somewhere in the middle)

Generated columnsGENERATEDALWAYSAS(<expr>)STORED

Chained Transactions[COMMIT|ROLLBACK]…ANDCHAIN

Hyperbolic functions sinh,cosh,tanh,asinh,acosh,atanh

Page 189: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM
Page 190: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

https://www.flickr.com/photos/mfoubister/25367243054/

Page 191: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

https://www.flickr.com/photos/mfoubister/25367243054/

A lot has happened

since SQL-92

Page 192: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

https://www.flickr.com/photos/mfoubister/25367243054/

SQL has evolved

beyond the relational idea

A lot has happened

since SQL-92

Page 193: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

https://www.flickr.com/photos/mfoubister/25367243054/

SQL has evolved

beyond the relational idea

If you use SQL for

CRUD operations only, you are doing it wrong

A lot has happened

since SQL-92

Page 194: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

https://www.flickr.com/photos/mfoubister/25367243054/

SQL has evolved

beyond the relational idea

If you use SQL for

CRUD operations only, you are doing it wrong

A lot has happened

since SQL-92

https://modern-sql.com@ModernSQL by @MarkusWinand

Page 195: Neues in Open-Source-SQL-Datenbanken · 2008R2 SQL Server. INTERSECT and EXCEPT ... LATERAL Since SQL:1999 Lateral derived queries can see table names defined before: SELECT * FROM

https://www.flickr.com/photos/mfoubister/25367243054/

SQL has evolved

beyond the relational idea

If you use SQL for

CRUD operations only, you are doing it wrong

A lot has happened

since SQL-92

https://modern-sql.com@ModernSQL by @MarkusWinand

Training:https://winand.at/


Recommended