XML, XML Schema, XPath and XQuery Query Languages CS561 Slides collated from several sources,...

transcript

XML, XML Schema, XPath and XQuery Query Languages

Slides collated from several sources, including D. Suciu at Univ. of Washington

XML Data

CS561 - Spring 2007. 3

W3C standard to complement HTML

• origins: structured text SGML

• motivation:– HTML describes presentation– XML describes content

• HTML e XML subset SGML

CS561 - Spring 2007. 4

From HTML to XML

HTML describes the presentation

CS561 - Spring 2007. 5

<h1> Bibliography </h1>

<p> <i> Foundations of Databases </i>

Abiteboul, Hull, Vianu

<br> Addison Wesley, 1995

Abiteboul, Buneman, Suciu

<br> Morgan Kaufmann, 1999

CS561 - Spring 2007. 6

XML<bibliography>

<book> <title> Foundations… </title>

<author> Abiteboul </author>

<author> Vianu </author>

<publisher> Addison Wesley </publisher>

</book>

</bibliography>XML describes the content

CS561 - Spring 2007. 7

XML Terminology• tags: book, title, author, …• start tag: <book>, end tag: </book>• elements: <book>…<book>,<author>…</author>• elements are nested• empty element: <red></red> abbrv. <red/>• an XML document: single root element

well formed XML document: if it has matching tags

CS561 - Spring 2007. 8

XML: Attributes

<title> Foundations of Databases </title>

<author> Abiteboul </author>

</book>

attributes are alternative ways to represent data

CS561 - Spring 2007. 9

More XML: Oids and References

</person>

</person>

oids and references in XML are just syntax

CS561 - Spring 2007. 10

So Far

• Differences between “xml data” versus “relational data” ?

– Data model?– Typed?– Homogeneity?– Correctness?– Usage/Purpose ?

CS561 - Spring 2007. 11

“XML Data Model”

Numerous competing models:

• Document Object Model (DOM):– class hierarchy (node, element, attribute,…)– defines API to inspect/modify the document

• XML query data model (formal)

CS561 - Spring 2007. 12

XML Namespaces

• http://www.w3.org/TR/REC-xml-names

• name ::= [prefix:]localpart

<isbn:number> …. </isbn:number>

</book>

<isbn:number> …. </isbn:number>

</book>

CS561 - Spring 2007. 13

<mystyle:title> … </mystyle:title>

<mystyle:number> …

</tag>

<mystyle:title> … </mystyle:title>

<mystyle:number> …

</tag>

XML Namespaces

• syntactic: <number> , <isbn:number>

• semantic: provide URL for “shared” schema

defined here

CS561 - Spring 2007. 14

So Far

• What are “namespaces” good for ?

• Are they typically available for relational databases?

Schemas for XML

CS561 - Spring 2007. 16

DTD - Element Type Definitions

<!ELEMENT paper (title,author*, year, (journal|conference) )>

CS561 - Spring 2007. 17

XML Schemas

• generalizes DTDs (SGML derivative)

• now, instead uses XML syntax

• two main documents: structure and data types

• XML Schema more powerful but more complex

CS561 - Spring 2007. 18

XML Schema<xsd:element name=“paper” type=“papertype”/>

<xsd:complexType name=“papertype”>

<xsd:sequence>

<xsd:element name=“title” type=“xsd:string”/>

<xsd:element name=“author” minOccurs=“0”/>

<xsd:element name=“year”/>

<xsd: choice> < xsd:element name=“journal”/>

<xsd:element name=“conference”/>

</xsd:choice>

</xsd:sequence>

</xsd:complexType

</xsd:element>

<xsd:element name=“paper” type=“papertype”/>

<xsd:complexType name=“papertype”>

<xsd:sequence>

<xsd:element name=“title” type=“xsd:string”/>

<xsd:element name=“author” minOccurs=“0”/>

<xsd:element name=“year”/>

<xsd: choice> < xsd:element name=“journal”/>

<xsd:element name=“conference”/>

</xsd:choice>

</xsd:sequence>

</xsd:complexType

</xsd:element>DTD: <!ELEMENT paper (title,author*,year, (journal|

conference))>

CS561 - Spring 2007. 19

So Far

• Differences between “xml schema” versus “relational schema” ?

– Purpose ? Do we need it ?– Definition time?– Strictness of typing ?– Underlying model ?

CS561 - Spring 2007. 20

Elements versus Types in XML Schema

<xsd:element name=“person”> <xsd:complexType> <xsd:sequence> <xsd:element name=“name” type=“xsd:string”/> <xsd:element name=“address” type=“xsd:string”/> </xsd:sequence> </xsd:complexType></xsd:element>

<xsd:element name=“person” type=“ttt” /><xsd:complexType name=“ttt”> <xsd:sequence> <xsd:element name=“name” type=“xsd:string”/> <xsd:element name=“address” type=“xsd:string”/> </xsd:sequence></xsd:complexType>

DTD: <!ELEMENT person (name, address) >

CS561 - Spring 2007. 21

• Types:– Simple types (integers, strings, ...)– Complex types (regular expressions, like in DTDs)

• Element-type-element alternation:– Root element has a complex type – Complex type is a regular expression of elements– Those elements have their complex types ...– ...– Leaves have simple types

Elements versus Types in XML Schema

CS561 - Spring 2007. 22

Local and Global Types in XML Schema• Local type: <xsd:element name=“person”>

[define locally the person’s type] </xsd:element>

• Global type: <xsd:element name=“person” type=“ttt”/>

<xsd:complexType name=“ttt”> [define here the type ttt] </xsd:complexType>

Global types: can be reused in other elements

CS561 - Spring 2007. 23

Local v.s. Global Elements inXML Schema

• Local element: <xsd:complexType name=“ttt”>

<xsd:sequence> <xsd:element name=“address” type=“...”/>... </xsd:sequence> </xsd:complexType>

• Global element: <xsd:element name=“address” type=“...”/>

<xsd:complexType name=“ttt”> <xsd:sequence> <xsd:element ref=“address”/> ... </xsd:sequence> </xsd:complexType> Global elements: like in DTDs

CS561 - Spring 2007. 24

Regular Expressions in XML Schema

Recall the element-type-element alternation: <xsd:complexType name=“....”>

[regular expression on elements] </xsd:complexType>

Regular expressions:• <xsd:sequence> A B C </...>• <xsd:choice> A B C </...>• <xsd:group> A B C </...> • <xsd:... minOccurs=“0” maxOccurs=“unbounded”> ..</...>• <xsd:... minOccurs=“0” maxOccurs=“1”> ..</...>

CS561 - Spring 2007. 25

Regular Expressions in XML Schema

Regular expressions:• <xsd:sequence> A B C </...> = A B C• <xsd:choice> A B C </...> = A | B | C• <xsd:group> A B C </...> = (A B C)• <xsd:... minOccurs=“0” maxOccurs=“unbounded”> ..</...> = (...)*• <xsd:... minOccurs=“0” maxOccurs=“1”> ..</...> = (...)?

CS561 - Spring 2007. 28

Derived Types by Extensions

</sequence>

</complexType>

</sequence>

</extension>

</complexContent>

</complexType>

</sequence>

</complexType>

</sequence>

</extension>

</complexContent>

</complexType>

Corresponds to inheritance

Key Constraints in XML

CS561 - Spring 2007. 30

Keys in XML Schema

</zip>

</zip>

</regions>

<parts>

<part number="872-AA">Lawnmower</part>

<part number="926-AA">Baby Monitor</part>

<part number="833-AA">Lapis Necklace</part>

<part number="455-BX">Sturdy Shelves</part>

</parts>

</purchaseReport>

</zip>

</zip>

</regions>

<parts>

<part number="872-AA">Lawnmower</part>

<part number="926-AA">Baby Monitor</part>

<part number="833-AA">Lapis Necklace</part>

<part number="455-BX">Sturdy Shelves</part>

</parts>

</purchaseReport>

</key>

</key>

XML Schema for Key :

CS561 - Spring 2007. 31

Keys in XML Schema• In general, syntax is :

</key>

</key>

Notes: All XPath expressions “start” at the element currently being definedThe fields must identify a single “node”.

CS561 - Spring 2007. 32

Keys in XML Schema

• Unique = guarantees uniqueness• Key = guarantees uniqueness and existence• All XPath expressions are “restricted”:

– /a/b | /a/c OK for selector– //a/b/*/c OK for field

• Note: better than DTD’s ID mechanism

CS561 - Spring 2007. 33

Examples of Keys in XML Schema

• Examples<key name="fullName">

</key>

</unique>

</key>

</unique>

Note: Must havesingle firstname,Single surname

CS561 - Spring 2007. 34

Foreign Keys in XML Schema

• Example

</keyref>

</keyref>

CS561 - Spring 2007. 35

So Far

• Differences between “keys/foreign-keys”in xml versus relational model?

– Purpose ? – Underlying model ?

“The Basic Building Block”

CS561 - Spring 2007. 38

• Goal = Permit access some nodes from document

• XPath main construct : Axis navigation

• Navigation step : axis + node-test + predicates

• Examples– descendant::node()– child::author– attribute::booktitle =“XML”

CS561 - Spring 2007. 39

XPath• XPath path consists of one or more navigation steps,

separated by “/”

• Navigation step : axis + node-test + predicates

• Examples– /descendant::node() /child::author– /descendant::node() /child::author [parent /attribute::booktitle =“XML”][2]

• XPath offers shortcuts :– no axis means child– // /descendant-or-self::node()/

CS561 - Spring 2007. 40

XPath- Child Axis Navigation• author is shorthand for child::author. • Examples:

– aaa -- all the children nodes labeled aaa – aaa/bbb -- all the bbb grandchildren of aaa children – */bbb all the bbb grandchildren of any child

• Notes:– . -- the context node– / -- the root node

ccc aaa

aaa bbb ccc

4 5 6 7

context node

CS561 - Spring 2007. 42

XPath- Child Axis Navigation

– /doc -- all doc children of the root– ./aaa -- all aaa children of the context node

(equivalent to aaa)

– text() -- all text children of context node– node() -- all children of the context node

(includes text and attribute nodes)– .. -- parent of the context node– .// -- the context node and all its descendants– // -- the root node and all its descendants– //text() -- all the text nodes in the document

CS561 - Spring 2007. 43

Predicates– [2] -- the second child node of the context node

– chapter[5] -- the fifth chapter child of context node

– [last()] -- the last child node of the context node

– chapter[title=“introduction”] -- the chapter children of the context node that have one or more title children whose string-value is “introduction” (string-value is concatenation of all text on descendant text nodes)

– person[.//firstname = “joe”] -- the person children of the context node that have in their descendants a firstname element with string-value “Joe”

CS561 - Spring 2007. 44

Axis navigation

• So far, our expressions have moved us down by moving to children nodes.

• Exceptions are :– . stay where you are– / go to the root– // all descendants of the root– .// all descendants of the context node

CS561 - Spring 2007. 45

Axis navigation

• XPath has several axes: ancestor, ancestor-or-self, attribute, child, descendant, descendant-or-self, following, following-sibling, namespace, parent, preceding, preceding-sibling, self

• Some of these describe single nodes:– self, parent

• Some describe sequences of nodes: – All others

CS561 - Spring 2007. 46

XPath Navigation Axesancestor

descendant

followingpreceding

following-siblingpreceding-sibling

attribute

namespace

CS561 - Spring 2007. 47

XPath Abbreviated Syntax

(nothing) child::@ attribute::// /descendant-or-self::node(). self::node().// descendant-or-self::node.. parent::node()/ (document root)

CS561 - Spring 2007. 49

So Far

Differences between SQL and XPATH?

• What are similar query capabilities?• What features does SQL have, but not XPATH?• What features does XPATH support, but not SQL?• Is XPath a full-fledged query language?

Query Languages - XQuery

CS561 - Spring 2007. 51

Summary of XQuery

• FLWR expressions• FOR and LET expressions• Collections and sorting

ResourcesXQuery: A Query Language for XML Chamberlin, Florescu, et al.W3C recommendation: www.w3.org/TR/xquery/

CS561 - Spring 2007. 52

XQuery

• Designed based on Quilt (which is based on XML-QL)

• http://www.w3.org/TR/xquery/2/2001

• XML Query data model (ordered)

CS561 - Spring 2007. 53

FLWR (“Flower”) Expressions

FOR ... LET... FOR... LET...

WHERE...

RETURN...

CS561 - Spring 2007. 54

XQuery

Find the titles of all books published after 1995:

FOR $x IN document("bib.xml")/bib/book

WHERE $x/year > 1995

RETURN $x/title

How does result look like?

CS561 - Spring 2007. 55

XQuery

Find all book titles published after 1995:

RETURN $x/title

Result: <title> abc </title> <title> def </title> <title> ghi </title>

CS561 - Spring 2007. 56

XQuery Example

FOR $a IN (document("bib.xml") /bib/book[publisher=“Morgan Kaufmann”]/author)

RETURN <result>

FOR $t IN /bib/book[author=$a]/title

RETURN $t

</result>

RETURN <result>

RETURN $t

</result>

CS561 - Spring 2007. 57

XQuery Example

RETURN <result>

RETURN $t

</result>

RETURN <result>

RETURN $t

</result>

For each author of a book by Morgan Kaufmann,

list all books she published:

What is query result ?

CS561 - Spring 2007. 58

XQueryResult: <result> <author>Jones</author> <title> abc </title> <title> def </title> </result> <result> <author>Jones</author> <title> abc </title> <title> def </title> </result>

<result> <author> Smith </author> <title> ghi </title> </result>

CS561 - Spring 2007. 59

XQuery Example: Duplicates

For each author of a book by Morgan Kaufmann, list all books she published:

FOR $a IN distinct(document("bib.xml") /bib/book[publisher=“Morgan Kaufmann”]/author)

RETURN <result>

RETURN $t

</result>

FOR $a IN distinct(document("bib.xml") /bib/book[publisher=“Morgan Kaufmann”]/author)

RETURN <result>

RETURN $t

</result>

distinct = a function that eliminates duplicates

CS561 - Spring 2007. 60

Example XQuery Result

Result: <result> <author>Jones</author> <title> abc </title> <title> def </title> </result>

<result> <author> Smith </author> <title> ghi </title> </result>

CS561 - Spring 2007. 61

XQuery

• FOR $x in expr – binds $x to each element in the list expr– Useful for iteration over some input list

• LET $x = expr – binds $x to the entire list expr– Useful for common subexpressions and for grouping

and aggregations

CS561 - Spring 2007. 62

XQuery with LET Clause

count = a (aggregate) function that returns number of elements

<big_publishers>

FOR $p IN distinct(document("bib.xml")//publisher)

LET $b := document("bib.xml")/book[publisher = $p]

WHERE count($b) > 100

RETURN $p

</big_publishers>

<big_publishers>

FOR $p IN distinct(document("bib.xml")//publisher)

LET $b := document("bib.xml")/book[publisher = $p]

WHERE count($b) > 100

RETURN $p

</big_publishers>

CS561 - Spring 2007. 63

XQuery

Find books whose price is larger than average:LET $a = avg(document("bib.xml")/bib/book/@price)

FOR $b in document("bib.xml")/bib/book

WHERE $b/@price > $a

RETURN $b

LET $a = avg(document("bib.xml")/bib/book/@price)

FOR $b in document("bib.xml")/bib/book

WHERE $b/@price > $a

RETURN $b

CS561 - Spring 2007. 64

FOR versus LET

• Binds node variables iteration

• Binds collection variables one value

CS561 - Spring 2007. 65

FOR v.s. LET

RETURN <result> $x </result>

Returns: <result> <book>...</book></result> <result> <book>...</book></result> <result> <book>...</book></result> ...

LET $x := document("bib.xml")/bib/book

Returns: <result> <book>...</book> <book>...</book> <book>...</book> ... </result>

CS561 - Spring 2007. 66

Collections in XQuery• Ordered and unordered collections

– /bib/book/author = an ordered collection

– distinct(/bib/book/author) = an unordered collection

• LET $a = /bib/book $a is a collection• $b/author a collection (several authors...)

RETURN <result> $b/author </result>RETURN <result> $b/author </result>Returns: <result> <author>...</author> <author>...</author> <author>...</author> ...</result>

CS561 - Spring 2007. 67

XQuery Summary

FOR-LET-WHERE-RETURN = FLWR

FOR/LET Clauses

WHERE Clause

RETURN Clause

List of tuples

Instances of XQuery data model

CS561 - Spring 2007. 68

XQuery

Some more query features

CS561 - Spring 2007. 69

Sorting in XQuery

<publisher_list> FOR $p IN distinct(document("bib.xml")//publisher) RETURN <publisher> <name> $p/text() </name> , FOR $b IN document("bib.xml")//book[publisher = $p] RETURN <book> $b/title , $b/@price </book> SORTBY (price DESCENDING) </publisher> SORTBY (name) </publisher_list>

CS561 - Spring 2007. 70

Sorting in XQuery

• Sorting arguments: refer to name space of RETURN clause, not of FOR clause

• TIP: To sort on an element you don’t want to display, first return it, then remove it with an additional query.

CS561 - Spring 2007. 71

If-Then-Else

FOR $h IN //holding

RETURN <holding>

$h/title,

IF $h/@type = "Journal"

THEN $h/editor

ELSE $h/author

</holding> SORTBY (title)

FOR $h IN //holding

RETURN <holding>

$h/title,

IF $h/@type = "Journal"

THEN $h/editor

ELSE $h/author

</holding> SORTBY (title)

CS561 - Spring 2007. 72

Existential Quantifiers

FOR $b IN //book

WHERE SOME $p IN $b//para SATISFIES

contains($p, "sailing")

AND contains($p, "windsurfing")

RETURN $b/title

FOR $b IN //book

WHERE SOME $p IN $b//para SATISFIES

AND contains($p, "windsurfing")

RETURN $b/title

CS561 - Spring 2007. 73

Universal Quantifiers

FOR $b IN //book

WHERE EVERY $p IN $b//para SATISFIES

RETURN $b/title

FOR $b IN //book

WHERE EVERY $p IN $b//para SATISFIES

RETURN $b/title

CS561 - Spring 2007. 74

So Far

• Similarities between SQL and XQuery?

• Differences between SQL and XQuery?

XML, XML Data Model

XML Schema, XPath XQuery

XML, XML Schema, XPath and XQuery Query Languages CS561 Slides collated from several sources,...

Documents