1
Lecture 13: XQueryXML Publishing, XML Storage
Monday, October 28, 2002
2
OrganizationProject:• Next phase, need to form companies• Please form group of 3, email Tessa by Thursday• Problems, little extra-credit:
– One group will have only two people– One billing, one shipping volunteers to do inventory
Homework: • Good practice for the midterm• Try to finish before Monday
3
Organization
Midterm
• Next Monday, 11/3
• Missed it ? You will get this score:– MidtermScore = 100 – 1.2(100 – FinalScore)– In other words, you will loose 20% more points
than on the final
4
Overview
• DTDs: elements and attributes
• XQuery
5
Very Simple DTD
<!DOCTYPE company [ <!ELEMENT company ((person|product)*)> <!ELEMENT person (ssn, name, office, phone?)> <!ELEMENT ssn (#PCDATA)> <!ELEMENT name (#PCDATA)> <!ELEMENT office (#PCDATA)> <!ELEMENT phone (#PCDATA)> <!ELEMENT product (pid, name, description?)> <!ELEMENT pid (#PCDATA)> <!ELEMENT description (#PCDATA)>]>
<!DOCTYPE company [ <!ELEMENT company ((person|product)*)> <!ELEMENT person (ssn, name, office, phone?)> <!ELEMENT ssn (#PCDATA)> <!ELEMENT name (#PCDATA)> <!ELEMENT office (#PCDATA)> <!ELEMENT phone (#PCDATA)> <!ELEMENT product (pid, name, description?)> <!ELEMENT pid (#PCDATA)> <!ELEMENT description (#PCDATA)>]>
6
Very Simple DTD
<company> <person> <ssn> 123456789 </ssn> <name> John </name> <office> B432 </office> <phone> 1234 </phone> </person> <person> <ssn> 987654321 </ssn> <name> Jim </name> <office> B123 </office> </person> <product> ... </product> ...</company>
<company> <person> <ssn> 123456789 </ssn> <name> John </name> <office> B432 </office> <phone> 1234 </phone> </person> <person> <ssn> 987654321 </ssn> <name> Jim </name> <office> B123 </office> </person> <product> ... </product> ...</company>
Example of valid XML document:
7
Content Model
• Element content: what we can put in an element (aka content model)
• Content model:– Complex = a regular expression over other elements
– Text-only = #PCDATA
– Empty = EMPTY
– Any = ANY
– Mixed content = (#PCDATA | A | B | C)*• (i.e. very restrictied)
8
Attributes in DTDs
<!ELEMENT person (ssn, name, office, phone?)><!ATTLIST person age CDATA #REQUIRED>
<!ELEMENT person (ssn, name, office, phone?)><!ATTLIST person age CDATA #REQUIRED>
<person age=“25”> <name> ....</name> ...</person>
<person age=“25”> <name> ....</name> ...</person>
9
Attributes in DTDs
<!ELEMENT person (ssn, name, office, phone?)><!ATTLIST person age CDATA #REQUIRED
id ID #REQUIRED
manager IDREF #REQUIRED
manages IDREFS #REQUIRED>
<!ELEMENT person (ssn, name, office, phone?)><!ATTLIST person age CDATA #REQUIRED
id ID #REQUIRED
manager IDREF #REQUIRED
manages IDREFS #REQUIRED>
<person age=“25” id=“p29432” manager=“p48293” manages=“p34982 p423234”> <name> ....</name> ...</person>
<person age=“25” id=“p29432” manager=“p48293” manages=“p34982 p423234”> <name> ....</name> ...</person>
10
Attributes in DTDs
Types:
• CDATA = string
• ID = key
• IDREF = foreign key
• IDREFS = foreign keys separated by space
• (Monday | Wednesday | Friday) = enumeration
• NMTOKEN = must be a valid XML name
• NMTOKENS = multiple valid XML names
• ENTITY = you don’t want to know this
11
Attributes in DTDs
Kind:• #REQUIRED• #IMPLIED = optional• value = default value• value #FIXED = the only value allowed
12
Using DTDs
• Must include in the XML document• Either include the entire DTD:
– <!DOCTYPE rootElement [ ....... ]>
• Or include a reference to it:– <!DOCTYPE rootElement SYSTEM
“http://www.mydtd.org”>
• Or mix the two... (e.g. to override the external definition)
13
FLWR (“Flower”) Expressions
FOR ...
LET...
WHERE...
RETURN...
FOR ...
LET...
WHERE...
RETURN...
14
<bib><book> <publisher> Addison-Wesley </publisher> <author> Serge Abiteboul </author> <author> <first-name> Rick </first-name> <last-name> Hull </last-name> </author> <author> Victor Vianu </author> <title> Foundations of Databases </title> <year> 1995 </year></book><book price=“55”> <publisher> Freeman </publisher> <author> Jeffrey D. Ullman </author> <title> Principles of Database and Knowledge Base Systems </title> <year> 1998 </year></book>
</bib>
<bib><book> <publisher> Addison-Wesley </publisher> <author> Serge Abiteboul </author> <author> <first-name> Rick </first-name> <last-name> Hull </last-name> </author> <author> Victor Vianu </author> <title> Foundations of Databases </title> <year> 1995 </year></book><book price=“55”> <publisher> Freeman </publisher> <author> Jeffrey D. Ullman </author> <title> Principles of Database and Knowledge Base Systems </title> <year> 1998 </year></book>
</bib>
Sample Data for Queries (more or less)
15
FOR-WHERE-RETURN
Find all book titles published after 1995:
FOR $x IN document("bib.xml")/bib/book
WHERE $x/year/text() > 1995
RETURN $x/title
FOR $x IN document("bib.xml")/bib/book
WHERE $x/year/text() > 1995
RETURN $x/title
Result: <title> abc </title> <title> def </title> <title> ghi </title>
16
FOR-WHERE-RETURN
Equivalently (perhaps more geekish)
FOR $x IN document("bib.xml")/bib/book[year/text() > 1995] /title
RETURN $x
FOR $x IN document("bib.xml")/bib/book[year/text() > 1995] /title
RETURN $x
And even shorter:
document("bib.xml")/bib/book[year/text() > 1995] /title document("bib.xml")/bib/book[year/text() > 1995] /title
17
FOR-WHERE-RETURN
• Find all book titles and the year when they were published:
FOR $x IN document("bib.xml")/ bib/bookRETURN <answer> <title>{ $x/title/text() } </title> <year>{ $x/year/text() } </year> </answer>
FOR $x IN document("bib.xml")/ bib/bookRETURN <answer> <title>{ $x/title/text() } </title> <year>{ $x/year/text() } </year> </answer>
18
FOR-WHERE-RETURN
• Notice the use of “{“ and “}”
• What is the result without them ?
FOR $x IN document("bib.xml")/ bib/bookRETURN <answer> <title> $x/title/text() </title> <year> $x/year/text() </year> </answer>
FOR $x IN document("bib.xml")/ bib/bookRETURN <answer> <title> $x/title/text() </title> <year> $x/year/text() </year> </answer>
19
XQuery: NestingFor each author of a book by Morgan
Kaufmann, list all books she published:
FOR $b IN document(“bib.xml”)/bib, $a IN $b/book[publisher /text()=“Morgan Kaufmann”]/authorRETURN <result> { $a, FOR $t IN $b/book[author/text()=$a/text()]/title RETURN $t } </result>
FOR $b IN document(“bib.xml”)/bib, $a IN $b/book[publisher /text()=“Morgan Kaufmann”]/authorRETURN <result> { $a, FOR $t IN $b/book[author/text()=$a/text()]/title RETURN $t } </result>
In the RETURN clause comma concatenates XML fragments
20
XQuery
<result> <author>Jones</author> <title> abc </title> <title> def </title> </result> <result> <author> Smith </author> <title> ghi </title> </result>
<result> <author>Jones</author> <title> abc </title> <title> def </title> </result> <result> <author> Smith </author> <title> ghi </title> </result>
Result:
21
Aggregates
Find all books with more than 3 authors:
count = a function that countsavg = computes the averagesum = computes the sumdistinct-values = eliminates duplicates
FOR $x IN document("bib.xml")/bib/bookWHERE count($x/author)>3 RETURN $x
FOR $x IN document("bib.xml")/bib/bookWHERE count($x/author)>3 RETURN $x
22
Aggregates
Same thing:
FOR $x IN document("bib.xml")/bib/book[count(author)>3] RETURN $x
FOR $x IN document("bib.xml")/bib/book[count(author)>3] RETURN $x
23
Aggregates
Print all authors who published more than 3 books – be aware of duplicates !
FOR $b IN document("bib.xml")/bib, $a IN distinct-values($b/book/author/text())WHERE count($b/book[author/text()=$a)>3 RETURN <author> { $a } </author>
FOR $b IN document("bib.xml")/bib, $a IN distinct-values($b/book/author/text())WHERE count($b/book[author/text()=$a)>3 RETURN <author> { $a } </author>
24
XQuery
Find books whose price is larger than average:
FOR $b in document(“bib.xml”)/bibLET $a:=avg($b/book/price/text())FOR $x in $b/bookWHERE $x/price/text() > $aRETURN $x
FOR $b in document(“bib.xml”)/bibLET $a:=avg($b/book/price/text())FOR $x in $b/bookWHERE $x/price/text() > $aRETURN $x
25
FOR-WHERE-RETURN
• “Flatten” the authors, i.e. return a list of (author, title) pairs
FOR $b IN document("bib.xml")/bib/book, $x IN $b/title/text(), $y IN $b/author/text()RETURN <answer> <title> { $x } </title> <author> { $y } </author> </answer>
FOR $b IN document("bib.xml")/bib/book, $x IN $b/title/text(), $y IN $b/author/text()RETURN <answer> <title> { $x } </title> <author> { $y } </author> </answer>
Result:<answer> <title> abc </title> <author> efg </author></answer><answer> <title> abc </title> <author> hkj </author></answer>
26
FOR-WHERE-RETURN
• For each author, return all titles of her/his books
FOR $b IN document("bib.xml")/bib, $x IN $b/book/author/text()RETURN <answer> <author> { $x } </author> { FOR $y IN $b/book[author/text()=$x]/title RETURN $y } </answer>
FOR $b IN document("bib.xml")/bib, $x IN $b/book/author/text()RETURN <answer> <author> { $x } </author> { FOR $y IN $b/book[author/text()=$x]/title RETURN $y } </answer>
What aboutduplicateauthors ?
Result:<answer> <author> efg </author> <title> abc </title> <title> klm </title> . . . .</answer>
27
FOR-WHERE-RETURN
• Same, but eliminate duplicate authors:
FOR $b IN document("bib.xml")/bibLET $a := distinct-values($b/book/author/text())FOR $x IN $aRETURN <answer> <author> $x </author> { FOR $y IN $b/book[author/text()=$x]/title RETURN $y } </answer>
FOR $b IN document("bib.xml")/bibLET $a := distinct-values($b/book/author/text())FOR $x IN $aRETURN <answer> <author> $x </author> { FOR $y IN $b/book[author/text()=$x]/title RETURN $y } </answer>
28
FOR-WHERE-RETURN
• Same thing:
FOR $b IN document("bib.xml")/bib, $x IN distinct-values($b/book/author/text())RETURN <answer> <author> $x </author> { FOR $y IN $b/book[author/text()=$x]/title RETURN $y } </answer>
FOR $b IN document("bib.xml")/bib, $x IN distinct-values($b/book/author/text())RETURN <answer> <author> $x </author> { FOR $y IN $b/book[author/text()=$x]/title RETURN $y } </answer>
29
FOR-WHERE-RETURN
Find book titles by the coauthors of “Database Theory”:
FOR $b IN document("bib.xml")/bib, $x IN $b/book[title/text() = “Database Theory”], $y IN $b/book[author/text() = $x/author/text()]RETURN <answer> { $y/title/text() } </answer>
FOR $b IN document("bib.xml")/bib, $x IN $b/book[title/text() = “Database Theory”], $y IN $b/book[author/text() = $x/author/text()]RETURN <answer> { $y/title/text() } </answer>
Result: <answer> abc </ answer > < answer > def </ answer > < answer > abc </ answer > < answer > ghk </ answer >
Question:Why do we get duplicates ?
30
Distinct-values
Same as before, but eliminate duplicates:
Result: <answer> abc </ answer > < answer > def </ answer > < answer > ghk </ answer >
distinct-values = a function that eliminates duplicates
Need to apply to a collectionof text values, not of elements – note how query has changed
FOR $b IN document("bib.xml")/bib, $x IN $b/book[title/text() = “Database Theory”]/author/text(), $y IN distinct-values($b/book[author/text() = $x] /title/text())
RETURN <answer> { $y } </answer>
FOR $b IN document("bib.xml")/bib, $x IN $b/book[title/text() = “Database Theory”]/author/text(), $y IN distinct-values($b/book[author/text() = $x] /title/text())
RETURN <answer> { $y } </answer>
31
SQL and XQuery Side-by-sideProduct(pid, name, maker)Company(cid, name, city)
Find all products made in Seattle
SELECT x.nameFROM Product x, Company yWHERE x.maker=y.cid and y.city=“Seattle”
SELECT x.nameFROM Product x, Company yWHERE x.maker=y.cid and y.city=“Seattle”
FOR $r in document(“db.xml”)/db, $x in $r/Product/row, $y in $r/Company/rowWHERE $x/maker/text()=$y/cid/text() and $y/city/text() = “Seattle”RETURN { $x/name }
FOR $r in document(“db.xml”)/db, $x in $r/Product/row, $y in $r/Company/rowWHERE $x/maker/text()=$y/cid/text() and $y/city/text() = “Seattle”RETURN { $x/name }
SQL XQuery
FOR $y in /db/Company/row[city/text()=“Seattle”], $x in /db/Product/row[maker/text()=$y/cid/text()]RETURN { $x/name }
FOR $y in /db/Company/row[city/text()=“Seattle”], $x in /db/Product/row[maker/text()=$y/cid/text()]RETURN { $x/name }
CoolXQuery
32
<db> <product> <row> <pid> ??? </pid> <name> ??? </name> <maker> ??? </maker> </row> <row> …. </row> … </product> . . . .</db>
33
XQuery
• FOR $x in expr -- binds $x to each value in the list expr
• LET $x := expr -- binds $x to the entire list expr– Useful for common subexpressions and for
aggregations
34
XQuery
$b is a collection of elements, not a single elementcount = a (aggregate) function that returns the number of elms
<big_publishers> { FOR $p IN distinct-values(//publisher/text()) LET $b := /db/book[publisher/text() = $p] WHERE count($b) > 100 RETURN <publisher> { $p } </publisher>}</big_publishers>
<big_publishers> { FOR $p IN distinct-values(//publisher/text()) LET $b := /db/book[publisher/text() = $p] WHERE count($b) > 100 RETURN <publisher> { $p } </publisher>}</big_publishers>
Find all publishers that published more than 100 books:
35
XQuery
Summary:
• FOR-LET-WHERE-RETURN = FLWR
FOR/LET Clauses
WHERE Clause
RETURN Clause
List of tuples
List of tuples
Instance of Xquery data model
36
FOR v.s. LET
FOR
• Binds node variables iteration
LET
• Binds collection variables one value
37
FOR v.s. LET
FOR $x IN /bib/bookRETURN <result> { $x } </result>
FOR $x IN /bib/bookRETURN <result> { $x } </result>
Returns: <result> <book>...</book></result> <result> <book>...</book></result> <result> <book>...</book></result> ...
LET $x := /bib/bookRETURN <result> { $x } </result>
LET $x := /bib/bookRETURN <result> { $x } </result>
Returns: <result> <book>...</book> <book>...</book> <book>...</book> ... </result>
38
Collections in XQuery
• Ordered and unordered collections– /bib/book/author/text() = an ordered collection: result is
in document order
– distinct-values(/bib/book/author/text()) = an unordered collection: the output order is implementation dependent
• LET $a := /bib/book $a is a collection• $b/author a collection (several authors...)
RETURN <result> { $b/author } </result>RETURN <result> { $b/author } </result>Returns: <result> <author>...</author> <author>...</author> <author>...</author> ...</result>