XMLXML

State of the ArtState of the Art

Jakub MalýJakub Malý

XML XML

XML documents

•eBook, •DocBook, •JATS, •DITA•TEI•….

XML documents

•eBook, •DocBook, •JATS, •DITA•TEI•….

Querying, Transformations, Programming

• XPath• XQuery• XSLT• XProc• XSpec

Schemas

•XML Schema•Relax NG•Schematron•DTD

Databases•Native•Hybrid

XPath 3.0XPath 3.0

W3C Recommendation Apr 2014

Core data model as in 2.0 (sequences)

Added: Functional capabilities

Concat operator ‘Hello’ || ‘World’

Simple map operator

(1 to 5)!(.*.) returns (1, 4, 9, 16, 25)

W3C Recommendation Apr 2014

Core data model as in 2.0 (sequences)

Added: Functional capabilities

Concat operator ‘Hello’ || ‘World’

Simple map operator

(1 to 5)!(.*.) returns (1, 4, 9, 16, 25)

Functional: Higher-Order functions

Runtime function discovery

Inline functions

Functional: Higher-Order functions

Runtime function discovery

Inline functions

fn:filter( $seq as item()*, $predicate as function(item()) as xs:boolean)) as item()*...

fn:filter(1 to 10, function($x) { $x mod 2 eq 0 })

Result: 2 4 6 8 10

XQuery (1.0)XQuery (1.0)

W3C Recommendation Dec 2010

Superset of XPath 2.0

SQL-like expressions

Querying collections of documents

Standalone implementations (Saxon) + database implementations

W3C Recommendation Dec 2010

Superset of XPath 2.0

SQL-like expressions

Querying collections of documents

Standalone implementations (Saxon) + database implementations

for $country in db:open('factbook')let $population := $country/populationwhere $population > 100000000order by $populationreturn <country name='{ $name }'></country>

for $country in db:open('factbook')let $population := $country/populationwhere $population > 100000000order by $populationreturn <country name='{ $name }'></country>

XQuery 3.0XQuery 3.0

W3C Recommendation Apr 2014 Superset of XPath 3.0 Higher order functions Enhanced flower expressions

(additional FLWOR, interleaving) Windowing & Partitioning

(Group by)FLWOR => FWLWOGR

Try/Catch, switch

W3C Recommendation Apr 2014 Superset of XPath 3.0 Higher order functions Enhanced flower expressions

(additional FLWOR, interleaving) Windowing & Partitioning

(Group by)FLWOR => FWLWOGR

Try/Catch, switch

for sliding window $w in (2, 4, 6, 8, 10, 12, 14) start at $s when fn:true() only end at $e when $e - $s eq 2return avg($w)

for sliding window $w in (2, 4, 6, 8, 10, 12, 14) start at $s when fn:true() only end at $e when $e - $s eq 2return avg($w)

for $ppl in doc('xmark')//people/person let $ic := $ppl/profile/@incomelet $income := if($ic < 30000) then

"standard" else if($ic >= 30000 and $ic < 100000) then "standard" else if($ic >= 100000) then "preferred" else "preferred" group by $incomeorder by $incomereturn element { $income } { count($ppl) }

XSLT 3.0XSLT 3.0

Next Time! Next Time!

XProcXProc

W3C Recommendation May 2010

Pipeline language

Provides platform-independent “plumbing code”

Pipelines made of “steps”

Steps can be

Read all files in a directory

Validate with XML schema

Apply XSLT

Call web service

Zip results

Store to a file

…

Everyone loves it

W3C Recommendation May 2010

Pipeline language

Provides platform-independent “plumbing code”

Pipelines made of “steps”

Steps can be

Read all files in a directory

Validate with XML schema

Apply XSLT

Call web service

Zip results

Store to a file

…

Everyone loves it

Validate

Query

Transform result

XSpecXSpec

Testing framework for XSLT/XQuery

Written in XSLT/XProc

Supply input and expected output for testing scenarios

Compare results

Testing framework for XSLT/XQuery

Written in XSLT/XProc

Supply input and expected output for testing scenarios

Compare results

escape-for-regex.xslt

escape-for-regex.xspec

escape-for-regex-result.html

escape-for-regex.xslt

escape-for-regex.xspec

escape-for-regex-result.html

DatabasesDatabases

Native XML

Usually

Schema-less

Document centric

XQuery as the main language

Additional types of content besides XML

Structured into collections (similar to file systems)

HTTP, REST APIs (+ drivers for Java, C#...)

Open Source

eXist (demo), BaseX

Enterprise

MarkLogic (BBC, legislation.gov.uk, Obamacare)

Native XML

Usually

Schema-less

Document centric

XQuery as the main language

Additional types of content besides XML

Structured into collections (similar to file systems)

HTTP, REST APIs (+ drivers for Java, C#...)

Open Source

eXist (demo), BaseX

Enterprise

MarkLogic (BBC, legislation.gov.uk, Obamacare)

XML Applications? XML Applications?

Written using XML tools over XML data

XML database is the core MarkLogic, eXist, BaseX

Database stores data, queries, templates

HTTP endpoint (RESTXQ) REST commands mapped to XQuery

queries

Web UI HTML + JS

HTML + XSLT in the browser

HTML + XForms (XRX applications)

Reported 50-80% reduction in LOC

Written using XML tools over XML data

XML database is the core MarkLogic, eXist, BaseX

Database stores data, queries, templates

HTTP endpoint (RESTXQ) REST commands mapped to XQuery

queries

Web UI HTML + JS

HTML + XSLT in the browser

HTML + XForms (XRX applications)

Reported 50-80% reduction in LOC

/products/15698

declare %rest:path(“products/{$id}")        %rest:GET        function product:detail($id as xs:integer) { ...}

<html><head>

<title>Product detail: 15698</title></head><body>

<div><label>Product id:</label><input type=“text”>15998</input>

<label>Name: </label>…

</div></body>

</html>

XML Schema 1.1XML Schema 1.1

W3C Recommendation Apr 2012

New in 1.1

Assertions (rules..)

Conditional Type Alternatives

Everyone hates it..

verbose

Overengineered

But something is good

Simple type system used everywhere

W3C Recommendation Apr 2012

New in 1.1

Assertions (rules..)

Conditional Type Alternatives

Everyone hates it..

verbose

Overengineered

But something is good

Simple type system used everywhere

Relax NGRelax NG

OASIS specification Dec 2001

Response to XML Schema

2 syntaxes (equivalent) XML

Compact (XQuery like)

„Return to the roots“ Tree-grammar patterns

No inheritance

No substitutions groups

Sadly, XML schema is everywhere…

OASIS specification Dec 2001

Response to XML Schema

2 syntaxes (equivalent) XML

Compact (XQuery like)

„Return to the roots“ Tree-grammar patterns

No inheritance

No substitutions groups

Sadly, XML schema is everywhere…

grammar { start = element addressBook element addressBook { element card { element name { text }, element email { text }, element prefersHTML { empty }? }* }}

SchematronSchematron

ISO specification (2006)

Rule based checks of a document

Based on XPath tests

Usually implemented using XSLT/XProc

ISO specification (2006)

Rule based checks of a document

Based on XPath tests

Usually implemented using XSLT/XProc

<rule context="bk:book" role="authorCountRule"> <assert test="count(bk:author)!= 0"> A book must have at least one author </assert></rule>

ConclusionConclusion

XML is here and is not going anywhere

In some fields, there is no other viable option

Use where appropriate

Use up-to-date technology (not XSLT 1.0)

Stay away from Microsoft

XML is here and is not going anywhere

In some fields, there is no other viable option

Use where appropriate

Use up-to-date technology (not XSLT 1.0)

Stay away from Microsoft

ReferencesReferences

Standard’s update sessions (XML Prague 2014)

XSPEC

XSpec plugin for TeamCity

BaseX + nice examples on their wiki

eXist

MarkLogic

oXygen – IDE for XML

Our paper on HoF + Schematron + XSLT 3.0

Standard’s update sessions (XML Prague 2014)

XSPEC

XSpec plugin for TeamCity

BaseX + nice examples on their wiki

eXist

MarkLogic

oXygen – IDE for XML

Our paper on HoF + Schematron + XSLT 3.0