Post on 21-Dec-2015
transcript
XGI: A Graphic Interface for XQuery XGI: A Graphic Interface for XQuery Creation and XML Schema Creation and XML Schema
Visualization Visualization
Xiang LiXiang Li
University of WashingtonUniversity of Washington
Biomedical and Health InformaticsBiomedical and Health Informatics
Master Thesis Committee:Master Thesis Committee:
James Brinkley, John GennariJames Brinkley, John Gennari
OutlineOutline
Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system My approach and XQuery Graphic My approach and XQuery Graphic
Interface (XGI) system overviewInterface (XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions
OutlineOutline
Needs for graphically querying XMLNeeds for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system My approach and the XQuery Graphic Interface My approach and the XQuery Graphic Interface
(XGI) system overview(XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions
XML and XQueryXML and XQuery
XMLXML De facto standard language for data exchange and De facto standard language for data exchange and
representation in biomedical researchrepresentation in biomedical research Semistructured data modelSemistructured data model Wide-spread adoption and many data sources in XML Wide-spread adoption and many data sources in XML
formatformat XQueryXQuery
W3C-candidate, textual XML querying languageW3C-candidate, textual XML querying language Turing-complete and well-supportedTuring-complete and well-supported Required programmatic skill similar to SQLRequired programmatic skill similar to SQL
ExamplesExamples<root> {<root> {
<patient> <patient>
<year_of_birth> </year_of_birth> <year_of_birth> </year_of_birth>
<wada_comments> </wada_comments> <wada_comments> </wada_comments>
<wada_memory> </wada_memory> <wada_memory> </wada_memory>
<wada_language> </wada_language> <wada_language> </wada_language>
<handedness> </handedness> <handedness> </handedness>
<sex> </sex> <sex> </sex>
<type> </type> <type> </type>
<gao_research_num> <gao_research_num> </gao_research_num> </gao_research_num>
<description> </description> <description> </description>
<pre> </pre> <pre> </pre>
<size> </size> <size> </size>
<pnum> </pnum> <pnum> </pnum>
<viq> </viq><viq> </viq>
<age_at_registration> <age_at_registration> </age_at_registration></age_at_registration>
....
……
……..
</patient></patient>
</root></root>
<result><result>
{{
for $pa in $root/patientfor $pa in $root/patient
where $pa/surgery/csmstudy/trial/stimulated/text() = 'Y'where $pa/surgery/csmstudy/trial/stimulated/text() = 'Y'
return <patient>return <patient>
{$pa/pnum}{$pa/pnum}
{$pa/viq}{$pa/viq}
{$pa/age_at_registration>{$pa/age_at_registration>
</patient></patient>
}</result>}</result>
XML source document
XQuery expression
Motivations for a graphic query Motivations for a graphic query systemsystem
Creating XQuery is difficultCreating XQuery is difficult Unfamiliar XQuery syntax and semanticsUnfamiliar XQuery syntax and semantics Need explicit knowledge of the source data schemaNeed explicit knowledge of the source data schema
Goals:Goals: Implement the Query-By-Example (QBE) paradigmImplement the Query-By-Example (QBE) paradigm Simplify query construction for novice usersSimplify query construction for novice users Expedite query generation process for expert usersExpedite query generation process for expert users Balance between usability and expressivityBalance between usability and expressivity
OutlineOutline
Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system My approach and the XQuery Graphic My approach and the XQuery Graphic
Interface (XGI) system overviewInterface (XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions
Design approachesDesign approaches Structured query approachStructured query approach
Graphical interface is simpleGraphical interface is simple
Query process is quick and uncomplicatedQuery process is quick and uncomplicated
Output format is pre-determinedOutput format is pre-determined
Users are not exposed to the underlying querying languageUsers are not exposed to the underlying querying language
Unstructured query approachUnstructured query approachArbitrarily construct the output formatArbitrarily construct the output format
Able to generate more expressive XQuery statementsAble to generate more expressive XQuery statements
Complex query interfaceComplex query interface
Existing systemsExisting systems
Structured query approachStructured query approach QSByEQSByE QURSEDQURSED XBrainXBrain
Unstructured query approachUnstructured query approach BBQBBQ XQBEXQBE Stylus StudioStylus Studio
Existing systemsExisting systems
Structured query approachStructured query approach QSByEQSByE QURSEDQURSED XBrainXBrain
Unstructured query approachUnstructured query approach BBQBBQ XQBEXQBE Stylus StudioStylus Studio
QURSEDQURSED
Query and Reporting Semistructured DataQuery and Reporting Semistructured Data Developers create XML-querying forms for end-Developers create XML-querying forms for end-
usersusers
Displays source schema in a treeWeb-based query forms
Developers have to create the query form andthe output template for every single query
End-users cannot modify the query form, the output template, and the generated query
Uses output template to define structure
XBrainXBrain
Distributed query systemDistributed query system Developers create saved queries for end-users to Developers create saved queries for end-users to
executeexecute
Allows users to modify and save custom queriesVery simple query interface
Novice users need developers to create more complex queriesOutput format is pre-determined
XQBEXQBE
XQuery By ExampleXQuery By Example Users use the interface define the source and the Users use the interface define the source and the
construct treeconstruct tree
Able to generate fairly complex queries
Accepts multiple source schema formatsFree and still is being supported
Requires users to have explicit knowledgeof the source data schemaElaborate syntax and semantics
Stylus StudioStylus Studio
Comprehensive XML editing, publishing, and querying Comprehensive XML editing, publishing, and querying tooltool
Novel query interface design by explicitly constructing Novel query interface design by explicitly constructing XQuery expressionXQuery expression
Able to create very expressive XQuery statements
Displays the source schema as a tree
Expensive
Not open source or web-based
OutlineOutline
Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful Requirements for a successful
systemsystem My approach and the XQuery Graphic Interface My approach and the XQuery Graphic Interface
(XGI) system overview(XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions
System requirementsSystem requirements
Reduced cost of implementationReduced cost of implementation Support for multiple schema formatsSupport for multiple schema formats Navigable source treeNavigable source tree Powerful XQuery creationPowerful XQuery creation Easy integration with other applicationsEasy integration with other applications Support collaborationSupport collaboration
OutlineOutline
Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system
My approach and the XQuery Graphic My approach and the XQuery Graphic Interface (XGI) system overviewInterface (XGI) system overview
Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions
Reduced cost of implementationReduced cost of implementation Web-based using Asynchronous JavaScript Web-based using Asynchronous JavaScript
and XML (AJAX) technologyand XML (AJAX) technology Shared installation and resourcesShared installation and resources
Support for multiple schema formatsSupport for multiple schema formats XML Schema, Document Type Data (DTD), XML Schema, Document Type Data (DTD),
and auto-extract schema from XML documentand auto-extract schema from XML document Navigable source treeNavigable source tree
Uses hierarchical tree to represent the nested Uses hierarchical tree to represent the nested XML source schemaXML source schema
My approachMy approach
My approach (cont.)My approach (cont.)
Powerful XQuery creationPowerful XQuery creation Implements a subset of XQueryImplements a subset of XQuery
Easy integration with other applicationsEasy integration with other applications Uses open source toolkitsUses open source toolkits Implements modular design Implements modular design
Support collaborationSupport collaboration Allows users to save queries with self-Allows users to save queries with self-
explanatory namesexplanatory names
System architectureSystem architectureWeb BrowserLocal
Filesystem
Graphic interface scripts
Client
Schemas File
Management
Schemas
Data Model
ServerFilesystem
TranslationEngine
SchemaPreprocessor
Server
Data ModelController
XQueryGeneration
Engine
XQuery generation engineXQuery generation engine
Uses an Extended Uses an Extended Backus-Naur Form Backus-Naur Form grammargrammar
Retrieve the query Retrieve the query schema from the data schema from the data model controllermodel controller Preprocess the schema Preprocess the schema
to fit the grammarto fit the grammar Return the generated Return the generated
query to the query query to the query interfaceinterface
<query> <query> := <flwor_exp> | <startTag> ‘{‘ := <flwor_exp> | <startTag> ‘{‘ <query> ‘}’ <endTag> <query> ‘}’ <endTag>
<flwor_exp><flwor_exp> := <for> <where>? := <for> <where>? <return><return>
<for><for> := ‘for’ <some_var> ( ‘,’ <some_var> )*:= ‘for’ <some_var> ( ‘,’ <some_var> )*
<some_var><some_var> := ‘$’<var_name> ‘in’ := ‘$’<var_name> ‘in’ <path_exp><path_exp>
<where><where> := ‘where’ <constraint>?:= ‘where’ <constraint>?
<constraint><constraint> := <predicate> ( ‘and’ := <predicate> ( ‘and’ <predicate> )*<predicate> )*
<predicate><predicate> := ‘exists(’ <path_exp> ‘)’ | := ‘exists(’ <path_exp> ‘)’ | < pred_exp>< pred_exp>
<pred_exp> <pred_exp> := <expression> <operator> := <expression> <operator> <expression><expression>
<expression><expression> := <const> | <variable> := <const> | <variable>
<operator><operator> := ‘=’ | ‘<’ | ‘>’ | ‘<=’ | ‘>=’ | := ‘=’ | ‘<’ | ‘>’ | ‘<=’ | ‘>=’ | ‘!=’‘!=’
<return><return> := ‘return’ ( <emptyTag> | <path_exp> := ‘return’ ( <emptyTag> | <path_exp> ) | ‘return’ <query>) | ‘return’ <query>
<startTag><startTag> := ‘<’<name> <attr>? ‘>’:= ‘<’<name> <attr>? ‘>’
<endTag><endTag> := ‘</’ <name> ‘>’:= ‘</’ <name> ‘>’
<emptyTag><emptyTag> := ‘<’ <name> <attr>? ‘/>’:= ‘<’ <name> <attr>? ‘/>’
<attr><attr> := (<name> ‘=’ <path_exp> ) + := (<name> ‘=’ <path_exp> ) +
<name><name> := valid name of schema elements:= valid name of schema elements
<path_exp><path_exp> := XPath expression:= XPath expression
<variable><variable> := any variable that has := any variable that has been bound to an outer schema elementbeen bound to an outer schema element
<const><const> := a constant value of the schema := a constant value of the schema elementelement
<var_name><var_name> := an automatically := an automatically generated, non-duplicative name for a variablegenerated, non-duplicative name for a variable
OutlineOutline
Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system My approach and XQuery Graphic My approach and XQuery Graphic
Interface (XGI) system overviewInterface (XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions
Interface layoutInterface layoutDocument name panel
Toolbar panel
Search box
Source panelQuery panel
Information panelSaved predicate panel
Example query 1Example query 1<results><results>
{{
for $pa in $root/patientfor $pa in $root/patient
return <patient id="{$pa/pnum/text()}">return <patient id="{$pa/pnum/text()}">
{$pa/sex}{$pa/sex}
{$pa/viq}{$pa/viq}
</patient></patient>
}</results>}</results>
<root> {<root> {
<patient> <patient>
<year_of_birth> </year_of_birth> <year_of_birth> </year_of_birth>
<wada_comments> </wada_comments> <wada_comments> </wada_comments>
<wada_memory> </wada_memory> <wada_memory> </wada_memory>
<wada_language> </wada_language> <wada_language> </wada_language>
<handedness> </handedness> <handedness> </handedness>
<sex> </sex> <sex> </sex>
<type> </type> <type> </type>
<gao_research_num> <gao_research_num> </gao_research_num> </gao_research_num>
<description> </description> <description> </description>
<pre> </pre> <pre> </pre>
<size> </size> <size> </size>
<pnum> </pnum> <pnum> </pnum>
<viq> </viq><viq> </viq>
<age_at_registration> <age_at_registration> </age_at_registration></age_at_registration>
....
……
……..
</patient></patient>
</root></root>CSM database schema
Load a source schemaLoad a source schema
Insert the root nodeInsert the root node
Add nodes to the query schemaAdd nodes to the query schema
Add an attributeAdd an attribute
Create XQueryCreate XQuery
Example query 2Example query 2<results><results>
{{
for $pa in $root/patientfor $pa in $root/patient
where exists($pa/is_public) and $pa/surgery/csmstudy/trial/stimulated/text() = 'Y'where exists($pa/is_public) and $pa/surgery/csmstudy/trial/stimulated/text() = 'Y'
return <patient id="{$pa/pnum/text()}">return <patient id="{$pa/pnum/text()}">
{$pa/sex}{$pa/sex}
{$pa/viq}{$pa/viq}
<age><age>
{$pa/age_at_registration/text()}{$pa/age_at_registration/text()}
</age></age>
{{
for $tr in $pa/surgery/csmstudy/trialfor $tr in $pa/surgery/csmstudy/trial
return <trials>return <trials>
{$tr/trial_num}{$tr/trial_num}
</trials></trials>
}}
</patient></patient>
}</results>}</results>
Add a user-defined nodeAdd a user-defined node
Change node relationChange node relation
Add existential quantificationAdd existential quantification
Add a predicateAdd a predicate
Change node nameChange node name
Create XQueryCreate XQuery
OutlineOutline
Needs for graphically querying XML Needs for graphically querying XML Existing graphical query solutionsExisting graphical query solutions Requirements for a successful systemRequirements for a successful system My approach and XQuery Graphic My approach and XQuery Graphic
Interface (XGI) system overviewInterface (XGI) system overview Using XGI to build a XQueryUsing XGI to build a XQuery System validation and evaluationSystem validation and evaluation Discussion and conclusionsDiscussion and conclusions
ValidationValidation
XGI vs. XQBEXGI vs. XQBE Compare featuresCompare features Compare capable queriesCompare capable queries
Recreate queries from XBrainRecreate queries from XBrain Language-error queriesLanguage-error queries Custom saved queriesCustom saved queries
Compare features with XQBECompare features with XQBEFeatureFeature XQueryXQuery XQBEXQBE XGIXGI
Existential Existential QuantificationQuantification
YesYes YesYes YesYes
ConjunctionConjunction YesYes YesYes YesYes
Breadth ProjectionBreadth Projection YesYes YesYes YesYes
Depth ProjectionDepth Projection YesYes YesYes YesYes
RenamingRenaming YesYes YesYes YesYes
New ElementNew Element YesYes YesYes YesYes
JoinJoin YesYes YesYes PartialPartial
Cartesian ProductCartesian Product YesYes YesYes PartialPartial
FlatteningFlattening YesYes YesYes YesYes
NestingNesting YesYes PartialPartial NoNo
FilteringFiltering YesYes YesYes NoNo
NegationNegation YesYes PartialPartial NoNo
AggregatesAggregates YesYes YesYes NoNo
Arithmetic Arithmetic ComputationsComputations
YesYes YesYes NoNo
SortingSorting YesYes YesYes NoNo
Recreate queriesRecreate queries
Able to generate 13/23 queries capable by Able to generate 13/23 queries capable by XQBEXQBE
Generate all language error queries from XBrainGenerate all language error queries from XBrain
Expert users created custom queries in XBrainExpert users created custom queries in XBrain Fully or partially recreate 22/43Fully or partially recreate 22/43 Cannot recreate 21/43Cannot recreate 21/43
User EvaluationUser Evaluation
Only one expert user used the system informallyOnly one expert user used the system informally InstallationInstallation
Require multiple packages that should be bundled Require multiple packages that should be bundled into a deployable archive fileinto a deployable archive file
Need to change a few system settingsNeed to change a few system settings Certain features could be more easily accessibleCertain features could be more easily accessible Need to support more featuresNeed to support more features More feedbacks on the limitations of the More feedbacks on the limitations of the
interfaceinterface
DiscussionDiscussion
XGI is capable of generating a large XGI is capable of generating a large subset of XQuerysubset of XQuery
Limitations:Limitations: Does not support functions and some query Does not support functions and some query
constructsconstructs Saved queries are constrained by the query Saved queries are constrained by the query
schemaschema
Future workFuture work
Support more extensive collaborationSupport more extensive collaboration Implement more XQuery featuresImplement more XQuery features
““Let”, “Order by”, etcLet”, “Order by”, etc XQuery functions and user-defined functionsXQuery functions and user-defined functions
Improve UI and enable browser Improve UI and enable browser compatibilitycompatibility
ConclusionConclusion
XGI is a simple and effective graphical XGI is a simple and effective graphical query tool for assisting both novice and query tool for assisting both novice and expert users to create complex XQuery expert users to create complex XQuery
statements quickly and accurately. statements quickly and accurately.
Thank You!Thank You!