Date post: | 10-Aug-2015 |
Category: |
Science |
Upload: | boris-glavic |
View: | 54 times |
Download: | 1 times |
Interoperability for Provenance-aware Databases
using PROV and JSON
Dieter Gawlick, Zhen Hua Liu, Vasudha Krishnaswamy
Oracle Corporation
Raghav Kapoor, Boris Glavic
Illinois Institute of Technology
Venkatesh Radhakrishnan
Xing NiuIllinois Institute of Technology
Outline
① Introduction
② Related work
③ Overview
④ Export and Import
⑤ Experimental Results
⑥ Conclusions and Future Work
Introduction
• The PROV standards A standardized, extensible representation of provenance
graphs Exchange of provenance information between systems
• Provenance-aware DBMS Computing the provenance of database operations E.g., Perm[1], GProM [2], DBNotes[3], Orchestra[4],
LogicBlox[5]
3
[1] B. Glavic, R. J. Miller, and G. Alonso. Using SQL for Efficient Generation and Querying of Provenance Information. In In Search of Elegance in the Theory and Practice of Computation, pages 291–320. Springer, 2013..[2] YB. Arab, D. Gawlick, V. Radhakrishnan, H. Guo, and B. Glavic. A generic provenance middleware for database queries, updates, and transactions. In TaPP, 2014.[3] D. Bhagwat, L. Chiticariu, W.-C. Tan, and G. Vijayvargiya. An Annotation Management System for Relational Databases. VLDB Journal, 14(4):373–396, 2005.[4] G. Karvounarakis, T. J. Green, Z. G. Ives, and V. Tannen. Collaborative data sharing via update exchange and provenance. TODS, 38(3):19, 2013.[5] Huang, S., Green, T., Loo, B.: Datalog and emerging applications: an interactive tutorial. In: SIGMOD, pp. 1213–1216 (2011)
5
Introduction
• Problem:No relational database system supports tracking of
database provenance as well as import and export of provenance in PROV
Not capable of exporting provenance into standardized formats
• E.g., GProM:Essentially produces wasDerivedFrom edges
• Between the output tuples of a query Q and its inputs.
However, not available as PROV graphs• No way to track the derivation back to non-database entities
6
Introduction
• GProM System
Computes provenance for database operations• Queries, updates, transactions
Using SQL language extensions• e.g., PROVENANCE OF (SELECT ...)
7
Introduction
• Example of GProM in actionThe result of PROVENANCE OF for query QEach tuple in this result represents one wasDerivedFrom
assertion
• E.g., tuple to1 was derived from tuple t1
8
Introduction
• Goal: make databases interoperable with other provenance systems
• Approach:Export and import of provenance
• PROV-JSON
Propagation of imported provenance Implemented in GProM using SQL
Outline
① Introduction
② Related work
③ Overview
④ Export and Import
⑤ Experimental Results
⑥ Conclusion and future work
10
Related Work
• How to integrate provenance graphs by identifying common elements? [6]
• Address interoperability problem between databases and other provenance-aware systems through– Common model for both types of provenance [7][8][9]– Monitoring database access to link database provenance with other
provenance systems [10][11]
[6] A. Gehani and D. Tariq. Provenance integration. In TaPP, 2014.[7] U. Acar, P. Buneman, J. Cheney, J. van den Bussche, N. Kwasnikowska, and S. Vansummeren. A graph model of data and workflowprovenance. In TaPP, 2010.[8] Y. Amsterdamer, S. Davidson, D. Deutch, T. Milo, J. Stoyanovich, and V. Tannen. Putting Lipstick on Pig: Enabling Database-style Workflow Provenance. PVLDB, 5(4):346–357, 2011.[9] D. Deutch, Y. Moskovitch, and V. Tannen. A provenance framework for data-dependent process analysis. PVLDB, 7(6), 2014.[10] F. Chirigati and J. Freire. Towards integrating workflow and database provenance. In IPAW, pages 11–23, 2012.[11] Q. Pham, T. Malik, B. Glavic, and I. Foster. LDV: Light-weight Database Virtualization. In ICDE, pages 1179–1190, 2015.
Outline
① Introduction
② Related works
③ Overview
④ Export and Import
⑤ Experimental Results
⑥ Conclusion and future work
12
Overview
• We introduce techniques for exporting database provenance as PROV documents
• Importing PROV graphs alongside data• Linking outputs of SQL operations to imported provenance
for their inputs– Implementation in GProM offloads generation of PROV documents
to backend database• SQL and string concatenation
Outline
① Introduction
② Related works
③ Overview
④ Export and Import
⑤ Experimental Results
⑥ Conclusion and future work
14
Export and Import
• Export– Added TRANSLATE AS clause
• e.g., PROVENANCE OF (SELECT ...) TRANSLATE AS …
– Construct PROV-JSON document from database provenance① Running several projections over the provenance
computation– E.g., ‘”_:wgb\(’ || F0.STATE || ‘|’ || F0.”AVG(AGE)” || ‘\)’…
② Uses aggregation to concatenate all snippets of a certain type– E.g., entity nodes, wasGeneratedBy edges, allUsed edges
③ Uses string concatenation to create final document
16
Export and Import
• Import Import PROV for an existing relation Provide a language construct IMPORT PROV FOR ... Import available PROV graphs for imported tuples and store
them alongside the dataAdd three columns to each table to store imported
provenance• prov doc: store a PROV-JSON snippet representing its
provenance• Prov_eid: indicates which of the entities in this snippet
represents the imported tuple• Prov_time: stores a timestamp as of the time when the tuple
was imported
17
Export and Import
• Import: exampleRelation user with imported provenanceAttribute value d is the PROV graph from running
example without database activities and entities
18
Export and Import
• Using Imported Provenance During Export Include the imported provenance as bundles in the
generated PROV graph• Bundles [13] enable nesting of PROV graphs within
PROV graphs, treating a nested graph as a new entity.Connect the entities representing input tuples in the
imported provenance to the query activity and output tuple entities
[13] P. Missier, K. Belhajjame, and J. Cheney. The W3C PROV family of specifications for modelling provenance metadata. In EDBT, pages 773–776, 2013.
20
Export and Import
• Handling Updates If a tuple is modified, that should be reflected when
provenance is exported• E.g., by running an SQL UPDATE statement
• Example Assume the user has run an update to correct tuple t1’s age value
(setting age to 70) before running the query
21
Export and Import
• Challenge How to track the provenance of updates under
transactional semantics
• SolutionGProM using the novel concept of reenactment
queries• User can request the provenance of an past update,
transaction, or set of updates executed within a given time interval
• Construct PROV document using provenance for updates computed on-the-fly
Outline
① Introduction
② Related works
③ Overview
④ Export and Import
⑤ Experimental Results
⑥ Conclusion and future work
23
Experimental Results
• TPC-H [14] benchmark datasets Scale factor from 0.01 to 10 (10MB up to 10GB size)
• Run on a machine with 2 x AMD Opteron 3.3Ghz Processors 128GB RAM 4 x 1 TB 7.2K RPM disks configured in RAID 5
• Queries Provenance of a three way join between relations customer,
order, and nation With additional selection conditions to control selectivity (and,
thus, the size of the exported PROV-JSON document).
[14] TPC. TPC-H Benchmark Specification, 2009.
Outline
① Introduction
② Related works
③ Overview
④ Export and Import
⑤ Experimental Results
⑥ Conclusions and Future Work
26
Conclusions and Future Work
Conclusions• Integrated import and export of provenance represented as
PROV-JSON into/from provenance-aware databases• Construct PROV graphs on-the-fly using SQL• Connect database provenance to imported PROV data
Future Work• Full implementation for updates• Automatic storage management (e.g., deduplication) for
imported provenance• Automatic cross-referencing
27
Questions
• My Webpage– http://www.cs.iit.edu/~dbgroup/people/xniu.php
• Our Group’s Webpage– http://cs.iit.edu/~dbgroup/research/index.html
• GProM– http://www.cs.iit.edu/~
dbgroup/research/gprom.php