Date post: | 22-Apr-2015 |
Category: |
Education |
Upload: | hong-linh-truong |
View: | 346 times |
Download: | 0 times |
Evaluating Data Concerns for DaaS
Hong-Linh Truong
Distributed Systems Group,
Vienna University of Technology
[email protected] http://www.infosys.tuwien.ac.at/staff/truong
1 ASE WS 2012
Advanced Services Engineering,
WS 2012, Lecture 5
Outline
Data concern-aware DaaS service engineering
Data concern evaluation
Data concern publishing
A Proof-of-concept: QoD Framework
ASE WS 2012 2
....
Recall -- DaaS Concerns
ASE WS 2012 3
data DaaS .... data assets
Data
concerns
Quality of
data Ownership
Price License ....
APIs, Querying, Data Management, etc.
DaaS concerns include QoS, quality of data (QoD),
service licensing, data licensing, data governance, etc.
4
Recall -- DaaS design &
implementation
Data
items
Data
items
Data
items
Data resource
Data
assets
Data resource Data resource
Data resource Data resource
Consumer
Consumer
DaaS
ASE WS 2012
HOW TO EVALUATE DATA
CONCENRS FOR DATA
ASSETS IN DAAS?
ASE WS 2012 5
Patterns for „turning data to DaaS“
ASE WS 2012 6
Storage/Database
-as-a-Service data DaaS
Storage/Databa
se/Middleware
data
Things DaaS
Storage/Database/
Middleware
data
People DaaS
DaaS data Build Data
Service
APIs
Deploy
Data
Service
Data-related activities
ASE WS 2012 7
Wrapping
data
Publishing DaaS
interface
Typical activities for data wrapping and publishing
Typical activities for data updating & retrieval
Updating
data
Selecting
data data
Provisioning
data
Typical data concern evaluation
ASE WS 2012 8
Evaluating data
concerns
Describing data
concerns
Data Concerns
Evaluation Tools Data Concerns
Representation Models
Populating data
concerns
Publishing services
What do we need in order to perform these activities?
9
Data concern-aware DaaS
engineering process Typical activities
for data wrapping
and publishing
Typical activities
for data updating &
retrieval
ASE WS 2012
Hong Linh Truong, Schahram Dustdar: On Evaluating and Publishing
Data Concerns for Data as a Service. APSCC 2010: 363-370
DaaS service operation
Wrapping, selecting, and updating
data in DaaS (1)
10 ASE WS 2012
Processing
parameter
Mapping parameters to
data queires parameter
Query content of
data resources
Mapping and
returning results
Mapping parameters to
metadata queries
Querying metadata of
data resources
Data
Consumer
different strategies for structured data and unstructured data
Wrapping, selecting, and updating
data in DaaS (2)
Different techniques exist for wrapping,
selecting, updating and retrieving data
How generic data concern evaluation and
publishing techniques can be integrated with
these techniques?
11 ASE WS 2012
WHICH TYPES OF DATA ARE NEEDED FOR
EVALUATING DATA CONCERNS?
WHAT IS THE IMPACT OF DATA
PROVISIONING MODELS (OFFLINE
VERSUS NEAR-REALTIME) ON CONCERN
EVALUATION/PUBLISHING?
Discussion
ASE WS 2012 12
Evaluating data concerns – the
three important points
13
• At which level the evaluation is performed?
evaluation scope
• When the evaluation is done?
evaluation modes
• How the evaluation tool is invoked?
integration model
ASE WS 2012
Hong Linh Truong, Schahram Dustdar: On Evaluating and Publishing Data Concerns for Data as a Service. APSCC
2010: 363-370
Evaluating data concerns –
evaluation scopes
Three scopes
data resource
DaaS operations
DaaS as a whole
14
Why multiple evaluation scopes make sense?
enable fine-grained evaluation
ASE WS 2012
Evaluating data concerns –
evaluation modes
Off-line
before the access to data
On-the-fly
when the data is requested
15
Why multiple evaluation modes make sense?
suitable for different types of data
ASE WS 2012
Evaluating data concerns –
integration modes
Push and pull data concerns
Pass-by-value versus pass-by-reference to data
concerns evaluation tools
16
Why multiple integration modes make sense?
suitable for different tool integration strategies
ASE WS 2012
Evaluating data concerns – some
patterns (1)
17
Pull, pass-by-references
ASE WS 2012
Evaluating data concerns – some
patterns (2)
18
Pull, pass-by-values
ASE WS 2012
Evaluating data concerns – some
patterns (3)
19
Push, pass-by-values
ASE WS 2012
BASED ON WHICH CRITERIA, AN EVALUATION
SCOPE, EVALUATION MODE OR INTEGRATION
MODE IS SELECTED?
Discussion time
ASE WS 2012 20
WHICH ARE OTHER COMPONENTS INTERACTING
WITH EVALUATION TOOLS?
WHY WE DO NOT REALLY DISCUSS HOW TO
IMPLEMENT EVALUATION TOOLS?
Publishing data concern
information (1)
Off-line publishing of data concerns
suitable for static data concerns
the publishing of data concerns of a data
resource is separated from the service
operation which provides the access to the
data resource
ASE WS 2012 21
Publishing data concern
information (2)
On-the-fly publishing of data concerns
associating concerns with retrieved data
resources
the resulting data resources (e.g., via queries)
are annotated with data concerns evaluated
by data concerns evaluation tools.
suitable for providing dynamic data concerns
ASE WS 2012 22
23
Publishing data concern
information (3)
On-the-fly publishing of data concerns through
queries
the use of different service operation
parameters to query data concerns of data
resources
suitable for validating data concerns before
accessing data resources
ASE WS 2012
WHAT ARE THE RELATIONSHIPS BETWEEN
CONCERN EVALUATION AND PUBLISHING
WHEN DATA IS DYNAMICALLY UPDATED?
Discussion time
ASE WS 2012 24
How do we utilize the data concern-
aware service engineering process?
Using this model we can determine and publish
several concerns
Our “a proof-of-concept”
A framework for evaluating and publishing QoD of
DaaS
A proof-of-concept implementation of data concern-
aware service engineering process
Another example: model and publish privacy
concerns for DaaS [ECOWS 2010]
ASE WS 2012 25
Michael Mrissa, Salah-Eddine Tbahriti, Hong-Linh Truong, "Privacy model and annotation for DaaS", The 8th European
Conference on Web Services (ECOWS 2010), (c)IEEE Computer Society, 1-3 December, 2010, Ayia Napa, Cyprus
26
QoD framework (1)
Pull QoD Evaluation Models for DaaS
Pass-by-references and pass-by-value
References of data resources: URI
Values: any object
Third-party data evaluation tools
ASE WS 2012
27
QoD framework (2)
ASE WS 2012
http://www.infosys.tuwien.ac.at/prototype/SOD1/dataconcerns/
28
QoD framework: publishing
concerns (1)
Off-line data concern
publishing
a common data concern
publication specification
a tool for providing data concerns
according to the specification
supported by external service
information systems
ASE WS 2012
QoD framework: publishing
concerns (2)
On-the-fly querying data concerns associated with data
resources
Using REST parameter convention
Based on metric names in the data concern
specification
ASE WS 2012 29
Hong Linh Truong, Schahram Dustdar, Andrea Maurino, Marco Comerio: Context, Quality and Relevance:
Dependencies and Impacts on RESTful Web Services Design. ICWE Workshops 2010: 347-359
QoD framework: publishing
concerns (3)
Specifying requests by using utilizing query parameters
the form of metricName=value
30
Obtaining contex and quality by using context and quality
parameters without specifying value conditions
GET/resource?crq.accuracy="0.5"&crq.location=’’Europe”
curl http://localhost:8080/UNDataService/data/query/Population annual growth rate
(percent)?crq.qod
{”crq.qod” : {
”crq.dataelementcompleteness ”: 0.8654708520179372,
”crq.datasetcompleteness”: 0.7356502242152466,
...
}}
ASE WS 2012
31
QoD framework: QoD monitoring
and composition
QoD concerns monitoring and composition are
useful for the evaluation of aggregated data
resources
Our approach
Utilizing monitoring rules
QoD metrics of data resources are passed to an rule
engine
Rules are user-defined for monitoring and composing
QoD metrics
ASE WS 2012
QoD framework experiments
Implementation
Java, JAX-RS/Jersey, Drools
Utilizing UNDataAPI - www.undata-api.org
XML data sets without QoD
Illustrating examples: check data from 1990-
2009
datasetcompleteness: the completeness of the list of
countries
dataelementcompleteness: the completeness of data
elements in the list metrics
RESTful services wrapping to UNDataAPI
ASE WS 2012 32
33
QoD framework experiment:
evaluating and annotating QoD
metrics
ASE WS 2012
34
QoD framework experiments:
publishing QoD with data
resources
ASE WS 2012
35
QoD framework experiments:
simple rules for monitoring and
composing QoD
ASE WS 2012
HOW TO DEAL WITH OTHER
CONCERNS?
Discussion time
ASE WS 2012 36
Exercises
Read mentioned papers
Identify and analyze the relationships between
data concerns evaluation tools and types of data
Analyze trade-offs between on-line and off-line
evaluation and when we can combine them
Analyze how to utilize evaluated data concerns
for optimizing data compositions
Analyze situations when software cannot be
used to evaluate data concerns
ASE WS 2012 37
38
Thanks for your attention
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
http://www.infosys.tuwien.ac.at/staff/truong
ASE WS 2012