Post on 05-Dec-2014
description
transcript
Working with real data
1
Payam Barnaghi
Centre for Communication Systems Research (CCSR)
Faculty of Engineering and Physical Sciences
University of Surrey
Guildford, United Kingdom
2
Things, Data, and lots of it
image courtesy: Smarter Data - I.03_C by Gwen Vanhee
Data is not what we want or is it?
What we need are insights and actionable-knowledge
Diffusion of innovation
image source: Wikipedia
IoT
Problem #1
Data: We seem to have lots of it…
Real World Data: it is always difficult to get (silos, format, privacy, business interests or lack of interest!...)
Problem #2
Data: interoperability and metadata frameworks…
Real World Data: there are solutions for service based (Restful) access, meta-data/semantic representation frameworks (W3C SSN, HyperCat,…) but none of them are widely adapted.
Problem #3
Data: quality, reliability…
Real World Data: data can be noisy, crowed source data can be inaccurate, contradictory, delay in accessing/processing the data…
Problem #4
Data: having too much data and using analytics tools alone won’t solve the problem…
Real World Data: in addition to the HPC issues, we need new methods/solutions that can provide real-time analysis of dynamic, variable quality and multi-modal streams…
Problem #5
Data: abstraction, discovering the associations…
Real World Data: co-occurrence vs. causation; we need hypothesis, background knowledge,…After all data is not what we are really after…
We need more linked open data
(near) real-time linked open data
Streams
Sometimes it’s even better if we have:
(near) real-time linked open data
Streams+
meta-data (semantic annotations)+
Adaptable and scalable analytics tools+
Sufficient background knowledge
or even better than that if we have:
Data analytics
14
Data:
DataData
Domain
KnowledgeDomain
Knowledge
Social
systemsSocial
systemsInteractionsInteractionsOpen
InterfacesOpen
Interfaces
Ambient
IntelligenceAmbient
IntelligenceQuality and
TrustQuality and
Trust
Privacy and
SecurityPrivacy and
Security
Open DataOpen Data
15
Challenges and opportunities
− Providing infrastructure − Publishing, sharing, and access solutions on a global scale− Heterogeneity and interoperability at different layers− Indexing, query and discovery (data and resources)− Aggregation, integration and fusion− Trust, privacy and security− Data analytics and creating actionable knowledge
− Integration into services and applications in e-health, the public sector, retail, manufacturing and personalised apps.− Mobile apps, location-based services, monitoring control etc.
− New business models
− Thank you.
− EU FP7 CityPulse Project:
http://www.ict-citypulse.eu/
@ictcitypulse
p.barnaghi@surrey.ac.uk