FRUCT 17, Conference Opening
Workshop on Cyber-Physical-Social Systems
Yaroslavl, Russia 22.04.2015
Cyber-Physical-Social Systems
Alexey Kashevnik, e-mail: [email protected]
PhD, Senior Researcher
Laboratory of Computer Aided Integrated Systems
St.Petersburg Institute for Informatics and Automation of the
Russian Academy of Sciences (SPIIRAS), St.Petersburg, Russia
International Research Laboratory
«Intelligent Technologies for Socio-Cyberphysical Systems»,
ITMO University, St.Petersburg, Russia
Table of Contents
Motivation
Cyber-Physical and Cyber-Physical-Social Systems
Development Technologies
Smart Space
Context for CPSS
Data, Information, Knowledge
Semantic Web, Ontologies
Ontology Matching Method
Smart Spaces
Smart-M3 Platform
Access Control Model for Smart Space
Application Examples
2
Motivation
Device interaction technologies
which allows to achieve maximum
profit: Internet of Things (IoT), World
Wide Sensor Network, Smart
Building Environment, and etc.
Limitation of cognitive human
abilities, which progress slowly than
computers (usually at the moment
human can not keep in mind all
information he/she need for making
decisions). 3
Increasing number of devices with built-in
processors and data storage possibilities
(more than 50 billions of devices, more than 5 billions of smartphones will be available in 10 years).
Top 12 Technologies by McKinsey Global
Institute (May 2013)
4
Source: McKinsey Global Institute, Report MGI “Disruptive technologies: Advances that will
transform life, business, and the global economy” (May 2013).
http://www.mckinsey.com/insights/business_technology/disruptive_technologies
Modern Technologies
5
Mobile Internet Increasingly inexpensive and capable mobile
computing devices and Internet connectivity
Automation of knowledge
work
Intelligent software systems that can perform
knowledge work tasks involving unstructured
commands and subtle judgments
The Internet of Things Networks of low-cost sensors and actuators for
data collection, monitoring, decision making, and
process optimization
Cloud computing Use of computer hardware and software
resources delivered over a network or the
Internet, often as a service
Advanced robotics Increasingly capable robots with enhanced
senses, dexterity, and intelligence used to
automate tasks or augment humans
From Industry 1.0 to Industry 4.0
6
Source: German Research Center for Artificial Intelligence (2011)
Cyber-Physical Systems (CPS)
7
“Cyber-physical systems are physical and engineered
systems whose operations are integrated, monitored,
and/or controlled by a computational core.
Components are networked at every scale.
Computing is deeply embedded into every physical
component, possibly even into materials. The
computational core is an embedded system, usually
demands real-time response, and is most often
distributed. ”
Helen Gill, NSF, USA
Cyber-Physical Systems or “smart” systems are co-engineered interacting
networks of physical and computational components. These systems will
provide the foundation of our critical infrastructure, form the basis of emerging
and future smart services, and improve our quality of life in many areas.
National Institute of Standards and Technology, USA
Difference Between CPS and CPSS
8
People
Interacted devices
Physical devices
Cyber-Physical-Social Systems (CPSS)
CPSS in contrast with CPS consist of not only cyberspace and physical
space, but also human knowledge, mental capabilities, and
sociocultural elements.
9
Physical world IT world
Social Networks
Cyber-
Physical-
Social
Systems
Example of CPSS
10
Communication harvester – back-office
o Cut tree; measure length, diameter, wood quality.
o Check markets prices and customer orders.
o Decide on usage of the tree (which product, what customer).
o Cut according to back-office decision and mark product.
Smart Space for CPSS
11
Semantic Integration
CPSS Smart Space Social Space Physical World
Semantic Integration
Smart Space – computational environment consisting of
multiple heterogeneous resources (electronic and
computational devices, Internet pages, data based, etc.) which
has intelligent behavior and can proactively provide services
taking into account current situation.
Types of Resources in CPSS
12
Cyber-Physical Social System
Acting
Resources
Computational
Resources
Social Network
Information
Resources
Physical
Environment Smart Space
Social Space
Crowdsourcing
13
*) Jeff Howe “Crowdsourcing: A Definition” http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html **) Yano Research Institute: http://www.yanoresearch.com/press/press.php/001161
Crowdsourcing market size in Japan**
“The act of a company or institution taking a function once performed
by employees and outsourcing it to an undefined (and generally large)
network of people in the form of an open call” (Jeff Howe*)
Crowd Computing
14
Crowd computing* – “an umbrella term to define a
myriad of tools that allow human interaction to exchange
ideas, non-hierarchical decision making and full use of
mental space of the globe”.
Characteristics**:
A crowd of humans
Computer-mediated interaction
Purposive crowd activity
Task utilize human capabilities
(optional) Harnessing collective intelligence
*) Schneider D, de Souza J, Moraes K. Multidões: a nova onda do CSCW? In Brazilian
Symposium on Collaborative Systems, 2011.
**) Parshotam K. Crowd computing: a literature review and definition. In Proceedings of the
South African Institute for Computer Scientists and Information Technologists Conference,
2013.
Example of Crowdsourcing-Based CPSS
15
Source: http://www.ecouterre.com/ca-debuts-clothes-hangers-that-display-facebook-likes-in-real-time/
Can’t decide which shirt to get? Let Facebook be the judge. C&A introduced a
high-tech hanger that tallies the number of Facebook “likes” an item of clothing
on its racks receives.
Introduction: Context in CPSSs
16
CPSSs are expected to be context-aware.
Context is any information that can be used to characterize the situation of an entity. An entity is a person, place or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves [Dey et al. 2000].
Context is described as an
ontology-based model specified
for actual settings. Multiple
sources of
data/information/knowledge
provide information about the
actual settings.
Fundamental categories for context information Zimmermann, A., Lorenz, A., Oppermann, R.: An Operational Definition of
Context. In: Kokinov, B. et al. (eds.) CONTEXT 2007. LNAI, vol. 4635, pp.
558–571. Springer-Verlag, Berlin, Heidelberg (2007)
Data, Information, Knowledge, Wisdom
17
Knowledge
Information
Data Symbols, words, numbers, …
Processed data in a context
Information Interpretation
Wisdom
«I know one thing: that I know nothing», Socrates
Applied
Context
Meaning
Raw 4, red
4 wheels
Red Car
Passenger car
has 4 wheels
It is better to stop
the car in this
situation
Adapted from: http://blog.falkayn.com/2011/03/is-knowledge-all-there-is.html
Examples: Data, Information, Knowledge,
Wisdom
18
Source: http://okcon.org/2013/08/14/okcon-2013-guest-post-open-data-and-increasing-the-impact-of-research-its-a-piece-of-cake/
Semantic Web
19
• Increasing number of Internet
pages and services.
• A lot of information is accessible
for the user.
• Different Internet Services has to
understand each other.
Semantic web allows to make possibility of
processing and “understandable” a web data
by machines.
Berners-Lee, Tim; James Hendler; Ora Lassila
(May 17, 2001)
Web Evolution
20
Years
Inte
llig
ence
Настоящее время 1990 гг
Adapted from: https://blog.law.cornell.edu/voxpop/category/legal-semantic-web/
Possible Scenarios of Services and Devices
Interaction in the Internet
21
• Please, order this book for me in the nearest
library.
• Taking into account public transport schedule and
my calendar, please book tickets for football
match.
• Please, advice and order a vine for these dishes in
vine shop (take into account that my wife does not
like Muscat).
• Microwave, please, download from manufacturer
web page heating parameters for these dishes.
Semantic Web Models. Ontology.
22
• XML (eXtensible Markup Language) determines syntax
and semantics of information in the Internet.
• RDF(S) (Resource Description Framework) – language
for recourse description.
• OWL (Ontology Web Language) – ontology language
based on RDF.
An ontology, is a domain vocabulary complete with a set
of precise definitions, or axioms, that constrain the
meanings of the terms sufficiently to enable consistent
interpretation of the data that use that vocabulary.
Графический Пример RDF(S) модели
23
http://www.foo.com/W.Smith http://www.foo.com/S.Smith
rdf: object
William Smith
First Name Last Name
Smith Susan
First Name Last Name
vat: Maried_With
rdf: subject
rdf: predicate
rdf: Statement rdf: type
vat: Holy_Father
vat: confirmedBy
vat: - http://vatican.va/
OWL: Main Concepts
24
Individuals is an elements of classes; properties link them
with each other. E.g., individual Smith can be described
as element of class person.
Two types of properties:
owl:ObjectProperty – object property, that links individuals
owl:DatatypeProperty – data property that links individuals
with values.
OWL Example
25
Megafone
company
Moskow
St.Petersburg
William Smith
Susan Smith
Organization
City
Person
hasMainOffice
hasAdditionalOffice
employ
workTogether
employ
Ontology Matching: Generic Scheme
26
Comparison of elements of two
ontologies using similarity-
based method
Comparison of elements of two
ontologies using semantic-based
distances search method
Graph-based distance
improvement
Comparison of elements of two
ontologies using synonyms
Linguistic
Contextual
Combined
Method class Matching model
S. Balandin, S. Boldyrev, I. J. Oliver, T. Turenko, A.V. Smirnov, N. G. Shilov, A. M. Kashevnik “Method and
apparatus for ontology matching”, US 2012/0078595 A1, March 2012.
Ontology Matching: Similarity-Based
Method
27
Improved FSC
Stemming Conventional Fuzzy string
comparison (FSC)
Normalization
“looking” “look”,
“device” “devic”,
“vertical” “vertic”
Normalization of
estimations Classical Algorithm:
Calculation of substrings of first string
entries to the second string:
“DTSTART” и “START_D”
Count of substrings of first string found in
the second: 16
Overall Count of substrings: 28
Similarity: 57%.
Улучшенный:
FC1 = FuzzyCompare(Element1, Element2)
FC2 = FuzzyCompare(Element2, Element1)
Re’=n*FC1+(1-n)*FC2
Ontology Matching: Semantic-Based
Distance Search Method
Empirically defined weights:
Semantic distance:
28 23.08.2010
Graph-Based Distance Improvement: Idea
Set X=(x1, x2, ..., xn) is the set of subjects and objects in the ontology of two knowledge processors
Set Dx = (d(xi, xj), ...) is a degree of similarity between xi and xj
Set R = (r1, r2, ..., rn) is a set of predicates in the ontology of two knowledge processors
Set Dr = (d(ri, rj), ...) is a set of degrees of similarity between ri and rj
Constant Tr is a threshold value that determines whether two ontology elements mapped to each other or not
29 23.08.2010
Graph-Based Distance Improvement:
Algorithms
For RDF Subjects and Objects:
For RDF Predicates
30 23.08.2010
Smart Space
31
Smart Space is service-oriented infrastructure which
provides possibilities of information sharing by different
devices and meet the following properties:
- Devices have to be integrated in the space or
dynamically join and leave it.
- Devices have to provide personalized user support.
- Devices have to take into account current situation in
smart space.
- Devices have to be adaptive (they have to respond to
other devices and users actions).
- Devices have to provide proactive behavior (provide
the user usefull at the moment services without
explicit query).
Smart-M3 Platform for Smart Space-Based
Application Development
Smart-M3 includes: SIB: Devices and software entities (applications) can publish their
embedded information for other devices and software entities through simple, shared Service Information Brokers.
The interface for managing information in the SIB is provided by Knowledge Processors (KP)
The understandability of information is based on the usage of the common RDF ontology models and common data formats.
September, 2010 32
Smart Space
SIB
Knowledge
base
…
Application
Application
KP
Ontology
KP
Ontology
Smart-M3 allows
user KP to: add,
remove,
change, and
subscribe,
on information in SIB.
Access Control Model For Smart Space
33
Access Control Service
Access Control Rules
Roles
Trust Values Permissions
Public Smart
Space
Private Smart
Space Smart Space Service
User
Context
Information
Request
Information
Request
Private Information
Flow
Private
Information
Flow
Access
Response
Access
Request
Access
Request
Access
Response
Cyber-Physical-Social System Example:
Tourist Assistant – TAIS
34 34
Tourist Attraction
Recommendation
Service
Location
Preferences
Geo2Tag
Search
https://play.google.com/store/apps/details?id=ru.nw.spiiras.tais
Recommended Attractions
Context Service Recommendation
Service
Attraction Information
Service
TAIS: UML Sequence Diagram of Services
Interaction
35
Client App.
Sharing tourist contextinformation (location,
preferences, ...)
Notification aboutchanges in the tourist
context
List of attractions nearby the tourist
Query internal identifier for an attraction and insert new one
Attraction identifier
Query default image
Default image
SS AIS RS Go2TagExternal Sources
Images Database
Context Service
Query attraction list nearby the tourist in radius R
Loop: for each external source
Loop: for each attraction
List of attractions withidentifiers and default
images from internal DB
Notification about accessible list of attractions and region context
Notification about changes in the tourist context
Context of the tourist location region
List of attractions ordered by recommendation service
Notification aboutordered attractions withdefault images accessible
Notification aboutordered attraction
accessible
Query I default images nearby the tourist
Default ImagesList of attractions ordered by recommendation service
with default images
Cyber-Physical-Social System Example:
Home Cleaning Scenario
Home cleaning scenario.
Devices:
User Mobile Device (schedule, preferences)
Robot vacuum cleaner (e.g. Yujin
Robot iClebo Arte or iRobot Roomba)
“Smart home” systems (illumination control, information
network, grid network, etc.)
Manipulating robot (e.g.
FESTO Robotino XT)
36
Home Cleaning:
Two Simplified Scenarios
Robots Interaction Scenario
Two or more robots receive a task to execute actions, e.g. find an
object and bring it to a storage.
Only one robot should handle this task.
Robots should interact to find the one who will bring the object to the
storage.
Pick-and-place scenario
The system solves the task of pick-and-place an object from one
point to another
Two types of robots participate in the system scenario:
- Pipeline robot (can scan object’s characteristics, can provide the
object to the end of pipeline)
- Manipulating robot (can take the object and move it to another
place based on object’s characteristics)
37
38
Lego® Mindstorms EV3 Kit
Large motor
Large
motor
Main block
Medium
motor
Ultrasonic
sensor
Touch
sensor Gyroscopic
sensor
Color
sensor
Lego Mindstorms Robot Example
39
Control block
Gyroscopic sensor
Wi-Fi USB-adapter
Ultrasonic sensor
Large motor
40
Robot’s Interaction in Smart Space Based
on Ontology Matching
Service2
Ontology
Matching Service Smart Space
Ontology Library
Ontology2
Service1
Ontology1
Smart Space
Robot1
Robot2
Smart
Space
Robots Interaction Scenario Live Demo
41
40 cm
30 cm
50 cm
Robot 1:
30 cm = 30 cm => It’s another robot.
40 cm – distance to object.
40 cm < 50 cm => I have to go to object 30 cm
Robot 2:
30 cm = 30 cm => It’s another robot.
50 cm – distance to object.
50 cm > 40 cm => I have to stay here
Cyber-Physical Systems Example: Self-
Organization in Industrial Systems
42
Service Linear Drive
Service Gripper
Interaction / Self-Organization
Ethernet
Ethernet
Unix
Android
Physical Space
Controller
Controller
Smart Space
Wi-Fi
Wi-Fi
Industrial Robots Interaction: Screenshots
43
Industrial Robots Interaction in the Cyber-
Physical System
44
Controller 2 Controller 1
Physical Interaction
Service 1 Service 2
Smart Space
Smart Space
Ontology Library
SSAP SSAP
REST API REST API
Dispatcher 1 Dispatcher 2
Executive
module 2
Pick-and-place robot Manipulating robot
Ontology 2 Ontology 1
STL STL Executive
module 1
Thank you for Attention
Questions are Welcome
45
Alexey Kashevnik, PhD
St. Petersburg, Russia, E-mail: [email protected]