Post on 11-Dec-2015
transcript
Query Processing in Mobile P2P Databases
IGERT Seminar PresentationBo Xu
joint work with Ouri Wolfson
April 18, 2023 IGERT seminar 2
Talk outline
Introduction System Model The MARKET Algorithm Evaluation Extension to CTS Conclusion and Future Work
April 18, 2023 IGERT seminar 3
Query Processing Environments
Query longevity
Continuous Instantaneous
Wireless ad-hocnetworks
Sensor MANET
Mobile Static
Connectivity
Sparse Dense
Query longevity
Continuous Instantaneous
Wireless ad-hocnetworks
Sensor MANET
Mobile Static
Connectivity
Sparse Dense
Motivation: a general purpose query processing strategy mobile disconnected wireless ad-hoc networks
Vehicular Sensor Network (VSN)
GPS receiverchemical spill detectorstill/video cameravibration sensoracoustic detector
April 18, 2023 IGERT seminar 4
Store-and-forward to deal with sparseness
q
Ar
Q
Q
A AqA
QA
April 18, 2023 IGERT seminar 5
Issues with Store-and-forward
How to manage limited memory, power, and bandwidth?
Which reports to save/transmit?
April 18, 2023 IGERT seminar 6
Difficulty of Store-and-forward
Assume that the trajectories of all nodes is known a priori at a central server. If memory, energy, and bandwidth are bounded at mobile nodes, then the problem of determining whether a set of data-items can be disseminated to all the mobile nodes is NP-complete.
Case: Each mobile node is interested in every data-item
Mobile P2P: Trajectories unknown a priori; Heuristics needed
April 18, 2023 IGERT seminar 7
Talk outline
Introduction System Model The MARKET Algorithm Evaluation Extension to CTS Conclusion and Future Work
April 18, 2023 IGERT seminar 8
Mobile P2P Database
report 8
query C
query Breport 4report 5
Local database
Local query query Areport 1report 2report 3
Pda’s, cell-phones, sensors, hotspots, vehicles, with short-rangewireless capabilities
A
B
C
• Applications coexist• Variable report sizes• A peer can be a produce, consumer, and broker
April 18, 2023 IGERT seminar 9
Queries
A query Q maps each report R to a match degree:
Examples: Top parking slots given my current location
Profile with expertise “children-periodontics” Similarity between two images
1),(0 QRmatch
match(R,Q)=e-t-d
April 18, 2023 IGERT seminar 10
Query/report Dissemination
Two peers within transmission range exchange queries and reports
Least relevant reports that do not fit in local broker database are purged
Exchange not necessarily synchronous (periodic broadcast)
April 18, 2023 IGERT seminar 11
Talk outline
Introduction System Model The MARKET Algorithm Evaluation Extension to CTS Conclusion and Future Work
April 18, 2023 IGERT seminar 12
Ranking Factors
Rank of a report R is determined by
Demand: What fraction of peers are querying R Probability that a peer is interested in R
Supply: What fraction of peers already have R Probability that a peer has R
Size of R
April 18, 2023 IGERT seminar 13
Rank of a report
expected benefit = demand(R)*(1supply(R))
reports database 0.3
0.8
0.5
0.4
0.5
0.7
reports benefit
0.5
0.7
Rank(R)=size(R)
demand(R)*(1supply(R))
April 18, 2023 IGERT seminar 14
Report Ranking: sample demand
q3q4
q1q7 O1
O3
O2
(a)
O3
O1
O2
(b)
r2r5
r1r4
q2q6
r3r6
r7r8
q5q8
q3q4
r1r4r7r8
q5q8
q1q3
r1r2r5r4
q4q7
q2q4
r4r3r6r1
q3q6
reportsrel ati on
queri esrel ati on
Queries relation is FIFO maintained
April 18, 2023 IGERT seminar 15
Rank of Reports
Demand for R
Qi’s are the members of the queries relation
Size of the queries relation determined based on Hoeffding’s inequality
n
QRmatchn
ii
1
),(
2)width_interval_confidence(221_levelconfidence neE.g., if n=108, then with 95% chance the demand estimation error is smaller than 0.08
April 18, 2023 IGERT seminar 16
How does peer O determine supply(R)?
A parametric formula giving the supply is beyond the state of the art
O machine-learns supply(R) based on meta-data of R: Age of R Number of times O sighted R from other
peers etc.
peersofnumbertotal
RhavethatpeersofnumberRsupply
___
_____)(
April 18, 2023 IGERT seminar 17
Computing Supply by Machine-learning
aro: The age rank order within O’s reports database
fin: The number of times O has sighted the report from other peers
MAchine LEarning based Novelty rAnking (MALENA)
report -idReport
descriptionaro fin
R1 … 1 1
R4 … 2 4
R2 … 3 2
R7 … 4 2
Reports database of O
report -idReport
descriptionaro fin
R1 … 1 1
R4 … 2 4
R2 … 3 2
R7 … 4 2
April 18, 2023 IGERT seminar 18
Reports database of O
report-id
Reportdescription
aro fin
R1 … 1 3
R2 … 2 1
TrackingSet of O
R2
R3
R1Advertise to peer
R1, R2
ReceiveREQ((R1,R2), R2)
Examples created by ReceiveREQ
aro fin label
1 3 old
2 1 new
insertExamples set
ES of O
ReceiveRPT(R4)
Reports database of O
report-id
Reportdescription
aro fin
R1 … 1 4
R4 … 2 1
TrackingSet of O
R2
R3
R1
Reports database of O
report-id
Reportdescription
aro fin
R1 … 1 4
R2 … 2 1
R2
R3
R1
TrackingSet of O
R4
(a)
(b)
(c)
Reports database of O
report-id
Reportdescription
aro fin
R1 … 1 3
R2 … 2 1
TrackingSet of O
R2
R3
R1Advertise to peer
R1, R2
ReceiveREQ((R1,R2), R2)
Examples created by ReceiveREQ
aro fin label
1 3 old
2 1 new
insertExamples set
ES of O
ReceiveRPT(R4)
Reports database of O
report-id
Reportdescription
aro fin
R1 … 1 4
R4 … 2 1
TrackingSet of O
R2
R3
R1
Reports database of O
report-id
Reportdescription
aro fin
R1 … 1 4
R2 … 2 1
R2
R3
R1
TrackingSet of O
R4
(a)
(b)
(c)
MALENA
B
B
Request R2
Examples created
positive
negative
April 18, 2023 IGERT seminar 19
MALENA Implementation Considerations
Minimize overhead No need to actually store examples Model incrementally built
Bayesian learning a simple but effective method
April 18, 2023 IGERT seminar 20
Talk outline
Introduction System Model The MARKET Algorithm Evaluation Extension to CTS Conclusion and Future Work
April 18, 2023 IGERT seminar 21
0
20
40
60
80
100
120
140
0 30 60 90 120 150 180 210 240 270 300
thro
ughp
ut (m
atch
es/p
eer)
response-time bound (second)
peer density (D) = 20, energy allocation fraction (F) = 0.15 report production rate (P) = 0.1 reports/second
MARKETRANDI
Ideal Benchmark
mobility model=random way point, average motion speed=1 mile/hourtransmission range=100 meters, mean of reports database size=100Kbytesqueries database size=100 queriesreport size uniformly distributed between 1K and 2K bytes0.1 report produced per second
Comparison with RANDI (MDM’07)
0
20
40
60
80
100
120
140
0 3600 7200 10800 14400 18000 21600 25200 28800
thro
ughp
ut (m
atch
es/p
eer)
response-time bound (second)
peer density (D) = 1, energy allocation fraction (F) = 0.15 report production rate (P) = 0.1 reports/second
MARKETRANDI
Ideal Benchmark
RANDI=MARKET-supply
20 peers within transmission range 1 peer within transmission range
MARKET half as good as ideal benchmarkMARKET twice better than RANDI
April 18, 2023 IGERT seminar 22
Comparison with LRU and LFU
2
2.5
3
3.5
4
4.5
5
5.5
0 10 20 30 40 50
MARKET
RANDI
LRU
LFU
response-time bound (second)
thro
ug
hput
(matc
hes/
peer)
mobility model=iMotes tracesmean of reports database size=150Kbytesqueries database size=10 queriesreport size uniformly distributed between 2K and 20K bytes0.1 report produced per second, transmission size=100Kbytes
(results obtained by Fatemeh Vafaee)
April 18, 2023 IGERT seminar 23
Evaluation of MALENA (TAAS’09)turn-over: peers enter/exit systeminjection: number of peers that have a report initially
mobility model=iMotes traces, reports database size=100 reports2 reports produced per second, transmission size=10 reports
MALENA always follows the best indicator
0
2000
4000
6000
8000
10000
12000
14000
0 5 10 15 20 25 30 35 40 45 50 55 60
thro
ughp
ut (r
epor
ts/p
eer)
response-time bound (minute)
low-turn-over/low-injection Bluetooth, M=100, f=2, q=0.1, flooding
MALENAaro-rankingfin-ranking
low-turn-over/low-injection
0
20
40
60
80
100
120
140
160
180
200
0 5 10 15 20 25 30 35 40 45 50 55 60
thro
ughp
ut (r
epor
ts/p
eer)
response-time bound (minute)
high-turn-over/high-injection Bluetooth, M=100, f=2, q=0.1, flooding
MALENAaro-rankingfin-ranking
low-turn-over/low-injection
0
20
40
60
80
100
120
140
160
180
200
0 5 10 15 20 25 30 35 40 45 50 55 60
thro
ughp
ut (r
epor
ts/p
eer)
response-time bound (minute)
high-turn-over/high-injection Bluetooth, M=100, f=2, q=0.1, flooding
MALENAaro-rankingfin-ranking
high-turn-over/high-injection
April 18, 2023 IGERT seminar 24
Application: K-nearest-neighbors
Query: K-nearest-neighbors of a fixed location (query-point) Reports: current locations of mobile sensors match(Q,R): in reverse proportion to the distance from query
point
sink
query-point
April 18, 2023 IGERT seminar 25
Itinerary based KNN processing
Phase I: Query delivered to the sensor closest to query point
Phase II: Query traverses an itinerary to collect answers
Phase III: Answers returned to sink
April 18, 2023 IGERT seminar 26
Simulation Results
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6
peer density
ac
cu
rac
y
MARKET
Upper bound of itinerary
mobility model=random way point, average motion speed=1 mile/hourtransmission range=100 meters report size=24 bytes, query size=16 bytesmean of reports database size=100 reportsone location report produced at each sensor per second
MARKET is especially suitable for sparse environments
April 18, 2023 IGERT seminar 27
Talk outline
Introduction System Model The MARKET Algorithm Evaluation Extension to CTS Conclusion and Future Work
April 18, 2023 IGERT seminar 28
TrafficInfo: Disseminating Traffic Information in VANET’s
April 18, 2023 IGERT seminar 29
What does relevance mean in TrafficInfo
A
B
A
B
A report is relevant if it changes the route
April 18, 2023 IGERT seminar 30
Which factors indicate relevance of report?
Distance to the reported road segment
Type of road segment Speed variance …
April 18, 2023 IGERT seminar 31
Conceptual Learning Procedure
An example is created for a received report
The example is labeled positive if the report changes route and negative otherwise
Individual vs. group How to deal with aggregation?
April 18, 2023 IGERT seminar 32
Query processing
Conclusion
Store-and-forward enables in-network processing in mobile disconnected networks
Ranking is important for dealing with memory, bandwidth, and energy constraints
sensor-richenvironment
short-rangewireless
Mobile P2P+
April 18, 2023 IGERT seminar 33
Future Work
Multimedia reports Utilization of metadata
Integration of stateless and stateful approaches
Starvation/fairness
April 18, 2023 IGERT seminar 34
Thanks!
Questions?
April 18, 2023 IGERT seminar 35
802.11 Basics
3 modes: transmitting, receiving, listening (order of power consumption)
When listening: if detecting a message destined to host receive-mode
Time divided into slots, 20microsecs each
Transmission: Listen for 1 time slot If channel free start broadcast (observe
collision possible) Broadcast may last for many time slots
April 18, 2023 IGERT seminar 36
Energy Efficiency of a Broadcast
X
Throughput (Th) = (expected number of neighbors that successfully receive broadcast) (broadcast size)
Power efficiency (PE) = En
Th
successfully receive the broadcast from x
Collisions occur at neighbor
April 18, 2023 IGERT seminar 37
Computation of Throughput
X Y
1. No green node inside starts to broadcast at the same time slot with X
2. No transmission from any purple node overlaps with that from X
Conditions for successful reception at an arbitrary node Y
April 18, 2023 IGERT seminar 38
Energy Constraints
Energy consumed by a 802.11 network interface for transmitting a message of size M bytes
En=fM+g
For 802.11 broadcast, g=26610-6 Joule, f=5.2710-6Joule/byte
April 18, 2023 IGERT seminar 39
Experimental MP2P Projects (Pedestrians)
7DS – Columbia University (web pages) iClouds – Darmstadt Univ. (incentives) MoGATU – UMBC (specialized query processing,
e.g., collaborative joins) PeopleNet – NUS, IIS-Bangalore (Mobile
commerce, information type location baazar)
MoB – Wisconsin, Cambridge (incentives, information resources e.g. bandwidth)
Mobi-Dik – Univ. of Illinois, Chicago (brokering, physical resources, bandwidth/memory/power management)
April 18, 2023 IGERT seminar 40
Vehicular Projects
Inter-vehicle Communication and Intelligent Transportation:
CarTALK 2000 is a European project VICS (The Vehicle Information and Control System) is a
government-sponsored system in Japan with an 11-year track record
FleetNet, an inter-vehicle communications system, is being developed by a consortium of private companies and universities in Germany
IVI (Intelligent Vehicle Initiative) and VII (Vehicle Infrastructure Integration), the US DOT
MP2P provides data management capabilities on top of these communication systems
Grassroots, TrafficView, SOTIS, V3 – P2P dissemination of traffic info to reduce travel times
April 18, 2023 IGERT seminar 41
RANk-based DIssemination (RANDI)
Ranking of reports Bandwidth/energy aware Exchange enhances
Consumer functionality Broker functionality
Consumer: Answer local query (pull) Broker: Transmit reports most likely requested
by future-encountered peers (push) Transmission trigger:
Encounter New reports
April 18, 2023 IGERT seminar 42
RANDIWhen two peers meet they conduct a two-phase exchange:
local query
answers
more reports
satisfied as a consumer (pull)
enhanced as a broker (push)
Phase 1: Exchange queries and receive answers (pull)Phase 2: Exchange more reports using available energy/bandwidth (push)
Phase 1
Phase 2
Combination of: unicast (thin line) and broadcast (thick lines) to enable overhearing.
April 18, 2023 IGERT seminar 43
RANDI (Cont’d)
To solve problem with static peers:Two interaction modes which combine pull and push
• Query-response: triggered by discovery of new neighbors
• Relay: triggered by receipt of new reports Disseminate to existing neighbors
new reports
April 18, 2023 IGERT seminar 44
query
queryquery
query
reports
reports
7DS
P2P mode: each node periodically broadcasts its query and receivesreports from neighboring peers. No strategy to determine query frequency and transmission size. Cache management based on web-page expiration time.
April 18, 2023 IGERT seminar 45
PeopleNet
before exchange after exchange
Peer A Peer B Peer A Peer B
random-spread
before exchange after exchange
Peer A Peer B Peer A Peer B
random-swap
Reports are randomly selected for exchanging and saving upon encountering.
April 18, 2023 IGERT seminar 46
query
queryquery
query
reports
reports
7DS
Each peer periodically broadcasts its query and receives reports from neighboring peers. No strategy to determine query frequency and transmission size. Cache management based on web-page expiration time.
April 18, 2023 IGERT seminar 47
PeopleNet
before exchange after exchange
Peer A Peer B Peer A Peer B
Reports are randomly selected for exchanging and saving upon encountering.
April 18, 2023 IGERT seminar 48
Mobile Local Search: Applications
transportation Announce sudden stop, malfunctioning brake light, patch of ice Floating car data Dissemination of multi-media traffic information (picture, video,
voice) Search close-by taxi customer, parking slot, ride-share
social networking (wearable website) Personal profile of interest at a convention Singles matchmaking Floating BBS
mobile electronic commerce Sale on an item of interest at mall Music-file exchange
emergency response Search for victims in a rubble
asset management and tracking Sensors on containers exchange security information => remote
checkpoints tourist and location-based-services
Closest ATM
April 18, 2023 IGERT seminar 49
Applications – Common features
Mobile/stationary peers Resources of interest
in a limited geographic area Short time duration
Can be solved by fixed servers, but Unlikely solution Proposed mp2p paradigm can enhance
fixed solution (reliability, performance, coverage)
April 18, 2023 IGERT seminar 50
MARKETWhen two peers meet they conduct a two-phase exchange:
Local query
answers
more reports
satisfied as a consumer (pull)
enhanced as a broker (push)
Phase 1: Exchange subscriptions and receive answers (pull)Phase 2: Exchange more publications using available energy/bandwidth (push)
Phase 1
Phase 2
Combination of: unicast (thin line) and broadcast (thick lines) to enable overhearing.
April 18, 2023 IGERT seminar 51
MARKET (Cont’d)
To solve problem with static peers:Two interaction modes which combine pull and push
• Query-response: triggered by discovery of new neighbors
• Relay: triggered by receipt of new publications Disseminate to existing neighbors
new publications
April 18, 2023 IGERT seminar 52
Query in static disconnected network
q
A
A
r
In-network query processing may not be possible
Q
Q
Q
April 18, 2023 IGERT seminar 53
Query in static connected sensor network
qA
A
A
r
Data transmission delay is 0. Answer can be obtained instantaneously
Q
Q
Q
Q
QA A
A
A
A
qA
April 18, 2023 IGERT seminar 54
Query in static disconnected network
q
A
A
r
In-network query processing may not be possible
Q
Q
Q
April 18, 2023 IGERT seminar 55
Query in mobile disconnected network
q
A
A
r
One hop case
QAqA
Query processing enabled by mobility and store-and-forward
April 18, 2023 IGERT seminar 56
Query in mobile disconnected network
q
Ar
Multil-hop case
Q
Q
Query can be in network processed, but it is delayed
A
Query processing alogrithm doesn’t control motion.
The answer is disseminated only after an answer node receives query
AqA
QA
First stage: query disseminated during encounter