Date post: | 13-Mar-2016 |
Category: |
Documents |
Upload: | karleigh-vinson |
View: | 42 times |
Download: | 0 times |
1
DyMND:Robust Adaptive Coordination in
Dynamic Meshes of Networked Devices
NEST PI MeetingJune 29, 2004
Prasanta BoseAdvanced Technology CenterLockheed Martin Space Systems CompanyPalo Alto, CA
2DyMND: P. Bose, ATC, LMSSC
Outline
Robust DyMND Engineering– DyMNDSim and DyMNDES: The first generation– Lessons learned– DyMND-EE: The next generation
• Architecture• Implementation• Experiments
Adaptive Coordination in DyMND– Explicit local coordination for optimal control– Dynamic programming (reinforcement learning)– Design and validation strategy within DyMND-EE
Summary and Future Plan
3DyMND: P. Bose, ATC, LMSSC
DyMNDSim and DyMNDES – The First Generation
DyMNDSim– Discrete Event Simulation– Full-featured tool for implementing
sensing/radio models– Integration with Motes and Tossim
DyMNDES– Real-time injection and data retrieval– Scenario driven interaction – Script, save, and load emulation
event sequences – Generation of abstract mote and
target capabilities via virtual objects – 3D visualization of aggregate state
DyMNDSim server
Simulation Director
Client Proxy
Motes
Active message handler
Radio SensorsEffectors
Targetmodels
SvcCoord TaskQ
4DyMND: P. Bose, ATC, LMSSC
Lessons Learned
Translation of services from DyMNDSim to motes required manual recoding– DyMNDSim emulated TinyOS active messages and
event/task distinction, but executed Java classes Hardware nodes led to noisy, non-repeatable
experiments Debugging and monitoring required source-code
modifications Principled integration of simulation and
emulation components required for– Accurate validation of NEST systems
5DyMND: P. Bose, ATC, LMSSC
DyMND-EE – The Next Generation
Goal: End-to-end validation of sensor net software – Efficient execution of embedded source code– Realistic models for radio, sensing, actuation– Mixed models of execution for staged development
Goal: Accurate validation environment– Model of execution (concurrency) faithful to TinyOS
Goal: Scale to NEST challenges (e.g. EXSCAL)– Exploit tiered structuring via aggregate simulation and
interaction across aggregates
6DyMND: P. Bose, ATC, LMSSC
Evolving DyMNDSim To Realize DyMND-EE Goals
Based on principled composition of existing simulators Target-level simulators
– Emsim (Emstar): cross-compiled nesC in Linux user space
High-level DE/ continuous-time simulators
– Ptolemy II/VisualSense: Multiple models of computation, actor-oriented programming– Simulink: Mostly continuous time, rich control/DSP libraries
Tossim
Emsim / Emstar
Ptolemy II / VisualSense
Simulink
DyMNDSim
Proc
esso
r fid
elity
Gen
eric
mod
elin
g
DyMNDEE
7DyMND: P. Bose, ATC, LMSSC
Ptolemy II Features in DyMND-EE
Actor-oriented development environment– Actors implement operations on data – Directors specify model of computation (timing and order of
communication between actors) Multiple models of computation in a designed system
– Example: discrete-event sensor model to track evader with continuous control law
Extensions to WirelessDirector (implicit connectivity)– ComposableChannel allows modification of each token– Channel modifiers for propagation delay, deterministic /
probabilistic message loss, bit errors Extension to create distributed director
8DyMND: P. Bose, ATC, LMSSC
EmStar Features in DyMND-EE
Emstar is a flexible software framework for heterogenous NesC Code simulation using the FUSD kernel module. (http://cvs.cens.ucla.edu/emstar)
Features– Executes TinyOS/NesC code via EmTOS– HIL and simulated execution– Heterogeneous node simulations– Can use real or simulated RF channel– FUSD user space devices– Non-TinyOS dependent as any device can be
registered under FUSD – Provides a clean interface to abstract hardware.
9DyMND: P. Bose, ATC, LMSSC
DyMND-EE Concept of Operations
Graphical representation of a configuration of the DyMND-EE simulation environment
EmStar runs NesC code
Data Logging to Files for Analysis
Ptolemy Executes aggregate sensingand comm. models
DyMNDES generates simulation configuration and generates PtolemyModels.
10DyMND: P. Bose, ATC, LMSSC
DyMND-EE Architecture
Decomposes simulation into sensing/actuation “universes”
Implemented as composite actors with directors– Radio and sensors as actors– Services as Emstar processes
with proxy actors Software services execute in
Emstar (native code) or Ptolemy (prototype code)
Communicates with external processes via XML/object streams
Configurations generated from DES script using XSLT and predefined Ptolemy II actors
Ptolemy II
Emstar (Linux user space)
Emlink
MagnetometerUniverse
AcousticUniverse
Processors Radio Universe
Node 1Microphone
Node nMicrophone
Node 1EmstarProxy
Node 2EmstarProxy
Node nEmstarProxy
Node 1Radio
Node 2Radio
Node nRadio
EmstarGateway
Radio ChannelAbstraction
Radio ChannelAbstraction
Sensing ChannelAbstractions
(EmSensorServer)
Radio Channel
AbstractionNode 1
Processes
Radio Channel
AbstractionNode 2
Processes
Radio Channel
AbstractionNode n
Processes
FUSD
11DyMND: P. Bose, ATC, LMSSC
DyMND-EE Component Collaboration
1. Target emits chirp2. WirelessDirector transmits chirp
to node k mic3. Mic processes chirp4. Ptolemy forwards ActiveMessage
(AM) to node k EmstarProxy5. Proxy routes AM to Emstar
gateway6. Gateway routes AM to
EmSensorServer7. EmSensorServer forwards AM to
node k processes via FUSD8. Node k process generates
detection message9. Node k process forwards radio
AM to radio abstraction via FUSD10. Radio abstraction forwards AM to
Emstar gateway11. Node k EmstarProxy forwards
AM to node k radio12. Node k radio broadcasts alert
Ptolemy II
Emstar (Linux user space)
Emlink
Acoustic Universe Processors Radio Universe
Node 1 Mic
Noden Mic
Node 1EmstarProxy
Node kEmstarProxy
Node nEmstarProxy
Node 1Radio
Node kRadio
Node nRadio
EmstarGateway
Radio ChannelAbstraction
Radio ChannelAbstraction
Sensing ChannelAbstractions
(EmSensorServer)
Radio Channel
AbstractionNode 1
Processes
Radio Channel
AbstractionNode k
Processes
Radio Channel
AbstractionNode n
Processes
Node k Mic
Target Chirp 1
23 4
5
6
7
8
9
10
11
12
12DyMND: P. Bose, ATC, LMSSC
Outline
Robust DyMND Engineering– DyMND-Sim and DyMNDES: The first generation– Lessons learnedDyMND-EE: The next generation
• Architecture• Implementation• Staged Development Configuration• Experiments
Adaptive Coordination in DyMND– Explicit local coordination for optimal control– Dynamic programming (reinforcement learning)– Design and validation strategy within DyMND-EE
Summary and Future Plan
13DyMND: P. Bose, ATC, LMSSC
Implementing DyMND-EE: Extensions To Emstar
Primary Modifications– Interfaces into TinyOS scheduler for fine grained control of timing
and interrupt execution– New sim-component EmLink implements radio and sensing link
layer driver registration for interface into Ptolemy– Sensing components in Emstar component library are rewritten to
use new ADC.nc components that register link users. No changes are required to get NesC code working as expected in DyMND-EE.
– Additional components, including hypothetical components, are emulated to examine effects of sensor networks with actuation capabilities via Ptolemy environment
– Extensions for non-TinyOS-based node synchronization with external simulations
14DyMND: P. Bose, ATC, LMSSC
Implementing DyMND-EE: Emstar-Ptolemy Link
EmLink– Encapsulates TinyOS ActiveMessages as Ptolemy tokens– EmSensorServer implements Linux user-space proxy for Ptolemy
sensor/radio models – P-Star protocol defines send/receive, device read/write,
synchronization, sense/actuate, and configuration messages– Communication between Ptolemy-space and user space
transparent to services and actors P-Star Protocol
– Protocol between Ptolemy and EmStar– Header format includes simulation time, device ids, group ids (for spatial or
logical partitioning of EmStar simulations), and various simulation options. EmStar Java Gateway
– Consumes P-Star messages and dispatches them to EmstarProxy actors– Actors can represent entire nodes, components of a node, node tasks and
services.
15DyMND: P. Bose, ATC, LMSSC
DyMND-EE Configurations
Configuration Options Modeling the physical environment
– Radio Channel models– Sensor channel models
Modeling component interactions– Service coordination– Timing & interrupts– Sensing & actuation
16DyMND: P. Bose, ATC, LMSSC
Staged Development in DyMND-EE
Matlab – processing
Ptolemy – distributed radio and sensing model
Matlab
Minimal
17DyMND: P. Bose, ATC, LMSSC
Staged Development in DyMND-EE (Contd.)
Introducing NesC code into the simulations
18DyMND: P. Bose, ATC, LMSSC
Staged Development in DyMND-EE (Contd.)
Replacing Ptolemy models with actual environment
Testing performance in real world
19DyMND: P. Bose, ATC, LMSSC
Staged Development in DyMND-EE (Contd.)
Transfer code to actual hardware and test like you fly
20DyMND: P. Bose, ATC, LMSSC
Outline
Robust DyMND Engineering– DyMND-Sim and DyMNDES: The first generation– Lessons learnedDyMND-EE: The next generation
• Architecture• Implementation• Staged Development Configuration• Experiments
Adaptive Coordination in DyMND– Explicit local coordination for optimal control– Dynamic programming (reinforcement learning)– Design and validation strategy within DyMND-EE
Summary and Future Plan
21DyMND: P. Bose, ATC, LMSSC
DyMND-EE Experiment 1: Target Detection
Simple translation of wireless sound detection Ptolemy demo (UCB) to nesC
Simulation setup models radio and sensing channels
NesC code performs triangulation
22DyMND: P. Bose, ATC, LMSSC
Target Detection Scenarios
Scenario Robustness Concerns Control Metrics Expt. Setup
Moving Target(s)
- latency - Sampling rate less than target speed
- sleep/wake cycle- Sample rate- Time windows
- false positives or negatives- deviation of track
- insert a moving target- set params
Distribution of Aggregation nodes and detection nodes
- Flooding- Holes
- Distribution of Aggregators- Distribution of Detectors- Detection weights-Time windows
-false positives or negatives-collisions
- vary # of aggregator and detection nodes- set params for weights and time windows
Environment Effects
- Increasing sensor drift
- Calibration frequency- threshold
- false positives or negatives
- add sensor error as a function of env. params.
23DyMND: P. Bose, ATC, LMSSC
Experiment Configurations
Experiment parameters are easily changed in Ptolemy. – Experiment parameters
can be set either by the DyMNDES or by the Vergil GUI in Ptolemy
– Experiment runs can be saved in Ptolemy or as a script by the DyMNDES.
24DyMND: P. Bose, ATC, LMSSC
Source Localization Results
Limited radio range More aggregators; modified density of detectors
Increased target speed Increased target speed; limited radio ranges
25DyMND: P. Bose, ATC, LMSSC
Issues
Insight from these simple experiments demonstrate that its easy to find the optimal simulation configuration to give data that “fits” your algorithm.
Results from the perfect simulation configuration
26DyMND: P. Bose, ATC, LMSSC
DyMND-EE Experiment 2: Localization
Kestrel localization code
No changes necessary in NesC code
Simulation setup models radio and sensing channels.
NesC code performs localization
27DyMND: P. Bose, ATC, LMSSC
Localization Scenarios
Scenario Robustness Concerns
Control Metrics Experiment Setup
IrregularDistribution
- detection bias- congestion
- power mgmt. schedule - clustering
- Probability of detection given node density & distribution- throughput
- vary layout of nodes- set power on/off schedule
Critical Node Death
-Failed processes (e.g. sentry) - Failed Links (e.g. node is gateway)
- routing- role reconfiguration
- data loss - retransmission- recovery time- delay
-kill key nodes along critical paths
Temporary Node Failure and Revitalization
-Self-stabilization- Failed processes - Failed links
- retransmission rate- receive/transmit gain
- data loss- retransmission- recovery time- delay
- temporarily disable nodes- set radio- set transmission params.
28DyMND: P. Bose, ATC, LMSSC
Comparing Fidelity of Simulation
DyMND-EE results match Kestrel’s results (TOSSIM w/custom sensor models)
Confirmed that localization produced standard deviation ~0.7-1.0% of total distance for accurate ranging
29DyMND: P. Bose, ATC, LMSSC
Advantages of the DyMND-EE Approach
Software reuse and rapid prototyping– Reuse/test embedded code – Implement software prototypes as Ptolemy actors or Linux
user-space processes (managed memory, out-of-band communications)
– Implement new sensing concepts as Ptolemy actors Extensibility
– Simulation container not restricted to single hardware model Scalability
– Functional decomposition allows tradeoffs between spatial locality (neighboring nodes) and functional locality (sensors vs. radios vs. processors) for distributed implementation
Synchronization– Ptolemy messaging bus synchronizes events and
communication between services
30DyMND: P. Bose, ATC, LMSSC
Limitations Limitations
– Task-level simulator executes in a single thread per node. Timing and interrupts may not be handled as expected if using interrupts directly
– NesC component libraries only partially ported – UART, ADC, and actuation not yet completed.
– Radio and sensor models still under development and require compilation for new models
– Each node runs C or NesC code
31DyMND: P. Bose, ATC, LMSSC
Outline
Robust DyMND Engineering– DyMND-Sim and DyMNDES: The first generation– Lessons learnedDyMND-EE: The next generation
• Architecture• Implementation• Staged Development Configuration• Experiments
Adaptive Coordination in DyMND– Explicit local coordination for optimal control– Dynamic programming (reinforcement learning)– Design and validation strategy within DyMND-EE
Summary and Future Plan
32DyMND: P. Bose, ATC, LMSSC
Optimal control: The DyMND Approach
Management of sensor nets can be framed as an optimal control problem– Optimal policy maximizes value function:
Expected (discounted) sum of future rewards– Engineer specifies rewards/penalties for good/bad events– Actions rewarded/penalized per node (no global oracle)
Objective function depends on …– Quality of sensor data (higher is better)– Power consumption (lower is better)– Detection performance (precision & recall)
… over the life of the network Large sensor nets require global optimization over
(large numbers of) local state variables– Low communications overhead (packet size, power management)– Low memory overhead (!)
33DyMND: P. Bose, ATC, LMSSC
Optimal control: An example
Intruder detection with wireless sensors State variables (per mote)
– Internal state: e.g., mote battery power, current detection assignment
– Observed state:e.g., amplitude and frequency of acoustic sample
Actions (per mote)– Changing sensor sampling rates– Sending a detection message– Switching between sensing “policies” (e.g. “vigilant” / “slack”)
Rewards/penalties (per mote)– Penalty proportional to power consumption:
Encourages long network life– Reward proportional to “quality” of sensor coverage
Variance of observations over mote and neighbors
34DyMND: P. Bose, ATC, LMSSC
Junction Tree RL Coordination Service
Initializing a junction tree Identify neighbors and shared state Build routing tree and propagate shared stateMaking decisions (policy improvement) Pass messages to ascertain neighbors’
policies Determine the optimal policy conditioned on
– Neighbors’ policies– Current local value function estimate
Pass state observations in toward root, then out toward leaves
Updating value estimates (policy evaluation) New estimate is convex combination of:
– Old estimate– New observation of (rewards + successor-state
expected value)
Sampling rateLo Hi
35DyMND: P. Bose, ATC, LMSSC
Loopy value propagation
Junction-tree control imposes routing tree on communications mesh
– Worst-case path length radius of network– Costly to maintain running intersection property (storage
exponential in path length) for large networks– Expensive recovery from link/node loss to maintain tree
Analogy to probabilistic inference– Junction-tree RL relies on optimal-control form of Generalized
Distributive Law (GDL)– Statistical-mechanics approximation to GDL assumes interaction
graphs have cycles (atoms in a crystal lattice) Junction graph relaxes global tree structure
– Tree-structured coordination graph per variable– Natural coordination structure for sensor nets (immediate
neighborhoods) Introduces circular dependencies to decision-making process
– Decisions are globally suboptimal, but locally optimal (and approach optimality after many iterations)
Standard JTRL:- 5 hops
2 extra nodes- 3 hops
1 extra node
Loopy JTRL:- 2 hops- 2 hops
0 extra nodes!
Junction-tree RL makes MDPs feasible for small sensor nets;Loopy junction-tree RL makes approximate MDPs feasible for large sensor nets!
36DyMND: P. Bose, ATC, LMSSC
RL in TinyOS: Design Constraints
RL implementation requires three components– Exploration strategy to balance “exploration” (visiting new states)
and “exploitation” (returning to old states)• Exploration can occur in simulation/lab• Deployed system will favor exploitation
– Value function approximator to identify best actions for states• Initial values biased to incorporate domain knowledge• Exact (tabular) value functions are storage-intensive• Nonlinear approximators (neural nets) are compute-intensive• Linear combinations of features are efficient
– Update rule to apply changes to value function/policyMinimal computational overhead
RL service requires access to service state Messages between nodes should be small and infrequent
– Loopy value propagation avoids long message paths– Coarse-grained policy learning reduces complexity of interaction
37DyMND: P. Bose, ATC, LMSSC
RL in TinyOS: Implementation Options
1. Runtime configuration: RL-aware services register callback with RL service Registered services provide accessors for relevant
state/action or Registered services fire events for state changes, and RL
service fires events for policy changes Similar to ServiceC/ServiceM approach Requires retrofit of existing services
2. Compile-time configuration: RL service code generated with static references to ordinary nesC services Exploits static allocation: RL service directly reads state
variables, writes configuration parameters Reuses services without modification Violates isolation of nesC services “Shared memory” hack risks race conditions
38DyMND: P. Bose, ATC, LMSSC
Summary
Developed DyMND-EE for – End-to-end validation of NEST systems– Staged development of NEST systems– Robust analysis via controlled experiments within an
accurate validation environment Experimented with DyMND-EE
– Replicated experiments to validate claims (localization) Design of adaptive mechanisms for optimal
performance in the context of change– Developed algorithms– API design for implementing within DyMND-EE
39DyMND: P. Bose, ATC, LMSSC
Future Plan
Extensions to DyMND-EE– Include actuation processes– Interruptable tasks– Higher fidelity sensor, radio models
Experimental approach to design/evaluation of robust sensing/actuator networks – Modeling (sensor, radio, actuation) in DyMND-EE – Experiment driven design of services (parameters)
for specific application contexts and dynamics Adaptive coordination implementation