6/16/2012

1

NSF‐PC3Pervasive Computing for 

Disaster ResponseJulian Bunn (Caltech)

Mani Chandy (Caltech)

Nalini Venkatasubramanian (UCI)

and

Sudhir Jain (IIT Gandhinagar)

Overview

• Application: Rapid, effective response to disasters with a focus on earthquakes and fires

• Technologies: pervasive sensor networks, Cloud computing, resilient communication, social networks

• Theories: big‐data and statistics for event detection; robust networking; integration of people and technology in systems; distributed algorithms for real‐time analysis 

• Demonstration: In Gujarat in Q1 2013 of low‐cost seismic network; planned demonstration of fire‐response system in 2014

6/16/2012

2

Project Team• Caltech – Models, Machine Learning, Statistics, Community 

Seismic Network, Sensors, Cloud– Researchers:  Julian Bunn, Mani Chandy, Community Seismic Network  

team (12 people)

• UC Irvine – Alerting, Human Input, Fires, Resilience of Networks, Evacuation Dynamics– Researchers: Nalini Venkatasubramanian,  Center for Emergency 

Response Technologies (10+ related individuals) 

• Current IIT Gandhinagar students – All of the above and local knowledge

JainilSatyendra Pritesh

M7.6 Bhuj Earthquake 2001

http://cires.colorado.edu/~bilham/Gujarat2001.html

http://www.geospatialworld.net/index.php?option=com_content&view=article&id=15879%

4

6/16/2012

3

Onlookers watch a cluster of shanties burning in Calcutta, India, Tuesday, Jan. 12, 2010. According to a local news agency, fire broke out in a shanty town near Bidhanagar train station Tuesday due to a gas cylinder explosion, burning the shanties and disrupting local train services. (AP Photos)

Fires in Shanties

Kolkata, Phillipines, Bogota, Dhaka, Kenya…..

6/16/2012

4

Data: Sensing Platforms And Devices (CSN)

Fire Spread Models

Data: Human as a Sensor

Sense, Analyze and Respond Platforms

Creating Situational Awareness

Shakemaps

Data: Images and video

Early Warnings and  Alerts 

6/16/2012

5

SENSING IN DISASTERS

Pervasive Computing Architecture used in Community Seismic Network (CSN)

SensorsCommunicationRemote servers(Cloud)Models & StatisticsCommunicationResponders

6/16/2012

6

CSN Phidget Accelerometer

Sheeva Plug, Battery, Sensor

Affordable Android Tablet (e.g. Indian Govt. Aakash/BSNL tablet)

Raspberry PiCellular Modem Dongle

IOIO Board

Sensor Platforms, Comms, Sensors

Single Board Computer CSN Client

Power from 12VDC Wall wart

Keyboard, VGA

ALIX.1D motherboard with 500MHz AMD Geode

USB Cellular modem with AT&T simcard

Windows XP OS on 2GB CompactFlash

LAN Connection

16bit 3‐axis PhidgetAccelerometer

6/16/2012

7

Event Detection in CSN

6/16/2012

8

Sensor Noise: Desk in Office, in Home

Sensors for fire awareness

IEEE 802.15.4 (zigbee)

Crossbow MIB510 Serial Gateway

To Server / Cloud

IMU (5 degrees

of freedom)

Crossbow MDA 300CA Data Acquisition board on MICAz2.4Ghz Mote

Inertial positioning

Carbon monoxide

Temperature, humidity

Carboxyhaemoglobin

Visibility, light,

particulate

Prototype multi sensor station based on Arduino

Hazard Sensors: Smoke, Carbon monoxide,Methane,Radiation (Geiger counter), Accelerometer, Air QualityOther Sensors: Sound, Light, Temperature, Barometric Pressure, Humidity

Carbon monoxide

6/16/2012

9

Irvine Sensorium: A MultiSensor Smartspace

Observe

AnalyzeAct

Applications: surveillance & monitoring 

Applications: SituationAwareness  in Fires

Shooterlocation: UCI#outdoors/(300,506)

!

nearby sensors

Event: shooter on campus

Shooterlocation: UCI#outdoors/(300,506)

events of interest

Create a digital representation of an evolving physical world

SATWARE – Semantic Middleware for Cyberphysical Spaces

18

EvacPack

SAFIRE- situational awareness System

Privacy Preserving Surveillance

!nearby sensors

Indoor Localization

Occupancy Forecasting System

Calit2Recycling Monitor

6/16/2012

10

19

Fire Situational Awareness

Sensor Data 

Collection

Multimedia Data 

Collection 

Sensor Data Ingest Unit

Sensor Stream Processing Module

Semantically Enriched Event Data

Visualization & Decision Support Services(Alerts, Queries, Replay, Triggers)

Demographics

Ebox  External Data Access

SAFIREStreams

Weather

Raw Sensor data(sensors, speech, 

video)Floor plans

HAZMAT

Occupancy

Temperature     humidity, visibility

SpCO, light, inertial, RFID,   heart rate, ..

Ambient CO 

Sensor/Incident  Storage& Archival

Event DBRaw Data DB

CAD  Systems

Audio/speech

Image/Video

Virtual Sensors for   Media

Level events

SensorFusion

Multisensor Event Extraction

Goal: Reliable Timely SA over Unpredictable Infrastructure

SAFIRE : Situational Awareness for Firefighters

Receive /display alert messages.

Available GIS layers

Firefighter Status

Dashboard

Mapping andLocalization

An End‐to‐End Situational Awareness Tool for Incident Commanders

6/16/2012

11

Speech‐Based Situational Awareness

Android SA phone:    (Fire Team b Leader) Broadcast Location, Speed and Direction

Local Field Server with Squad Leader Man Pack Radio

SA Display Battle Captain at  Base

Android  SA phone:   (Fire Team a  Leader)Broadcast  Location, Speed and Direction

Speech

Voice Processing

Conversation Monitoring & Playback

Image & Video Tagging

Acoustic Capture Acoustic Analysis SA Applications

Spatial Messaging

Localization via Speech

Alerts

Resilience and dependability IssuesWhat can go wrong?Infrastructure failures

• Device Failures, Network Failures, Congestion and Overloads

Techniques – exploit multiple access networks, leverage mobility, multisensor scheduling

Data Interpretation/Information  errors

• Uncertainty in Processing (e.g. speech/image processing)

• Contextual errors (e.gocclusions to a light sensor)

• Accelerometer false positives

Techniques: Sensor fusion, data cleaning and entity resolution 

• CYPRESS@UCI– A Reflective Architecture

• Digital state representation of ICPS guides a range of “safe” adaptations to achieve end‐to‐end infrastructure and 

information dependability. 

• Safe Adaptations

22

6/16/2012

12

Participatory Sensing Challenges: Data  & Network Reliability

DATA RELIABILITY  Environmental noise

Particularly mobile devices

Manufacturing differences Sensor placement

Tables amplify vibration vs. cellar floors Loose substrates amplify vibration vs. solid 

rock

Availability 

NETWORK RELIABILITY

Get data out of disaster region

Reestablish connections with outside networks (router updates, application overlays) 

Establish local connections within  network to aid disaster response

Store data locally if server is unreachable, use data mules 

Technique: Human Input

Did you feel it?

Inherently noisy and biased

Can aid situational awareness

Direct response efforts

More damaged regions

Heavier populations

Map of “Did you feel it?” responses.Image courtesy earthquake.usgs.gov

6/16/2012

13

Technique: Use Mobility to Enhance Reliability in Disconnected Networks

Aggregatecontextual data

Incident Commander 

Board

Forward bundlesupon device encounters

Forward bundlesupon gateway encounters

Periodic sensing e.g., WiFi AP fingerprints, accelerometer readings, residue battery, snapshots, audio/video recording, etc.

Visualizing the task execution process  spatially and temporally

Easy deployment of one or several mesh routers at the edge 

of the area

A Store‐Move‐and‐Forward (DTN) based approach

Mobile User, Mobile Robot

Develop mechanisms/algorithms for Replication,  Forwarding, Purging

ALERTING/NOTIFICATION  IN DISASTERS

6/16/2012

14

27

Early Warning of Earthquakes

Slow down trains

Open fire engine doors

Stop elevators

Disconnect transformers

ActuateDevicesautomatically

Short & Long Term Alerts To CitizensDelivery Layer  (High coverage and reliablility, fast notification) ‐ Peer oriented

– Peer‐oriented Flash Dissemination  using  reliable application layer multicast (FARECAST – Middleware 2010)

– Reliable Flash dissemination of  rich data  and websites using gossip‐based protocols  (ReCREW‐ IEEE Trans. On Computers 2011, Flashback – ICDCS 2007, P2P 2004)

– Flash Dissemination  over MANETS (Infocom 2010, WCNC 2009) and hybrid networks (WoWMoM 2011, SECON 2011)

Content Layer (scalability, personalization) ‐ Publish/Subscribe Architectures– Warning Specificity:  Tell users what to do ‐ varies w/ physical geography and situation– Customize content  (overcome device, language, economic, ability barriers) – In‐network Content Adaptation – Rich personalized alerts, bandwidth efficiency 

(Middleware 2008, Middleware 2009, DEBS 2009)

28

Visual and sound notifications of “Duck 

and Cover”

TimelineShakecast Information Detailed InformationEarthquake Early Warning

Earthquake hits the area

6/16/2012

15

Crisis Alert System

Crisis Information from Emergency Operations Center

Automatic notification systems (Triggered by Sensors, CAP based systems. ..)

Proprietary delivery mechanism used by the organization

Content Generator

Delivery Components

Recipient Selector

Organization Selector

Crisis PolicySpecificationsCrisis Policy

Specifications

Crisis Alert

Decides how to react to the crisis. Based on the crisis policies

Customizes the messages according to information about organizations

Decides who receives the notification according to the organization’s policy

Sends the messages to the recipients in different modalities

GIS Information

Delivery Layer: Exploiting Correlations for Reliable Alerting in disasters

• Geographical and Societal Correlations– Need Correlation

• Inside/Outside region of an event• Societal correlation of information needs

– Failure Correlation• Simultaneously regional failure• Correlation between multilevel networks

6/16/2012

16

GeoSocial Alerting exploiting geosocial information

Proximity‐based neighbor selection:  Create a reliable overlay that exploits correlation between overlay and underlay

Exploit  diffusion through social networks:  to reach recipients in failed regions:  Phone, Human contact, data mules

High Reliability under large scale geographical failure

Geographical failure causes significant loss of reliability

during a flash dissemination, even with reliable mesh/forest

overlay.

Proximity based selection helps an reliable overlay achieve higher reliability under higher failure

y

Under lower failure, any reliable overlay can achieve high

reliability

Forest

6/16/2012

17

GSFord: A Reliable GeoSocial Notification System

GSFord reaches up to 99.9% of desired recipients even under massive geographically correlated regional failures.

Content Layer: Scalable and Dynamic Publish/Subscribe Systems

6/16/2012

18

Scalable Broker Overlays for Dynamic Pub/Sub

• Broker overlay architecture• Fault-tolerance • Load balancing

• Broker internal operations– MICS: Using multidimensional

indexing techniques for efficient content matching and subscription management

– An efficient subscription subsumption detection technique for multidimensional content space

• Beyond pub/sub backbone– Content customization in broker

overlay network

35

Alerting Next Steps

• Delivery‐Layer research– Mobile GeoSocial Alerting: over hybrid networks and highly dynamic 

environments• Cellular, WiFi, WiFi adhoc,  human DTN networks

– Provide seamless support of user mobility through networks

• Content‐Layer research– Publish/subscribe as a basis for large scale personalized alerting 

• Deal with high dynamicity of subscriptions under user mobility

• Scale to large numbers of users (peer broker networks)

• System Implementation– Apps on devices and a notification infrastructure

6/16/2012

19

Why Collaborate?• Economic Impact

• Societal Impact

• Enhancing the research ecosystem in India

– Undergraduate research projects

• Making the research concrete

– Fires and earthquakes are important to both Gujarat and California

– Long term system operations in both places

– Low‐cost, large scale platforms 

First Interstate Bank FireMay 4, 1988 

Team Collaboration

• 2 IIT students in 2011, 3 in 2012 at Caltech

• 4 visits by IIT‐Gn director, Dr. Sudhir Jain to Caltech to discuss implementation of disaster response in Gujarat.

• Campus visits between UCI and Caltech

• Course on cyber physical systems with applications to disaster response will be taught by Caltech faculty at IIT‐Gn in January, 2013; System Artifacts  for emergency response course taught by UCI PI. 

• Design of systems specifically for Indian environment (example of computers turned off at night; Indian tablets such as BSNL and Aakash).