Post on 12-Jan-2022
transcript
Goal: Provide a Smart Emergency Response System (SERS) that connects cyber-physical technologies with humans in the loop to save lives, rescue people, and attend to their critical needs when disaster strikes.
The system includes human first responders, heterogeneous ground and aerial autonomous vehicles, human-operat-ed tele robots, and trained search and rescue dogs. It is aided with real-time sensors, help request apps, optimized re-source deployment, real-time visualization, and robust communication using diverse network types.
Smart Emergency Response System – Architecture View
Smart Emergency Response Saves Lives
Field of Study Smart Emergency Response System Feature Societal Impact
Shared autonomy Augmenting first-responder capabilities Improving availability and quality of emergency response
Operations Integrating field resources into a coherent mission Empowering citizens
Networks Providing an adaptive, robust, and broadband wireless response network Connecting people anytime and anywhere
Optimization Minimizing delivery time for life- essential supplies Saving lives and providing quicker medical assistance
Robotics Enabling tele-operated and autonomous robots, biobots, and humanoids Using machines for dangerous and challenging tasks
Co-robotics Enhancing mixed-initiative collaboration among machines and between humans and machines
Serving a population’s needs more quickly and more comprehensively
System integration Integrating humans and various levels of machines in one mission Leveraging engineering disciplines to solve societal challenges
Education/training Participating in simulated emergency response scenarios Preparing highly qualified personnel and workforce
Real-time mission command and control:
• Field, prioritize, and handle requests
• Optimize resources in a timely manner
• Dynamically provision and allocate assets
• Remotely control vehicle fleet dynamics
• Receive, organize, and display sensing and status information from assets
• Perform real-time visualization in Google Earth
Mission Command and Control Center Networks Field Operations
Adaptive network-to-network mechanism:
• Broadband WiFi networks via commodity drones with directional antennas (5km)
• Ad-hoc wireless networks for cellphones
• Adaptive relay networks to command and control center (10m)
• Secure, robust, dynamic, and physically private network
Autonomous and semi-autonomous vehicles, humanoids, robots, and biobots with:
• Real-time sensing
• Tele-operation using haptic control
• Video and audio communication link
• Dynamic provision for specific needs of emergency scenario
• Registration of citizen sourcing
• Automatic update of survivors’ social media
Technological Breakthrough: The confluence of cyber-physical technologies, data-driven predictions, and human-in-the-loop telerobotics drives innova-tions in the Smart Emergency Response System.
Features
Technology
• Remote two-way communication between human, robotic, and canine field assets and mission command and control center
• Ad-hoc wireless communications between people’s cellphones without rely-ing on cellular networks
• Opportunistic, wireless, secure, and robust communications using a network of commodity drones with WiFi technologies
• Real-time video and audio streaming from field assets
• Modular sensing technologies to make provision “plug and play” for various emergency missions
• High-performance mission command and control center
• Dynamic optimization of time and resources
• Predictive estimation of mission progress using simulation
• Autonomous collaborative fleet of vehicles
• Various types of lifting robots, humanoids, and biobots
• Full automation of select stages in the deployment process
• Enhancement of relief operations
• Extendable system architecture
Human in the Loop
• People’s smartphones serving as ad-hoc network relay nodes
• Smartphone apps for people to report and request help
• Tele-operation of field robots using haptic control to give an operator the sense of touch
• Real-time update apps for people to understand the emergency response devices operations around them
• Engaging mechanisms for citizens to register their devices in the mission
Impact
Saving Lives
• Minimum emergency response time
• Real-time update and automated emergency response
• Reduced risk during disaster scenarios
• Optimized city response units and medical infrastructure
Job Creation
• Telerobotic operators (as an opportunity for returning veterans)
• Usability experts
• Human-machine interface experts
• Device app designers and developers
• Public service experts and entrepreneurial citizen scientists
• Unmanned aerial vehicle pilots
New Businesses
• Device-based services and apps
• Automated pickup and delivery service
• Remote, continuous, and automated inspection and surveillance
• Private, opportunistic, physical network service
• Supply chain optimization
Economic Growth
• Human productivity growth
• Ecological footprint reduction
• New pricing models for transportation and delivery services
• Decrease in maintenance and operation expenses
• Service time reduction
Vision: Empowering and augmenting humans with actionable artificial in-telligence and smart devices to raise society’s level of prosperity and pre-pare a workforce qualified to operate and exploit technologies of today and tomorrow.
Quadrotor/drone (WIFI) Directional antenna (5 km)
RF controller (2.4 GHz)
Arduino WIFI Access Point
Biobot (e.g., dog) Video camera Gas detector
Raspberry PI WIFI
ShAir Relay (10 m) Android app
Android WIFI
ATLAS humanoid
WIFI
Haptic device NI CompactRIO controller
Depth sensor
LabVIEW WIFI
Ground vehicle Storage platform
Simulink
Parrot AR.Drone Video camera
WIFI
Fixed wings UAV
Simulink
Command and control Planning
Optimization Resources distribution
Simulation Visualization Navigation
Mission User Interface
MATLAB
Video analysis Face detection Face following
MATLAB process (DLL)
AR.Drone controller design
Simulink WIFI
Google Earth visualization
Google Earth Internet
MATLAB (v. 32 bit) JavaScript
CoordinatesCommunication Objects switch Views switch Geometry switch
Geo-location, time {latitude, longitude, time}Communication (IP address, port, port width)
Video stream
Request [‘predefined list’]Request type [supply, pickup]
Video stream
Geo-location, time {latitude, longitude, time}Communication (IP address, port, port width)
Geo-location, time {latitude, longitude, time}Communication (IP address, port, port width)
Geo-location, time {latitude, longitude, time}Communication (IP address, port, port width)
Geo-location, time {latitude, longitude, altitude, time} Communication (IP address, port, port width)Orientation (heading, tilt, roll)
KUKA robot
LabVIEW Real-Time LabVIEW FPGA
NI EtherCAT RIO
Waypoints Ad-h
oc W
IFI
Net
wor
k O
ppor
tuni
stic
Rela
y N
etw
ork
Valve geo-location
Waypoints
Waypoints
Video stream
Geo-location, time {latitude, longitude, time}Communication (IP address, port, port width)
Gas detection (gas type, latitude, longitude, time)
Audio to the field (ASCII text transformation)
Component Testbed of SERS
Used Technology
Physical device
Legend:
UDP / IP TCP / IP Bluetooth Serial communication Before optimization After optimization
Mission observation mode
Google Glass
Virtual Reality mode
Virtual Gaming Engine
Project realized for SmartAmerica Challenge, www.smartamerica.org, 2013–2014. Team Lead: Justyna Zander, MathWorks Fellow at WPI, MathWorks, 3 Apple Hill Dr., Natick, MA 01760, USA. Contact: dr.justyna.zander@ieee.org.
BluHaptics | Boeing | MathWorks | MIT Media Lab | National Instruments | North Carolina State University University of North Texas | University of Washington | Worcester Polytechnic Institute