1
Goal: Provide a Smart Emergency Response System (SERS) that connects cyber-physical technologies with humans in the loop to save lives, rescue people, and attend to their critical needs when disaster strikes. The system includes human first responders, heterogeneous ground and aerial autonomous vehicles, human-operat- ed telerobots, and trained search and rescue dogs. It is aided with real-time sensors, help request apps, optimized re- source deployment, real-time visualization, and robust communication using diverse network types. Smart Emergency Response System – Architecture View Smart Emergency Response Saves Lives Field of Study Smart Emergency Response System Feature Societal Impact Shared autonomy Augmenting first-responder capabilities Improving availability and quality of emergency response Operations Integrating field resources into a coherent mission Empowering citizens Networks Providing an adaptive, robust, and broadband wireless response network Connecting people anytime and anywhere Optimization Minimizing delivery time for life- essential supplies Saving lives and providing quicker medical assistance Robotics Enabling tele-operated and autonomous robots, biobots, and humanoids Using machines for dangerous and challenging tasks Co-robotics Enhancing mixed-initiative collaboration among machines and between humans and machines Serving a population’s needs more quickly and more comprehensively System integration Integrating humans and various levels of machines in one mission Leveraging engineering disciplines to solve societal challenges Education/training Participating in simulated emergency response scenarios Preparing highly qualified personnel and workforce Real-time mission command and control: • Field, prioritize, and handle requests • Optimize resources in a timely manner • Dynamically provision and allocate assets • Remotely control vehicle fleet dynamics • Receive, organize, and display sensing and status information from assets • Perform real-time visualization in Google Earth Mission Command and Control Center Networks Field Operations Adaptive network-to-network mechanism: • Broadband WiFi networks via commodity drones with directional antennas (5km) • Ad-hoc wireless networks for cellphones • Adaptive relay networks to command and control center (10m) • Secure, robust, dynamic, and physically private network Autonomous and semi-autonomous vehicles, humanoids, robots, and biobots with: • Real-time sensing • Tele-operation using haptic control • Video and audio communication link • Dynamic provision for specific needs of emergency scenario • Registration of citizen sourcing • Automatic update of survivors’ social media Technological Breakthrough: The confluence of cyber-physical technologies, data-driven predictions, and human-in-the-loop telerobotics drives innova- tions in the Smart Emergency Response System. Features Technology • Remote two-way communication between human, robotic, and canine field assets and mission command and control center • Ad-hoc wireless communications between people’s cellphones without rely- ing on cellular networks • Opportunistic, wireless, secure, and robust communications using a network of commodity drones with WiFi technologies • Real-time video and audio streaming from field assets • Modular sensing technologies to make provision “plug and play” for various emergency missions • High-performance mission command and control center • Dynamic optimization of time and resources • Predictive estimation of mission progress using simulation • Autonomous collaborative fleet of vehicles • Various types of lifting robots, humanoids, and biobots • Full automation of select stages in the deployment process • Enhancement of relief operations • Extendable system architecture Human in the Loop • People’s smartphones serving as ad-hoc network relay nodes • Smartphone apps for people to report and request help • Tele-operation of field robots using haptic control to give an operator the sense of touch • Real-time update apps for people to understand the emergency response devices operations around them • Engaging mechanisms for citizens to register their devices in the mission Impact Saving Lives • Minimum emergency response time • Real-time update and automated emergency response • Reduced risk during disaster scenarios • Optimized city response units and medical infrastructure Job Creation • Telerobotic operators (as an opportunity for returning veterans) • Usability experts • Human-machine interface experts • Device app designers and developers • Public service experts and entrepreneurial citizen scientists • Unmanned aerial vehicle pilots New Businesses • Device-based services and apps • Automated pickup and delivery service • Remote, continuous, and automated inspection and surveillance • Private, opportunistic, physical network service • Supply chain optimization Economic Growth • Human productivity growth • Ecological footprint reduction • New pricing models for transportation and delivery services • Decrease in maintenance and operation expenses • Service time reduction Vision: Empowering and augmenting humans with actionable artificial in- telligence and smart devices to raise society’s level of prosperity and pre- pare a workforce qualified to operate and exploit technologies of today and tomorrow. Quadrotor/drone (WIFI) Direconal antenna (5 km) RF controller (2.4 GHz) Arduino WIFI Access Point Biobot (e.g., dog) Video camera Gas detector Raspberry PI WIFI ShAir Relay (10 m) Android app Android WIFI ATLAS humanoid WIFI Hapc device NI CompactRIO controller Depth sensor LabVIEW WIFI Ground vehicle Storage plaorm Simulink Parrot AR.Drone Video camera WIFI Fixed wings UAV Simulink Command and control Planning Opmizaon Resources distribuon Simulaon Visualizaon Navigaon Mission User Interface MATLAB Video analysis Face detecon Face following MATLAB process (DLL) AR.Drone controller design Simulink WIFI Google Earth visualizaon Google Earth Internet MATLAB (v. 32 bit) JavaScript Coordinates Communicaon Objects switch Views switch Geometry switch Geo-locaon, me {latude, longitude, me} Communicaon (IP address, port, port width) Video stream Request [‘predefined list’] Request type [supply, pickup] Video stream Geo-locaon, me {latude, longitude, me} Communicaon (IP address, port, port width) Geo-locaon, me {latude, longitude, me} Communicaon (IP address, port, port width) Geo-locaon, me {latude, longitude, me} Communicaon (IP address, port, port width) Geo-locaon, me {latude, longitude, altude, me} Communicaon (IP address, port, port width) Orientaon (heading, lt, roll) KUKA robot LabVIEW Real-Time LabVIEW FPGA NI EtherCAT RIO Waypoints Ad-hoc WIFI Network Opportunisc Relay Network Valve geo-locaon Waypoints Waypoints Video stream Geo-locaon, me {latude, longitude, me} Communicaon (IP address, port, port width) Gas detecon (gas type, latude, longitude, me) Audio to the field (ASCII text transformaon) Component Testbed of SERS Used Technology Physical device L e g e n d : U D P / I P T C P / I P B l u e t o o t h S e r i a l c o m m u n i c a t i o n B e f o r e o p t i m i z a t i o n A f t e r o p t i m i z a t i o n Mission observaon mode Google Glass Virtual Reality mode Virtual Gaming Engine Project realized for SmartAmerica Challenge, www.smartamerica.org, 2013–2014. Team Lead: Justyna Zander, MathWorks Fellow at WPI, MathWorks, 3 Apple Hill Dr., Natick, MA 01760, USA. Contact: [email protected]. BluHaptics | Boeing | MathWorks | MIT Media Lab | National Instruments | North Carolina State University University of North Texas | University of Washington | Worcester Polytechnic Institute
