Communication and Delivery-- Improving Outreach --

NWSChatFY09 Technology Day

National Weather ServiceOctober 2008

Darone JonesDaryl Herzmann

Mike Doney

Agenda ● Brief Introduction to NWSChat ● Infrastructure / Growth Plans ● “The Feature Presentation” ● Lessons Learned ● Seamless Integration

Introduction to NWSChat● NWSChat is an Internet based collaboration tool that facilitates real-time communication of operational meteorologists, emergency managers, and other local government partners

● Adapted from the Iowa Environmental Mesonet (IEM Chat) program

● Chatrooms are WFO-centric (e.g., slcchat, bmxchat, lwxchat)

● WWA products are automatically posted into chatrooms

● NWSChat = IEM Chat with the addition of NWS Policy controls, tiered support, and office-level control of user access

IEM Chat Stats 88 of 122 WFOs using it today

~3000 Users5 x Best Practice in Service Assessments over the last 6 years

NWSChat Logical Diagram

NOAAnet Infrastructure Internet / Internet2

NWSChatService

CWA-basedChatrooms

NWS Partners - EM Community - Gov’t Partners - Electronic Media

NWS DomainPublic Access Area

NWS Staff - WFOs - RFCs - CWSUs

NOAAnet FirewallPerimeter Security

NWSChat server location and accessis similar to the NWS consolidated web farms

NWSChat Infrastructure and Growth Plans

NWSHQCRHWRH SRH ERH

Western

Central Southern

Eastern

WFORFC

WFO

NOAAnetBackbone

NWSChat Servers

High-Availability Server links (logical)

Internet Internet2

WFORFC

WFO

WFORFC

WFO

WFORFC

WFO

PR AR

How does NWSChat help WFOs improve

● IM is utilized by NWS operational personnel to share critical warning decision expertise and other types of significant weather information essential to the NWS’s mission of saving lives and property

● This information is exchanged in real-time with the media and emergency response community, who in turn play a key role in communicating the NWS’s hazardous weather messages to the public

● IM provides media and emergency response partners with the ability to communicate significant event reports back to NWS operational personnel, who in turn utilize the information to make effective warning decisions

● NWS partners can also use IM as an efficient means of seeking clarifications and enhancements to the communication stream originating from the NWS during a fast-paced significant weather or hydrologic event

Decision Support?

http://www.weather.gov/infoservicechanges/IMServiceDescriptionDocument-1.pdf

“The fact is, NWS services – principally direct interaction with decision makers – are in greater demand than at any time in our nearly 140-year history.”

Jack Hayes, 2008

What is NWSChat?What is NWSChat? It is more than just a “chat server”It is more than just a “chat server”

It’s a data moverIt’s a data mover

What kind of data?What kind of data? Text based messages (“Hi, it sure is windy here”)Text based messages (“Hi, it sure is windy here”) Presence notifications (Presence notifications (what I am doing, sleepingwhat I am doing, sleeping)) Most anything else of a digital nature is possible.Most anything else of a digital nature is possible.

More than just a “chat More than just a “chat server”server”

Powering NWSChat, Powering NWSChat, XMPPXMPP

NWSChat is a service implementing XMPP.NWSChat is a service implementing XMPP. Extensible Messaging and Presence Extensible Messaging and Presence

Protocol (XMPP) is the standard used to Protocol (XMPP) is the standard used to relay “messages” between authenticated relay “messages” between authenticated clients/servers. These XML messages look clients/servers. These XML messages look like:like: <message <message

to=“my_buddy_id”>payload</message>to=“my_buddy_id”>payload</message> Where the payload could be:Where the payload could be:

XHTML marked up textXHTML marked up text Simple text messageSimple text message Encoded data (images, media, etc)Encoded data (images, media, etc) XML dataXML data

The Basic XMPP SystemThe Basic XMPP System1. Client authenticates to theserver and establishes a session.

2. XML messages areexchanged on demand and as requested during the session.

Servers share XML messages between each other (called Federation).

Advantages of XMPP over:Advantages of XMPP over:HTTP (web) and SMTP HTTP (web) and SMTP

(email)(email) HTTP is stateless, unidirectional, and HTTP is stateless, unidirectional, and

initiated from the client side (“Come and initiated from the client side (“Come and get it”)get it”)

HTML (webpages) present data with the HTML (webpages) present data with the providers idea of how the data should be providers idea of how the data should be displayed. Little separation of data and displayed. Little separation of data and presentation.presentation.

SMTP is “slow”, somewhat tedious to SMTP is “slow”, somewhat tedious to manage, and has a huge SPAM problem.manage, and has a huge SPAM problem.

Disadvantages of XMPPDisadvantages of XMPP

At the mercy of the client side to present At the mercy of the client side to present the data. Outside of simple text the data. Outside of simple text messages, no “robust” display like web messages, no “robust” display like web browsers.browsers.

Management of accounts, issues with Management of accounts, issues with persistent network connections, and persistent network connections, and issues with perceptions of ‘chat issues with perceptions of ‘chat security’.security’.

Messages are sent one at a time (some Messages are sent one at a time (some scalability concerns).scalability concerns).

How NWSChat supports How NWSChat supports Web 3.0Web 3.0

Elimination of Data Silos: Elimination of Data Silos: XMPP Federation allows “servers” to talk to each other to deliver XMPP Federation allows “servers” to talk to each other to deliver

clients dataclients data Ubiquitous Connectivity: Ubiquitous Connectivity:

Always logged in / presenceAlways logged in / presence Network Computing: Network Computing:

Developing XMPP specs for software as a service. Called server Developing XMPP specs for software as a service. Called server components.components.

Open Technologies: Open Technologies: Standards based protocol, open source server/clientStandards based protocol, open source server/client

The intelligent web: The intelligent web: Separation of data and presentation thru the use of XML based Separation of data and presentation thru the use of XML based

datastores. datastores. Open Identity:Open Identity:

Thru XMPP FederationThru XMPP Federation Intelligent Applications: Intelligent Applications:

XMPP ‘speaks’ in a computer readable language (XML). NWSBot et XMPP ‘speaks’ in a computer readable language (XML). NWSBot et al….al….

Information Flow: The Information Flow: The PastPast

SRUS56 KBMX 141414SVRBMX

SEVERE THUNDER…THE NATIONAL ….

NWS ISSUES DECISION MAKER INTERPRETS

PUBLIC ACTS

Information Flow: Information Flow: The Present with ChatThe Present with Chat

SRUS56 KBMX 141414SVRBMX

SEVERE THUNDER…THE NATIONAL ….

NWS ISSUES WHILE COLLABORATING WITHDECISION MAKERS

PUBLIC ACTS

Information Flow: Information Flow: Vision of the futureVision of the future

Single version of Single version of the product used by the product used by all with clear all with clear separation between separation between data and format data and format (XML).(XML).

Collaboration and Collaboration and feedback thru the feedback thru the entire product entire product lifecycle in real lifecycle in real time by all parties time by all parties (XMPP).(XMPP).

Seamless Integration of NWS andTrusted partners in the issuing process.

Practical ExamplePractical Example 88D Doppler RADAR tracks a mesocyclone88D Doppler RADAR tracks a mesocyclone This storm attribute appears on the forecaster’s This storm attribute appears on the forecaster’s

AWIPS II Display.AWIPS II Display. The forecaster right clicks the icon and requests The forecaster right clicks the icon and requests

any web cameras to turn to this location.any web cameras to turn to this location. An XMPP message is sent to a server An XMPP message is sent to a server

component, which moves any nearby component, which moves any nearby webcamera in the direction of the storm based webcamera in the direction of the storm based on AWIPS II provided location details.on AWIPS II provided location details.

A still image or video is streamed back to the A still image or video is streamed back to the display via XMPP showing the storm and display via XMPP showing the storm and location of the attribute on the image.location of the attribute on the image.

NWSChat’s role in the future: NWSChat’s role in the future: Connecting Weather Software Connecting Weather Software

SystemsSystems The software system could be an The software system could be an

authenticated ‘client’ to the server authenticated ‘client’ to the server and ‘chat’ with other software and ‘chat’ with other software systems. Its all just XML…systems. Its all just XML…

NWS AWIPS II

NCF

Memory UsageLocalization

Hurricane Hunter

ObservationsFlight Guidance

Trusted ThirdParty

WhiteboardingShared workspace

NWSChat’s role in the NWSChat’s role in the future: future:

Relay of CAP via PubSubRelay of CAP via PubSub Publish / Subscribe Publish / Subscribe

(PubSub) describes a (PubSub) describes a means to publish and means to publish and subscribe to feeds within subscribe to feeds within XMPP.XMPP.

NWS could publish CAP NWS could publish CAP messages, which would messages, which would be instantly relayed to be instantly relayed to authenticated subscribers authenticated subscribers (eliminates HTTP polling (eliminates HTTP polling every few seconds).every few seconds).

Offline XML storage Offline XML storage would relay these would relay these messages in a delayed messages in a delayed fashion if the customer fashion if the customer was not connected at the was not connected at the time of publishing.time of publishing.

HTTP

XMPP

How could NWSChat fit in How could NWSChat fit in with other technologies with other technologies

presented today?presented today? E-spotter, POPE-spotter, POP

The backend that delivers observations from the The backend that delivers observations from the collection server to the NWS and partners…collection server to the NWS and partners…

SevereClearSevereClear Collaboration component. Sharing application Collaboration component. Sharing application

state and whiteboarding between running clientsstate and whiteboarding between running clients GIS/KMLGIS/KML

Distribution method over internet. Push it to users Distribution method over internet. Push it to users instead of them pulling it. Provide security layer instead of them pulling it. Provide security layer for sensitive datasets.for sensitive datasets.

IRIS / MamaIRIS / Mama Provide a data service backend to the clients. Provide a data service backend to the clients.

Push alerts out in realtime.Push alerts out in realtime.

The future is NWSChatThe future is NWSChat Do we believe XML should be used for a Do we believe XML should be used for a

datastore? datastore? XMPP is how you move it around.XMPP is how you move it around.

Do we wish to integrate external and Do we wish to integrate external and internal data/software systems together?internal data/software systems together? XMPP is the language agnostic glue XMPP is the language agnostic glue

(federation)(federation) Do we wish to build truly interactive web Do we wish to build truly interactive web

services?services? XMPP provides the presence and XMPP provides the presence and

bidirectional communication route.bidirectional communication route.

Integrating NWSChat Technology – What made it Successful?

Local Innovation- Local offices capitalized on solutions available “today”- Willingness and flexibility to meet forecaster and user demands- Opensource client, server, and collaboration software- Project development funding from Regions and WFOs

Responsiveness- Tom Bradshaw and Darone Jones’ quick response on SDD- Good collaboration from National Tech Team members- Daryl Herzmann and Shane Searcys’ ability to stand-up a server quickly after Adrian Gardner called for a 60 day solution

Prototypes / Testbeds / Partnering- 4 years of chat experience between NWS and Iowa State University- 3 Regions using Openfire IM servers internally- Success Sells! (5 x Best Practice)

● We need to adopt emerging consumer technologies that most community leaders have already adopted!

● We need to do this in a timely manner that is consistent with technology trends

● We need to leverage IT resources better via Opensource solutions, collaboration with partners, and more agile development

● We need to emphasize mission and customer focused solutions verses internal oriented processes

The Challenge of Integrating Technology Infusion Projects at NWS

● Provide a group or Champion at Headquarters ready to receive new technologies/solutions for swift integration into Operations

● Implement a central test-bed environment for integrating (not developing) new technologies at the national level after extensive field testing (NWSChat implemented on borrowed systems)

● Demonstrate commitment to Technology Infusion through NWS-wide contingency funding to support and promote leading edge projects at field/regional sites of innovation

How to Address this Challenge?

From Hayes 2009 President’s Budget Rollout presentationFebruary 27, 2008

-Observations-Modeling-Forecast generation-Service Delivery Decision support assistance-User Feedback Mechanism

Protecting Lives andProtecting Lives andLivelihoods: Livelihoods:

What We Need to Get There

NWSChat NWSChat ISIS the first step the first steptowards interactive towards interactive

decision support decision support assistance!!!assistance!!!

What Else What Else ISIS A picture is worth a 1000 wordsA picture is worth a 1000 words

GoToMeetingsGoToMeetings Partners can not get enoughPartners can not get enough Meetings are conducted in preparation of on upcoming eventMeetings are conducted in preparation of on upcoming event Impromptu meetings are conducted within an hour of an eventImpromptu meetings are conducted within an hour of an event Meetings are conducted for post eventsMeetings are conducted for post events Meetings are conducted weekly and bi-weeklyMeetings are conducted weekly and bi-weekly Partners can request emergency briefings for HAZMAT or other accidentsPartners can request emergency briefings for HAZMAT or other accidents