ELECTRICITY SUBSECTOR TRANSMISSION RESILIENCE …

PNNL-30543

ELECTRICITY SUBSECTOR TRANSMISSION RESILIENCE MATURITY MODEL (TRMM) Facilitator Guide

Draft Version 1.0 October 2020

DISCLAIMER

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor Battelle Memorial Institute, nor any of their employees, nor other organizations participating in the production of this report makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or Battelle Memorial Institute, or the other organizations participating in the production of this report. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

PACIFIC NORTHWEST NATIONAL LABORATORY operated by BATTELLE

for the UNITED STATES DEPARTMENT OF ENERGY

under Contract DE-AC05-76RL01830

i

Transmission Resilience Maturity Model Facilitator Guide Table of Contents

TABLE OF CONTENTS 1. INTRODUCTION ............................................................................................................................................... 1

1.1 Purpose of This Guide .................................................................................................................................. 1 1.2 Intended Audience for the Facilitator Guide ............................................................................................... 1 1.3 How to Use This Guide ................................................................................................................................. 2 1.4 Organization of This Guide ........................................................................................................................... 3

2. PREPARATION ................................................................................................................................................. 4

2.1 Key Roles in the Self-Assessment Process .................................................................................................... 4 2.2 Key Skills for a Facilitator ............................................................................................................................. 6 2.3 Key Skills for a TRMM coordinator ............................................................................................................... 6 2.4 Key Skills for a Constructive Critic ................................................................................................................ 7 2.5 Obtaining the Latest Version of TRMM Facilitation Materials ..................................................................... 8 2.6 Becoming Familiar with the TRMM and Self-Assessment Materials ........................................................... 8 2.7 Meeting with the Sponsor and Other Stakeholders..................................................................................... 9 2.8 TRMM Design and Focus .............................................................................................................................. 9 2.9 Identifying and Preparing Participants and Support Personnel ................................................................. 10 2.10 Scheduling the Self-assessment ................................................................................................................. 13 2.11 Planning and Logistics of the Self-assessment ........................................................................................... 14

3. ASSESSMENT ................................................................................................................................................. 15

3.1 Managing the Assessment ......................................................................................................................... 15 3.2 Setting up the Meeting Rooms .................................................................................................................. 17 3.3 Kicking Off the Assessment ........................................................................................................................ 18 3.4 Facilitating the Assessment ........................................................................................................................ 19 3.5 Processing the Collected Data .................................................................................................................... 20 3.6 Presenting the Core Assessment Report .................................................................................................... 20

3.6.1 Domain-level Charts ...................................................................................................................... 20 3.6.2 Objective-level Charts .................................................................................................................... 23 3.6.3 Summary of Identified Gaps .......................................................................................................... 24

3.7 Treatment of All Self-Assessment Materials .............................................................................................. 25

4. FOLLOW-UP ACTIVITIES ................................................................................................................................ 26

4.1 Analyzing Identified Gaps ........................................................................................................................... 26 4.1.1 Setting a Target Profile – Method 1 .............................................................................................. 26 4.1.2 Setting a Target Profile – Method 2 .............................................................................................. 27

4.2 Prioritizing and Planning ............................................................................................................................ 27 4.3 Implementing Plans.................................................................................................................................... 28

5. SUMMARY .................................................................................................................................................... 29

APPENDIX A: FREQUENTLY ENCOUNTERED DISCUSSIONS ................................................................................... 30

A.1 Discussions Relevant to the Entirety of a TRMM Self-Assessment ............................................................ 30 A.2 TRMM Domain-Specific Discussions .......................................................................................................... 31

APPENDIX B: REFERENCES ................................................................................................................................... 33

ii

Transmission Resilience Maturity Model Facilitator Guide LIST OF FIGURES

LIST OF FIGURES Figure 1.1: Typical Phases of Self-Assessment ...................................................................................... 3 Figure 3.1: Sample Bar Chart Presenting MIL + Progression Toward the next MIL for each Domain 21 Figure 3.3: Sample Donut Chart Domain Summary of an Assessment ............................................... 22 Figure 3.4: Sample – Workforce and Family Care Management Domain Bar Chart Presenting MIL +

Progression Toward the next MIL by Objective ................................................................ 23 Figure 3.5: Sample – Workforce and Family Care Management Domain Donut Chart Summary by

Objective ........................................................................................................................... 24 Figure 3.6: Sample Excerpt of Summary of Identified Gaps Table for Workforce and Family Care

Management Domain ....................................................................................................... 24

LIST OF TABLES Table 2.1: Key Roles in the Self-Assessment Process ............................................................................ 4 Table 2.2: TRMM Materials Useful for a Self-Assessment .................................................................... 8 Table 2.3: Example SMEs and Support Personnel .............................................................................. 10 Table 2.4: Steps and Activities Involved in Scheduling the Self-assessment ...................................... 14 Table 2.5: Planning and Logistics Tasks............................................................................................... 14 Table 3.1: Example Time Tracking Tool During an Assessment .......................................................... 16 Table 3.2: General Assessment Management Tasks for Assessment Days ........................................ 17 Table 3.3: Room Set-up Tasks for Day of the Assessment .................................................................. 17 Table 3.4: Topics for Discussion at the Start of the Assessment ........................................................ 18

iii

Transmission Resilience Maturity Model Facilitator Guide ACKNOWLEDGMENTS

ACKNOWLEDGMENTS The U.S. Department of Energy (DOE), Electric Power Research Institute (EPRI) and North American Transmission Forum (NATF), and Pacific Northwest National Laboratory (PNNL) acknowledge the dedication and technical expertise of the organizations and individuals who have provided the critiques, evaluations, and modifications to enable the development of this first release of the Transmission Resilience Maturity Model (TRMM) Guide.

Department of Energy PNNL NATF/EPRI Members David Meyer Cliff Glantz Jeff Schraufnagel ATC

EPRI Paul Skare Dave McRee Duke Energy Kevin Berent Sri Nikhil Gourisetti Tom Pruitt Duke Energy Laura Fischer Grace Mcnally Steve Ladd Duke Energy

NATF Devan Farrell Floyd Galvan Entergy Ed Ernst Easton Gervais Mark Peterson Great River Energy Lynna Estep Gordie Halt ITC Ken Keels Dennis Snook OPPD Todd Lucas Southern Company Grant Smedley Salt River Project Mark Wehlage Xcel Energy Brian Long Xcel Energy

1

Transmission Resilience Maturity Model Facilitator Guide INTRODUCTION

1. INTRODUCTION 1.1 Purpose of This Guide The purpose of this document is to enable organizations to use the Transmission Resilience Maturity Model (TRMM) to conduct a successful internally or externally facilitated self-assessment of their transmission resilience programs.

Internally facilitated self-assessments: Entities access the publicly available TRMM and conduct an internally led self-assessment without external facilitation.

Externally facilitated self-assessments: An experienced practitioner of the TRMM facilitates the member’s self-assessment of their resilience maturity via the TRMM.

This guide:

▪ provides information on how to prepare for the self-assessment (including obtaining the latest version of the TRMM and support materials)

▪ provides information on the roles and skills needed to conduct the TRMM assessment

▪ provides guidance on which SMEs to include in the self-assessment

▪ assists the facilitator(s) in applying the TRMM to evaluate transmission resilience capabilities

▪ provides guidance for follow-on activities to prioritize and implement a plan to close identified capability gaps.

The TRMM Facilitator Guide, and the TRMM itself, are designed and constructed based on previous maturity model work. As such, readers familiar with the Electricity Subsector Cybersecurity Capability Maturity Model (ES-C2M2)1 and other C2M2-based models will note the similar approach and style in TRMM products and guides.

1.2 Intended Audience for the Facilitator Guide This guide is intended for use by an individual or a small team selected by the organization to plan and facilitate a TRMM self-assessment. Individuals conducting the assessment are called facilitators. Facilitators can be internal or external to the organization. The organization should name a TRMM coordinator to work in concert with the facilitators. The TRMM coordinator serves as the liaison between the facilitator(s) and the organization Subject Matter Experts (SME). The TRMM coordinator role is especially important when using external facilitators.

For internally facilitated self-assessments and depending on the individuals’ expertise, one of the facilitators may also take on the role of TRMM coordinator.

1 http://energy.gov/node/369271

http://energy.gov/node/369271

2


The TRMM coordinator is accountable to a sponsor within the organization who has requested the self-assessment. The TRMM coordinator is also responsible for identification of SMEs and scheduling their interviews.

1.3 How to Use This Guide Facilitators and the TRMM coordinator should use this guide as a starting point for preparing and executing the TRMM self-assessment. The sections of the guide correspond to the three key phases of a typical self- assessment: Preparation, Assessment, and Follow-Up. Facilitators and the TRMM coordinator should read through the entire guide and other supporting documents to become familiar with the TRMM as well as the end-to-end process of executing the self-assessment. Familiarity with the materials is important because each organization is unique, and therefore each self-assessment will be different. Although the information in this guide is presented in a logical sequence, it does not necessarily indicate a specific order of actions, as this depends on the organization. In addition, some iteration of activities may be necessary.

Model-specific issues or instructions will be found in boxes like this one throughout this guide.

3


1.4 Organization of This Guide This guide is organized as shown in Figure 1.1. Sections 2–4 provide detailed descriptions of the three key phases of a typical self-assessment process:

▪ Section 2: Preparation phase (getting ready for an assessment)

▪ Section 3: Assessment phase (conducting an assessment)

▪ Section 4: Follow-Up phase (analyzing the results and determining next steps)

A brief summary is provided in Section 5, followed by appendices containing information on frequently encountered discussions and a list of references.

Figure 1.1: Typical Phases of Self-Assessment

Preparation• A facilitator works with the organization to prepare for the

TRMM self-assessment process.• TRMM Coordinator identifies and schedules SMEs for the

assessment

Assessment• A facilitator assists the organization to complete the

TRMM self-assessment.

Follow-up• The organization uses the results of the self-assessment to

evaluate the maturity of its transmission resilience capabilities and determine next steps.

4

Transmission Resilience Maturity Model Facilitator Guide PREPARATION

2. PREPARATION This section describes the roles and desired skills of those participating in the TRMM self-assessment, along with the necessary planning and preparation activities.

2.1 Key Roles in the Self-Assessment Process A successful TRMM self-assessment requires the involvement and active participation of members of the organization who serve in a variety of roles. The key roles involved in a typical TRMM self-assessment are summarized in Table 2.1 below.

Table 2.1: Key Roles in the Self-Assessment Process

Role Description and Responsibilities

Sponsor The sponsor should have a broad understanding of the status and components of the function for which the TRMM is being completed. The model defines a function as the subset of the operations of the organization that are being evaluated. It is most helpful for a sponsor to be: • part of the senior management team • a respected executive • acknowledged by the staff members as being in charge of their efforts and responsible for results • able to give this role sufficient time and thoughtful attention.

General responsibilities include: • deciding whether the organization should participate in the TRMM self-assessment process • selecting facilitator(s), TRMM coordinator, and constructive critic • ensuring that the necessary resources for the TRMM self-assessment process are available • ensuring that the output from the project will receive the attention it deserves across the

organization • participating in resolving issues and problems • committing resources and access to those resources • assigning the point of contact and other personnel resources • communicating the organization’s support for the TRMM self-assessment process, asking the

team members to provide the necessary support • kicking off the TRMM self-assessment

Facilitator (can be either internal or external)

The sponsor determines whether the assessment will be internally or externally facilitated. The facilitator(s) work closely with the TRMM coordinator for planning and execution of the assessment. General facilitator responsibilities include: • completing the three phases of a typical TRMM self-assessment process in coordination with the

TRMM coordinator • ensuring that all activities in the self-assessment process are executed efficiently and effectively • working with the organization to ensure the self-assessment produces high-quality results • facilitating the TRMM self-assessment • recording responses and comments during the TRMM self-assessment • generating the TRMM Core Assessment Report • distributing the TRMM Core Assessment Report to the organization • reviewing the detailed outcomes with the organization

5



TRMM coordinator

The TRMM coordinator plays serves as liaison between the organization SMEs and the facilitators and may augment many of the responsibilities listed above for facilitators. The role of the TRMM coordinator is even more important when external facilitators are used. General TRMM coordinator responsibilities include: • serve as liaison between the facilitator and the organization

o assisting the facilitator(s) in understanding the organization and how it functions o serve as the main contact for the organization on preparation and training calls o working with the facilitator(s) to ensure proper participation for the assessment

• identify organization SMEs for each domain • schedule organization SMEs for TRMM training and information sessions during the planning

phase • schedule organization SMEs for domain interviews • ensure that the organization SMEs have all TRMM supporting documentation they need in order to

prepare for the assessment • observe and participate in all domain interviews during the assessment • ensuring that proper facilities and support staff are available for the assessment • assist in the planning of follow-up activities

Constructive critic

A constructive critic is an individual from the organization with sufficient knowledge to challenge SME responses to ensure good thought and discussion on TRMM topics in order to achieve an accurate assessment. Key roles of the constructive critic are to advocate for honesty from the SMEs and keep the focus of SME answers related to resilience (i.e., ensure SMEs are considering the practices through a resilience lens.) Basic skills should include: • good understanding of resilience activities for the organization and the personnel/groups that

perform those activities • good interpersonal and communication skills • personality that is comfortable challenging dogma in a diplomatic way • ability to read the room/body language • assertiveness to step in if there is concern about biases

o especially important if management is included in the reviews to ensure they do not bias the results

Subject matter experts (SMEs)

SMEs provide answers to the self-assessment questions that best represent the organization’s current resilience capabilities in relation to the domain/objective/practice being evaluated. It is most helpful for a SME to be: • closely involved in the planning, implementation, or management of the domain represented • able to understand or speak about one or more of the domain objectives • able to represent organizational practices that are being evaluated

Reference Section 2.9 for details regarding likely job positions/responsibilities of SMEs involved in a successful self-assessment.

Participants All individuals whose presence and active participation is necessary and critical during the self- assessment process (e.g., sponsor, facilitator, constructive critic, SMEs) are referred to as participants. The TRMM coordinator should ensure all participants are available for the duration of the self-assessment.

Observers All individuals whose presence and active participation are optional during the self-assessment are referred to as observers. Attendance of observers should be approved by the sponsor or designee.

6



Support staff In collaboration with the sponsor, the TRMM coordinator should identify all other individuals whose support is necessary during all three phases of a typical TRMM self-assessment process. Those individuals can include: • administrative assistants (to send meeting invitations, coordinate calendars, copy and assemble

materials) • scribes (to take notes during meetings as necessary) • technology support staff (to provide and set up all necessary information technology [IT] and non-

IT hardware and software required for presentations or meetings, including visitor internet access) • site security staff (to issue visitor badges and enable proper physical access to any visitors serving

as facilitators or contributing stakeholders during the self-assessment)

2.2 Key Skills for a Facilitator A facilitator is someone who helps a group of people understand their common objectives and assists them in planning to achieve these objectives without taking a particular position in the discussion.

The key roles of the facilitator are facilitating the self-assessment, recording responses and comments, generating the TRMM Core Assessment Report, and reviewing the detailed outcomes with the organization’s management team.

The basic skills of a facilitator consist of effective and efficient meeting management and interview practices: timekeeping, following an agreed-upon agenda, and keeping a clear record of the meeting. The higher-order skills involve observing an interviewee or group of individuals and adjusting the meeting flow as needed. For TRMM facilitation, this requires sufficient knowledge of both resilience and the transmission business: the facilitator must have a high-level, comprehensive understanding of resilience and how it relates to transmission and any supporting enterprise groups related to the topics in the TRMM.

In addition, facilitators need a variety of listening skills, including the ability to paraphrase, draw people out, balance participation, and make space for more reticent group members. The facilitator must have the knowledge and skill to be able to intervene in a way that adds to an interviewee’s answer or a group’s creativity. A successful facilitator embodies respect for others and a watchful awareness of the many perceptions of reality. In the event that a consensus cannot be reached among interview subjects or in group discussions, the facilitator should assist the subjects in understanding the differences that divide them in order to come to a logical compromise.

2.3 Key Skills for a TRMM coordinator The TRMM coordinator works in concert with the facilitators and acts as the liaison between the facilitators, the organization’s management, and the organization SMEs. The TRMM coordinator is responsible for identification of SMEs and scheduling their time for information, training, and domain interviews. The TRMM coordinator also ensures that the SMEs have the TRMM supporting documentation they need in order to prepare for the assessment.

The TRMM coordinator is expected to participate in assessment planning meetings, introduction and exit presentations, and may participate in some or all assessment interviews

7


(participation in interviews with SMEs is particularly helpful during externally facilitated assessments).

A TRMM coordinator should embody many of the skills listed above for facilitators. Additionally, a TRMM coordinator should have a good understanding of resilience activities for the organization and the personnel/groups that perform those activities.

2.4 Key Skills for a Constructive Critic A constructive critic is an individual from the organization with sufficient knowledge to tactfully challenge SME responses to ensure good thought and discussion on TRMM topics in order to achieve an accurate assessment. Past assessments have shown that having a constructive critic attend all domain interviews during the assessment is incredibly beneficial in removing bias from the assessment.

Key roles of the constructive critic are to advocate for honesty from the SMEs and keep the focus of SME answers related to resilience (i.e., ensure that SMEs are considering the practices through a resilience lens.)

The assessment works well if the organization approaches the assessment openly, honestly, and with egos and preconceptions left behind. One of the constructive critic’s most important responsibilities is to ensure this posture for the organization. It is important to note that the assessment is not intended to be like an audit, as there is no need for supporting evidence. The assessment provides a roadmap to help the organization identify areas for improvements. Candid interviews will produce an accurate roadmap and provide the most benefit to the organization. The constructive critic plays a big role in achieving that.

The basic skills of a constructive critic consist of:

▪ good understanding of resilience activities for the organization and the personnel/groups that perform those activities

▪ good interpersonal and communication skills

▪ personality that is comfortable challenging dogma in a diplomatic way

▪ ability to “read the room” and understand body language

▪ assertiveness to step in if there is concern about biases (especially important if management is included in the reviews to ensure they do not bias the results)

Oftentimes, the TRMM coordinator has the expertise to fill the role of constructive critic. However, with all the responsibilities of the TRMM coordinator, it is advised to name a separate constructive critic so that they can work with the TRMM coordinator to maintain balance in the assessments of all domains.

8


2.5 Obtaining the Latest Version of TRMM Facilitation Materials The TRMM website2 contains the latest resource materials useful for facilitators and TRMM coordinators. The facilitators and TRMM coordinators should leverage these materials for preparation and implementation activities.

Table 2.2: TRMM Materials Useful for a Self-Assessment3

Title Brief Description File Type

TRMM tool4 Online tool containing the list of questions associated with the TRMM and that generates the Core Assessment Report

Online; web-based tool

TRMM User Guide One-stop documentation for the TRMM, especially for SMEs or sponsors of an assessment; contains comprehensive information about the model including domains, objectives, and practices

PDF

TRMM Facilitator Guide TRMM documentation targeted for individuals or teams facilitating an assessment

PDF

TRMM Supplementary Explanations

Provides listing of TRMM domains, objectives, and practices alongside further explanations of the practices, as deemed necessary

PDF

2.6 Becoming Familiar with the TRMM and Self-Assessment Materials It is critical for the facilitators and TRMM coordinators to be familiar with the TRMM, this Guide, and the other materials listed in Table 2.2 above. It is recommended that a new facilitator or TRMM coordinator follow the steps detailed below to gain the necessary familiarity with the TRMM, the facilitation process, and the required materials:

1. Read the TRMM User’s Guide and become familiar with its

▪ goals

▪ objectives

▪ model architecture

▪ domains and domain structure

▪ maturity indicator levels/scoring

▪ details of each of the nine domains (including domain-specific practices and examples)

▪ instructions for using the TRMM tool.

2. Read this Facilitator Guide in its entirety. The insights provided in this document should help understanding how to facilitate the assessment.

2 TRMM Home page: https://trmm.labworks.org 3 Additional resources are available on the TRMM website: https://trmm.labworks.org/resources 4 TRMM tool page: https://trmm.labworks.org/trmmtool

https://trmm.labworks.org/

https://trmm.labworks.org/resources

https://trmm.labworks.org/trmmtool

9


3. Ensure you can access and use the TRMM tool. Open the TRMM tool and practice using it – including recording assessment results, inserting comments and explanatory materials, and generating the core self-assessment report.

If you have questions about any of the materials associated with the self-assessment, email addresses are provided on the TRMM website contacts page.5

2.7 Meeting with the Sponsor and Other Stakeholders Prior to setting dates for the planned self-assessment, the TRMM coordinator should organize a meeting with internal stakeholders identified by the sponsor to prepare for the self-assessment process. A meeting with the sponsor should take place prior to scheduling the self-assessment.

The objectives of this meeting include the following:

▪ Familiarize the sponsor and stakeholders with the TRMM (e.g., the TRMM information and introduction presentation materials could be used during this meeting).

▪ Obtain strong and visible executive support for the self-assessment and the associated meetings with SMEs.

▪ Familiarize the stakeholders with the organization’s operating environment, the business drivers influencing its resilience efforts, and how the TRMM self-assessment will be used by the organization.

▪ Discuss the sponsor’s expectations for the self-assessment process (e.g., the three phases of the process, required resources, timeframe involved, personnel roles and responsibilities).

▪ Discuss a desired future state of organizational resilience capabilities, consistent with the organization’s business objectives and risk environment and the TRMM as a framing structure.

▪ Discuss plans for next steps after the self-assessment is conducted.

▪ Discuss the need for an additional preparatory meeting(s) with the sponsor and other stakeholders in the organization.

2.8 TRMM Design and Focus The TRMM is a programmatic-level assessment, not a technical assessment. The TRMM was intentionally designed to focus on an organization’s resilience-related processes, procedures, and general practices.

Additionally, the TRMM is designed to be “threat agnostic.” There will be little-to-no mention of specific threats, such as cybersecurity, ice storms, or geomagnetic disturbances (GMD) in the domains other than occasional parenthetical references.

5 TRMM Contacts page: https://trmm.labworks.org/contacts

https://trmm.labworks.org/contacts

10


2.9 Identifying and Preparing Participants and Support Personnel For the TRMM self-assessment to be successful, participants should have enough knowledge of the operations of the organization to answer questions within their area of functional expertise, including the practices related to the domain. The self-assessment is focused on all processes, resources and capabilities the Transmission Business Unit (TBU) uses to meet its resilience needs and objectives. In this self-assessment, non-TBU parts of the enterprise (e.g., supply chain and purchasing, corporate communication and public affairs, government relations) are included in the self-assessment process to extent that the they provide support and capabilities that the TBU uses to meet its resilience needs and objectives. The resilience capabilities of these non-TBU parts of the enterprise would be assessed along with those resilience capabilities under the direct management control of the TBU (e.g., system operations, field operations, transmission engineering, etc.) Therefore, SMEs from all appropriate areas of the organization, not just the TBU, should be included in the assessment.

There should be SMEs representing how the organization operates in all nine TRMM domains (see Table 2.3). It is not necessary to have a unique SME for each domain or limit yourself to one SME per domain. An individual can be an SME for multiple TRMM domains, and it is often necessary to engage multiple SMEs to fully cover a single domain. SMEs for any one domain may come from a single group or represent several groups within the organization. For continuity, the TRMM coordinator and constructive critic would benefit from hearing the discussions for all domains.

Table 2.3: Example SMEs and Support Personnel

Domain/Expertise/Function Example Job Functions/Titles Notes

Resilience Program Management Functions: • Grid Resilience/Enterprise Resilience • Transmission Business • Rates & Regulatory Affairs • Strategy & Corporate Development • Asset Management and Strategy • Transmission Business Development • Transmission Policy

Example Titles: • Director, Transmission Business Operations • Director, Emergency Preparedness • Manager, Enterprise Preparedness (may be

part Security Department) • Senior Director for Energy Security and

Resilience Programs

This will vary by company structure, depending on how resilience is covered (transmission specific vs. enterprise-wide, consolidated vs. dispersed)

To help with organizing the needed SMEs, it might be helpful for one expert to be identified as the primary contact for each domain to assist the TRMM coordinator in scheduling meetings with other SMEs for that domain.

11



Risk Identification, Assessment, and Management

Functions: • Risk Management • Transmission Engineering • Transmission Planning • Transmission Operations • Transmission OT • Asset Management • Cyber/Physical Security

Example Titles: • Director, Enterprise Risk Management • Manager, Enterprise Response Center (may be

part of Corporate Security Department)

Include those responsible for managing risk, risk matrix, risk register

Include those responsible for various critical areas of risk such as cyber, OT, etc.

Situational Awareness Functions: • Transmission Operations • Transmission Control Centers • EMS/Operation Technology • Cyber/Physical Security

Example Titles: • Manager, Enterprise Response Center (may be

part of Corporate Security Department) • Chief System Operator

Event Response and Recovery Functions: • Incident/Area Commanders • Emergency Response/Preparedness • Transmission Operations/Control Centers • Transmission Field Operations • Cyber/Physical Security • Media Relations/Corporate Communications • Government Relations

Example Titles: • Manager, Work Management • VP, Transmission Construction and

Maintenance • Senior Director, Transmission Field Operations • Senior Director, System Operations • Senior Director, Distribution Operations • Director, Gas Emergency Response

May need both transmission-focused SMEs as well as SMEs with enterprise-wide responsibilities

12



Transmission and Supporting Equipment Management

Functions: • Transmission Asset Management • Transmission Operations • Transmission Engineering Standards (includes

Lines and Substations) • Transmission Design, Construction and

Maintenance (includes Lines and Substations) • System Protection and Control • Telecommunication Systems • Security Services

Example Titles: • Senior Director – Transmission Asset

Management • Senior Director, Transmission Field Operations • Director, Transmission Business Development • Senior Director, Distribution Operations • Director, Gas Emergency Response

Note: This domain is applicable beyond just transmission equipment (typically managed in a dedicated Asset Management department.) The scope includes assets such as people, software tools, telecommunications, databases, OT, etc.

Information Sharing and Communications

Functions: • Telecommunications Systems • Media Relations/Corporate Communications • Government Relations • Human Resources • Business Continuity • System Operations • Cyber/Physical Security

Example Titles: • Senior Director of Communications • Director, Cybersecurity • Director, Physical Security • Chief System Operator

If a company has multiple operating companies, there may need to be representation for all; can help identify efficiencies and alignment.

Supply Chain and Critical Entities Management

Functions: • Supply Chain/Procurement • Transmission Customer Account Management

Example Titles: • Director of Strategic Sourcing, in Supply Chain

Department • Director, Procurement • Director, Cybersecurity

Note: This domain is applicable beyond just cyber supply chain. The scope includes both supply chain for cyber assets (including relays, telecommunications equipment, etc.) and other critical entities/interdependencies such as gas pumping stations.

Transportation Management Functions: • Transportation/Fleet Services (including rail) • Fuel procurement

Example Titles: • Director of Fleet, part of Supply Chain

Department • Director, Transportation • General Manager, Transmission Operations

13



Workforce and Family Care Management

Functions: • Human Resources • Transmission Operations • Training

Example Titles: • Director, Human Resources • Director, Transmission Training and Workforce • General Manager, Transmission Operations • Manager, Enterprise Preparedness, part of

Director of Security Department

In addition to SMEs discussed above, the TRMM coordinator should identify support staff that may be required to assist in conducting the self-assessment (e.g., IT support, audio/visual, etc.)

Although not required, it is helpful if the SMEs, executives, and other participants are familiar with the TRMM prior to beginning the self-assessment or conducting the assessment introduction presentation. The TRMM coordinator or facilitator can help prepare participants by providing them with TRMM documentation for review or providing introductory briefings.

Past assessments have shown that a good practice is for the SMEs to review the practices and assign pre-scoring6 to as many as possible. When there are multiple SMEs for a domain, objective, or practice, it is good for pre-scoring to be done individually by each SME and then for follow-up conversations to be held (either before or during the assessment) to reconcile differences and determine the organization’s overall score.

2.10 Scheduling the Self-assessment In collaboration with the sponsor and support staff, the TRMM coordinator schedules the self-assessment. Assistance from the sponsor or executive management might be necessary to raise the level of priority for the TRMM assessment to make room on the calendars of SMEs and other critical participants during the assessment period. Tasks in scheduling include but are not limited to the items listed in Table 2.4.

Assessments are typically conducted over a period of a few days. This allows for a series of appropriately spaced interviews with individual subject matter experts (SMEs) or groups of SMEs in a manner that makes efficient use of their time and expertise. For example, the pilot assessments for the TRMM were conducted over the course of two and a half days, including the introduction presentation, interviews with the SMEs, and the exit presentation.

6 Pre-scoring is SMEs providing an initial scoring based on their interpretation prior to any facilitation or collaboration

14


Table 2.4: Steps and Activities Involved in Scheduling the Self-assessment

Task Description Identify the date(s) for the assessment based on the availability of the sponsor and participants Send invitations to selected participants (as described in Section 2.6) Request that the sponsor communicate to the invitees the importance of the process and their active

participation Ask for acknowledgments and confirmation from invitees Set expectations and restrictions for invitees about sending alternates Ensure there are sufficient confirmed participants to conduct the self-assessment

2.11 Planning and Logistics of the Self-assessment Thorough logistical preparation is necessary to ensure a successful self-assessment. In collaboration with support staff, the TRMM coordinator is expected to plan for all logistics, including but not limited to the tasks in Table 2.5.

Table 2.5: Planning and Logistics Tasks

Task Description Identify and reserve appropriate meeting space for the self-assessment Communicate requirements for the meeting spaces to the support staff (e.g., type and quantity of computer

projectors; Audio/Video equipment; dry-erase boards and pens; easels, easel pads, and markers) If using external facilitators, communicate necessary information and logistics (e.g., company and area

courtesy map, hotel information, organization overview information, including relevant company and Incident Command Structure (ICS) org charts

Test all the tools (hardware and software) ahead of time Coordinate travel arrangements as necessary Arrange for catering as necessary Arrange for building access for all participants Establish non-disclosure agreements (NDAs) if necessary (e.g., if some of the participants are not members

of the organization)

15

Transmission Resilience Maturity Model Facilitator Guide ASSESSMENT

3. ASSESSMENT This section describes the second phase of the TRMM self-assessment process – the assessment itself. Assessments can be conducted either in person or virtually. While in-person assessments are recommended, successful assessments can be completed virtually as well. Some of the subsequent sections are specific to in-person assessments and are noted as such. There are also several items in the subsequent sections that are recommended, but not imperative. Those have been marked as optional.

3.1 Managing the Assessment It is highly recommended for facilitators to cover the scribe roles below:

• Tool entry scribe (recommended, but optional – only if actively using the tool during the interview process)

o Final scores per SME consensus

o Pertinent notes per comments that will help the organization revisit specific items and understand the basis for a score

• “Hardcopy” scribe

o Most importantly, document the ‘final answer’ grade the organization picks (fully, largely, partial, not implemented)

o Document potential changes or improvements to the practices

Pay attention to questions the SMEs stumble over, that were unclear, that the team had to explain more, etc.

Include adequate notes so they can be addressed afterwards (the entire assessment team should be doing this as well in order to capture all perspectives)

Note spelling errors, inconsistent wording or phrases, illogical MIL levels (that should be higher or lower), etc.

o Not necessary to document ‘evidence’ or the organization’s reasoning for the MIL they picked

However, if some of their discussions have the potential to help clarify a practice, it is not bad to note information that could help improve the model

• Timekeeper

o Ensures the assessment stays on schedule

o Note the start/stop times for each objective.

These times can be rolled up afterwards to determine the length of time spent on each domain

16


Helps with pacing and timing analysis for the after-action discussion

o Track progress of practices discussions and prompt actions to remain on schedule.

See example in Table 3.1 below.

Table 3.1: Example Time Tracking Tool During an Assessment

Domain Total # of Practices

Minimum Timeslot

Suggested

(hours)

[Organization] Scheduled Timeslot

(hours)

1. Resilience Program Management 29 1.25 1.5

2. Risk Identification, Assessment, and Management

48 2.0 2.0

3. Situational Awareness 26 1.25 1.5

4. Event Response & Recovery 40 1.5 1.5

5. Transmission and Supporting Equipment Management

33 1.25 1.5

6. Information Sharing and Communications

28 1.25 1.5

7. Supply Chain and Critical Entities Management

37 1.25 1.5

8. Transportation Management 18 0.75 1.0

9. Workforce and Family Care Management

48 2.0 2.0

Having facilitators fill the scribe roles above allows the SMEs and TRMM coordinator to focus on the assessment and related discussions. It is recommended to have multiple facilitators (at least two) to fill the scribe roles in order for the lead facilitator to focus on the discussion and help the organization arrive at a consensus response. When completing an assessment with two facilitators, the lead facilitator can perform the tool entry duties. (This has worked quite well in past assessments.) Another facilitator performs the “hardcopy” scribe roles, using a computer or written notes (e.g., using the blank paper copy of the TRMM) to capture responses and notes. This separate notetaking is a great backup to the tool itself in case any issues are encountered.

Table 3.2 lists general assessment management tasks. More specific tasks for room set-up are provided in section 3.2.

17


Table 3.2: General Assessment Management Tasks for Assessment Days

Task Description Lead greetings and introductions for all sessions, as necessary Provide information packets, as needed Encourage feedback and questions from all participants throughout Facilitate interviews Engage, contribute, ask questions and provide feedback as necessary Identify gaps, action items, changes in approach Assist in identifying common threads and observations by the team Check with TRMM coordinator/SMEs on how the assessment is going and determine any changes

necessary Work with entire team to finalize the tool inputs Produce final report and create exit presentation

3.2 Setting up the Meeting Rooms If the assessment is taking place in person, the TRMM coordinator should ensure that the rooms are properly configured to conduct the assessment prior to the scheduled start times.

If the assessment team will use the TRMM tool during the interviews to log scores and notes, appropriate pre-testing of the tool is necessary. Using the current version of the TRMM tool, a facilitator should validate that the tool is operational on the computer(s) to be used during the assessment. One or more (for backup purposes) personal computers should be available to conduct the assessment.

One personal computer should be connected to the projector to show introductory information and the tool for the participants in the room. To support this, the TRMM coordinator will ensure wireless access to the internet in all meeting rooms and interview locations. If some participants are remotely attending the briefings, the remote meeting technology should be tested to ensure TRMM products and presentations materials are properly displayed and the audio and visual equipment are working. When external facilitators are involved, the TRMM coordinator should coordinate with them to meet either the day before or early on day one in order to ensure internet access, presentation projection, etc. are working properly on external computers.

Table 3.3 lists the typical room preparation tasks.

Table 3.3: Room Set-up Tasks for Day of the Assessment

Task Description Sufficient seating is available for all expected participants and any observers The room is set up to facilitate dialog among participants For remote participants, adequate web and voice systems are in place and tested (This should be used as a

last resort as it is recommended to have SMEs attend in person) The screen is visible to the participants Flip chart paper and/or white boards (with markers) are available if needed

18


Task Description Test all the tools (hardware and software) The TRMM tool is accessible using the entity’s wireless access to the internet and the assessment team has

access to that wireless connection in the briefing rooms and all interview rooms (highly recommended, but optional. Assessment interviews can be done with pen and paper)

3.3 Kicking Off the Assessment The “TRMM Introduction” session kicks off the assessment. Typically, the audience invited includes all participants (e.g., all facilitators, SMEs, the TRMM coordinator, constructive critic, the executive sponsors, TBU line management). It is often useful to have this session begin with comments from organization senior management. These comments can help emphasize the importance of:

1. resilience (in general)

2. the TRMM assessment

3. identifying areas of strength, superior practices, and areas needing improvement

4. the active participation of the attendees

Next, the TRMM coordinator delivers a presentation to overview the TRMM assessment process. The purpose of the presentation is to establish a common, foundational understanding for the participants regarding resilience and maturity models in general, as well as the TRMM content and its objectives and scoring.

TRMM participants should be reminded that the model is intended to provide a snapshot of the maturity of the organization’s resilience posture. Each utility will have their own “journey” reflecting their unique situations. The TRMM coordinator should ensure that the participants are prepared for the self-assessment and feel comfortable giving responses about organizations performance regarding the resiliency program.

Table 3.4 describes several topics that experience has shown deserve special emphasis prior to beginning the assessment.

Table 3.4: Topics for Discussion at the Start of the Assessment

Topic Discussion

TRMM definitions Having a copy of the glossary of terms from Appendix C of the TRMM is useful for discussions during the assessment. Participants can review this information prior to and during the assessment.

Organization’s vocabulary

Discussion of terms found in the TRMM may prompt discussions relating to terms used within an organization. Although not all terms can be anticipated in advance, this discussion is useful to highlight possible conflicts.

Agreed-upon function and scope

It is important to remind the participants that the assessment is being applied to a specific set of activities performed by the organization and to describe those activities prior to beginning the assessment. For example, it might be helpful to remind the participants that the scope is “transmission” and not “generation” or “distribution,” etc.

19


Topic Discussion

Implemented practices When completing the TRMM, participants must consider practices as they are implemented on the day the TRMM is evaluated. Do not consider activities that are planned or in the process of implementation, being updated, being considered, etc. Likewise, do not consider practices that have not been performed for extended periods of time. For example, if the organization has a disaster recovery plan for its computer systems which, in the opinion of the participants, is out of date to the point of being unusable, then that plan should not be considered even though a document technically exists.

Four-point response scale

Participants use a four-point response scale (i.e., fully, largely, partially, and not implemented) to evaluate the degree to which the organization has implemented each practice. Review with the participants the meaning of each of the four response options so that all participants have a common understanding of when a particular response will be used. This is a crucial item to the success of the assessment, and may need to be reviewed several times, especially when new participants join a session.

Follow-on activities The sponsor and TRMM coordinator set the expectation for the assessment and the roles of the participants. It is important to discuss how the assessment results will be used within the organization. The sponsor and TRMM coordinator should emphasize that next steps will be based on the organization’s risks and maturity. The output of the TRMM should drive conversations about risk and continuous improvement. The results may also prove helpful in planning future reviews of their programs to track progress and validate goals. The sponsor and TRMM coordinator should also point out the roles of participants in these follow-up activities.

3.4 Facilitating the Assessment A facilitator guides the participants through the TRMM questions which fall within their domain. Open dialog and consensus building among multiple SMEs is important in completing the assessment.

To begin each meeting with the SMEs, the facilitator should introduce the agenda for an interview session, provide an overview of the TRMM, review meeting guidelines, and review the scoring system for the practices. Next, the facilitator should read the description of the domain, the first objective, and the first practice verbatim. The facilitator should describe the intent of the practice and remind participants of the scoring guidelines.

The facilitator documents the score given by the SME(s) for the practice. Results are typically recorded using the TRMM tool but can also be recorded manually (for later transcription into the TRMM tool). Notes regarding the discussions should be documented to record the rationale behind the responses given. The TRMM tool provides a space for documenting notes.

There is significant value in allowing participants to interact and discuss as a group what the consensus answer will be rather than using individual responses. Consensus for each practice being assessed has been achieved when every applicable participant feels that their views have been heard and when the majority of these participants feel they can support the proposed

20


scoring. The facilitator assists the SMEs representing a domain in formulating high-quality, consistent responses based upon consensus.

At times, the facilitator should remind participants not to get stuck on the specific phrasing of a practice but to focus on its intent. The facilitator has several resources that may be useful in breaking any impasse:

▪ TRMM User Guide, specifically the examples in section 7

▪ TRMM Supplementary Explanations document

▪ Facilitator’s resilience knowledge and past TRMM assessment experience

The facilitator should progress through all practices in an objective, followed by a quick review of the scoring for the objective’s practices. Sometimes, the SMEs might want to change the scoring for one or more practices based on what they learned. That is perfectly acceptable.

Continuing, the facilitator should move on to the next objective and its first practice. The process described above is repeated until all practices in each objective and domain are covered.

3.5 Processing the Collected Data The TRMM tool can generate a TRMM Core Assessment Report (the “TRMM report”) with the click of a button after one or more domains are completed. Partial TRMM reports can be generated to provide preliminary results prior to answering all practices. After all practices have been assessed in each domain, the facilitator can generate a complete report. The report may be either downloaded or used online (i.e., within the tool) where additional interactive displays provide more details. Please note that the downloaded PDF version has a slightly different look due to different background colors as compared to the online, interactive report. For more information on how to understand and interpret the various charts and results in the TRMM report, reference the TRMM User Guide.

3.6 Presenting the Core Assessment Report The facilitator will use the TRMM report to prepare an end-of-assessment exit presentation for the sponsor, organization management, and review participants. The presentation typically centers on overall results, complemented by a few highlights for specific areas, depending on the assessment discussions and results.

3.6.1 Domain-level Charts

For example, it is common to begin the presentation of results with the domain-level bar charts. Figure 3.1 provides an easy-to-understand overview of the organization’s maturity levels. The solid blue bar, brightening in color as it increases in height, presents the achieved MIL. The shaded or darkened portion of the bar above the top of the achieved MIL indicates the status of the progression toward the next MIL (i.e., the percentage of the practices that evaluate the next MIL level that are fully or largely implemented).

21


Figure 3.1: Sample Bar Chart Presenting MIL + Progression Toward the next MIL for each

Domain

Figure 3.2 shows the traditional C2M2 donut chart. It provides more detailed information on the distribution of the practice scores for each domain and each MIL.

Along with the message of the organization’s overall results, the facilitator should emphasize that simply getting high MIL scores is not the goal of completing a TRMM assessment. Some practices do not make sense for an organization based on its risk profile. Depending on risks, budgets, stakeholder priorities, and other factors not all transmission resiliency programs need, or even aspire, to reach MIL3. For some, a MIL1 or MIL2 maturity level in selected domains may be acceptable.

22


Figure 3.3: Sample Donut Chart Domain Summary of an Assessment

23


It can be helpful to point out to the participants the specific areas preventing the TBU from achieving a higher MIL within any domain. This can be shown using the donut chart (reference Figure 3.3) and reviewing areas in red or orange that represent “not” or “partially” implemented practices, respectively. These must transition to “largely” or “fully” implemented scores to allow the transmission resilience program to achieve a higher MIL. Therefore, the red and orange areas of the display offer some initial insights into potential investments in resilience program and process improvements.

3.6.2 Objective-level Charts

Following the domain-centric results, a review of similar charts but with more granularity can be instructive. The objective-level results, also presented in both bar and donut charts (see examples in Figures 3.4 & 3.5, respectively), present additional detail by providing a graphical representation of the practice responses for a domain by its objectives (as opposed to the breakdown at the domain level in section 3.6.1 above). This helps the organization pinpoint more specifically the areas for focus.

Figure 3.4: Sample – Workforce and Family Care Management Domain Bar Chart Presenting

MIL + Progression Toward the next MIL by Objective

24


Figure 3.5: Sample – Workforce and Family Care Management Domain Donut Chart Summary

by Objective

3.6.3 Summary of Identified Gaps

Follow-up activities will be discussed in more detail in the next chapter but should be a part of the exit presentation which should highlight gap areas and potential next steps. This can be supported by the information in the TRMM report’s “Summary of Identified Gaps” table (see sample excerpt in Figure 3.6). This table summarizes, in one convenient location, practices that were marked as “Not Implemented” or “Partially Implemented” and organized by domain.

Figure 3.6: Sample Excerpt of Summary of Identified Gaps Table for Workforce and Family

Care Management Domain

25


It is recommended that the facilitator(s) provide an opportunity for all participants to make any last comments or observations, as well as provide an opportunity for the sponsor to make closing remarks.

Lastly, be sure to thank all participants, collect all relevant materials, and securely dispose of all materials that must not be removed from the premises after the completion of the exit presentation.

3.7 Treatment of All Self-Assessment Materials All data collected and model results belong solely to the organization undergoing the self-assessment; no data is retained by or shared by the TRMM tool. All input and results are kept on the local computer on which the assessment is done. The electronic files involved in the self-assessment, including the generated reports, are the property of the organization undergoing the self-assessment. The TRMM coordinator should be given these files upon completion of the assessment. The TRMM coordinator should collect any other notes taken but not entered into the tool and consolidate them with their own notes in preparation for working with the sponsor to plan follow-up activities.

26

Transmission Resilience Maturity Model Facilitator Guide FOLLOW-UP ACTIVITIES

4. FOLLOW-UP ACTIVITIES This section describes the third phase of the self-assessment process, including analyzing identified gaps, prioritizing and planning, and implementing plans.

Follow-up actions are guided in part by:

▪ the organization’s TRMM self-assessment, which determines maturity levels of the topic areas represented by the TRMM domains and identified gap areas

▪ a subjective prioritization of each domain’s practices considering applicable regulations, the organization’s mission/vision/goals, corporate values, and risk tolerance.

The TRMM coordinator should remain engaged in follow-up activities, as familiarity with the TRMM content, assessment, and results can help identify appropriate actions.

4.1 Analyzing Identified Gaps Once the assessment is complete and the organization has its TRMM report (reference section 3), the next step is to understand how the organization is scored in comparison to its target profile.

The target profile is the desired maturity level, or MIL scores, for the TRMM domains that align with the organization’s business and resilience goals. Note that typically it is not optimal for organizations to strive to achieve the highest MIL in all domains.

There are two common approaches for identifying a target profile. The first approach, which involves using the results of the TRMM assessment to identify the profile, is often adopted by organizations that are new to using the TRMM and have not previously established a target profile. The second approach, which involves walking through the practices before performing an assessment, is most typically adopted by organizations that have more experience and familiarity with the model. These methods are discussed in more detail in the subsequent sections.

4.1.1 Setting a Target Profile – Method 1

In this approach, an organization uses the results of a completed TRMM assessment to jumpstart the identification of a target profile. The organization begins by walking through the results section in each domain and performing the following steps:

1. Review the “Summary of Identified Gaps” section to identify all the practices that have been scored as “Not Implemented” or “Partially Implemented” per Domain

2. For each practice identified in step #1, review the practice and determine whether the practice needs to be performed to meet the organization’s business and resilience objectives.

3. If yes, then document that practice.

27


4. If no, then move on the next “Not Implemented” practice.

5. Repeat for all nine model domains.

Once this review is complete, the organization should have a documented list of practices that needs to be started and performed at least at the “largely” implemented level. This list, in combination with the TRMM report’s list of practices already performed at the “largely” or “fully” implemented level, sets the organization’s target profile.

An advantage of this method is that the generated list of practices needed to be performed at least at the “largely” implemented level also serves as the list of gaps to be addressed. This list of gaps gives the organization a starting point for prioritizing and planning improvements to its resilience capabilities.

4.1.2 Setting a Target Profile – Method 2

In this approach, an organization walks through the TRMM practices before undergoing an assessment to identify its target profile. The organization begins by walking through each of the practices in all domains of the model and performing the following steps:

1. Review the practice and determine whether the practice needs to be performed to meet the organization’s business and resilience goals.

2. If yes, then document that practice.

3. If no, then move on to the next practice in the domain.

4. Repeat for all nine model domains.

Once this review is complete, the organization will have a documented list of practices that it believes it needs to perform to meet its goals. This listing of practices is the organization’s target profile. The target profile can then be compared against the results of the assessment to determine where gaps exist that need to be addressed.

4.2 Prioritizing and Planning After the gap analysis is complete, the TRMM coordinator should work with the sponsor to prioritize the actions needed to achieve the desired domain maturity levels. The prioritization should be done using criteria such as: how gaps affect organizational goals, including performance of the organization’s infrastructure; the criticality of the business objectives supported by the domain; the cost of implementing the necessary practices; the time necessary to implement certain practices; and the availability of resources to implement the practices. A cost-benefit analysis for gaps and activities can inform the prioritization of the actions needed.

Next, a resilience enhancement plan is developed to address the selected gaps in a methodical manner. The plan may recognize that some gaps are easily and quickly addressed, and these actions may be prioritized and implemented ahead of others to achieve immediate successes. To address the identified gaps more comprehensively, the resilience enhancement plan will likely span a period of years (e.g., three to five years depending on the extent of improvements needed to close the selected gaps and achieve the desired capabilities.) The sponsor would ideally be the owner of the plan, although responsibility for implementation might be assigned to person(s) designated by the sponsor.

28


4.3 Implementing Plans The sponsor must provide adequate resources for the resilience enhancement plan to be a success. This includes people with the necessary skills to accomplish the planned tasks and a budget that will allow them to be successful. In addition, the sponsor must continue supporting the execution of the plan through tracking progress and recognizing accomplishments.

After plans have been developed and implemented to address selected gaps, the sponsor should ensure organizational goals are periodically re-evaluated to check for changes in desired capabilities. In addition, a periodic re-assessment using the TRMM can track progress towards the organization’s desired target profile.

29

Transmission Resilience Maturity Model Facilitator Guide SUMMARY

5. SUMMARY This document describes how an organization should prepare and conduct a TRMM self-assessment. This guide contains information about how to prepare for the self-assessment, how a facilitator assists the organization in evaluating the maturity of its resilience program, procedures, and capabilities during the assessment, and guidance for follow-on activities to prioritize and implement a plan to close identified capability gaps.

For additional assistance, participants can visit the TRMM website: https://trmm.labworks.org/. For contact information, visit the “Contacts” page.


https://trmm.labworks.org/Contact

30

Transmission Resilience Maturity Model Facilitator Guide APPENDIX A

APPENDIX A: FREQUENTLY ENCOUNTERED DISCUSSIONS Experiences using the model and facilitating self-assessments have revealed many topics that commonly surface during discussions. The facilitator should prepare for these discussions in advance. The most common discussion topics are documented below.

A.1 Discussions Relevant to the Entirety of a TRMM Self-Assessment

Scope of resilience

Because the definition and scope of resilience is not as well-known as reliability, SMEs will often revert to assessing the TRMM practices according to reliability criteria. This is also indicative of the overall level of maturity of the industry. Resilience is a much newer term with fewer concrete bounds. Adding to the level of complexity, resilience for one entity is not the same as for another due to the threats specific to them (e.g., inherent threats due to their infrastructure design, their geographical location, etc.) It is vital to the success of the assessment that SMEs assess the practices looking through a resilience “lens.” To facilitate this, the TRMM presentation templates and the User Guide include information and visuals explaining the definitions and scope of resilience to be considered when assessing implementation levels per the TRMM.

The distinction between largely implemented and partially implemented

Participants will arrive with their own ideas of what these responses mean. The facilitator must provide a means for the group to come to a consensus on a definition of these responses early on so that the response has a consistent meaning throughout the assessment process. A useful technique is to ask, “How many actions do we need to take before we can consider this practice fully implemented?” If participants name more than one action, the practice should be considered partially implemented. If only one action is required, or the group views the actions described as minor, consider the practice largely implemented. The facilitator should record what action(s) the group articulates. This information can be useful to the organization when reviewing the core assessment report and planning follow-up actions.

The meaning of “implemented in an ad hoc manner”

When reading the domain-specific practice questions in the TRMM, you will encounter the phrase “at least in an ad hoc manner.” All MIL 1 practice questions in the TRMM contain this phrase. If the participant is familiar with the TRMM only through review of the model documentation, the participant will not encounter this phrase while reviewing the domain-specific practices. It is good practice to keep the glossary handy for this discussion.

It is important to remind participants that even ad hoc practices must meet business and operations objectives to be considered fully implemented.

31


Reminders when reviewing common objectives

The last few practice questions in each domain are listed under the heading “Management Activities.” These questions are similar in criteria for each domain, but the phrasing of each question often changes to focus on the domain at hand.

These questions help the organization determine the degree to which practices have been institutionalized—that is, the extent to which a practice or activity is established in an organization’s operations. The more established an activity, the more likely it is that the organization will continue to perform the activity over time.

When discussing the common objectives and practices, it is important to remind participants that their responses to these questions should consider the specific domain and all its objectives and practices. If participants appear to be arriving at their responses too quickly, it is often worth rephrasing or re-asking these questions.

A.2 TRMM Domain-Specific Discussions Each domain begins with a purpose statement and introductory material. Reading this purpose statement and allowing participants to view the introductory material helps prepare participants to begin the new domain.

As each domain is addressed in the TRMM, there can be questions about unfamiliar terms and concepts as well as uncertainty about how to answer some questions. The explanations provided below address many common discussion points. This subsection is organized according to TRMM domain, and its content can help participants to better understand the intent of the questions.

Risk Identification, Assessment, and Management Domain

What is risk management?

Risk management is a key component of the TRMM. Each facilitator should understand the concept of risk management before beginning the survey. In addition, the TRMM introduces a new methodology for evaluating risk since resilience risks are often high impact, low frequency (HILF) with limited information on likelihood accompanied by often unacceptable impacts. Reference the User Guide, domain #2 section for more information.

The TRMM self-assessment can assist utilities in identifying gaps in their adoption of a risk management plan across an organization. The TRMM self-assessment examines how utilities have constructed an enterprise risk management strategy and risk management program and asks about the use of enterprise-derived criteria within key risk management practices. It also requires that organizations investigate their practices for developing and stabilizing important resilience-related practices and ensure those practices are consistent and institutionalized.

What are risk criteria?

Risk criteria articulate an organization’s tolerance for risk as well as its risk response approaches. Linking resilience risks to organizational risks in a defined and documented manner is a reflection on the overall maturity of the organization’s risk management program. Participants should focus on their response to this practice question without regard to the

32


dependency between the implementation of risk criteria and responses to practice questions in the various domains. This dependency can be articulated as those practice questions are asked.

What is a risk register, and why is it important?

A risk register is a structured repository where identified risks are recorded to support risk management. Documenting and recording risk in a risk register ensures that these risks are monitored and addressed in a timely manner and assists in identifying trends. As a MIL 3 product, the risk register represents a resource developed and maintained by an organization with mature risk management practices.

Situational Awareness Domain

Several practices require that a common operating picture (COP) be established and maintained. It is important to emphasize to the participants that the goal refers to establishing an aggregated, near-real-time understanding of the operational state being examined. It does not necessarily require that a visual representation be rendered. The emphasis should be on developing an understanding of the state of operations, not the manner in which this understanding is visually conveyed.

Supply Chain and Critical Entities Management Domain

The facilitator should reinforce the difference between the three types of critical entities covered by this domain.

1. Supply chain critical entities are providers that have a formal, contractual relationship with the TBU or enterprise to provide essential products or services to the TBU. Whether a provider falls into this category is based upon the formality of the relationship. These entities are much better known and typically actively managed in the TBU’s supply chain process.

The other two categories of critical entities are dependent on the TBU (either directly or indirectly through a distribution provider) for their electricity needs.

2. Electricity subsector critical entities provide essential products or services to the TBU or other electricity subsector entities. Identifying these entities requires more research and rigor since there is not an agreement with the TBU. Oftentimes, this is due to these entities being only indirectly connected to the TBU, however dependent on them for their electricity supply. These entities are often referred to as interdependencies, such as gas compressor stations, generators, and the Bulk Electric System (BES), of which the TBU is a part.

3. Society critical entities provide essential products or services to society at large. Similar to category 2, identifying these entities requires more research and rigor. These are critical entities that supply public health and safety products and services such as water, telecommunications, and medical services.

33

Transmission Resilience Maturity Model Facilitator Guide APPENDIX B

APPENDIX B: REFERENCES

[ES-C2M2] Cybersecurity Capability Maturity Model (C2M2). <http://energy.gov/node/795796> Electricity Subsector Cybersecurity Capability Maturity Model (ES-C2M2). <http://energy.gov/node/369271>

[TRMM User Guide]

Electricity Subsector Transmission Resilience Maturity Model (TRMM) User Guide <https://trmm.labworks.org/Resources/TRMM-User-Guide-v1.0.pdf> Contains comprehensive information regarding the TRMM, including links to pertinent supporting documents such as this Facilitator Guide.

[TRMM Tool] Online Transmission Resilience Maturity Model (TRMM) Assessment Tool <https://trmm.labworks.org>

TRMM Supplementary Explanations

Transmission Resilience Maturity Model (TRMM) Supplementary Explanations. <https://trmm.labworks.org/Resources/TRMM-Supplementary-Explanations-v1.0.pdf> Provides listing of TRMM Domains, Objectives, and Practices alongside further explanations of the practices, as deemed necessary.






Date post:	17-Oct-2021
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

ELECTRICITY SUBSECTOR TRANSMISSION RESILIENCE …

Documents