Date post: | 29-Dec-2015 |
Category: |
Documents |
Upload: | octavia-robertson |
View: | 214 times |
Download: | 0 times |
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | Apr 19, 2023 |
External Review of Nodal Program PMO Control System
Executive Discussion
Electric Reliability Council of Texas, Inc. (ERCOT) Board of Directors MeetingMay 16, 2007
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | Apr 19, 2023 |2
Agenda
Background
Findings
Recommendations
Other Topics – Software Effort Estimation
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | Apr 19, 2023 |3
Nodal Market Protocols and Requirements
Nodal Program People
Plans, Policies, Templates, Tools and
Techniques
Results
Actions
Nodal Program Management Control System provides order to the Actions of Nodal Program
X X =Nodal
DeliveryERCOT
Readiness
Market Participant Readiness Nodal
Success
ERCOT Nodal ProtocolsSection 2: Definitions and
AcronymsAugust 1, 2006
(Effective Upon Texas Nodal Market Implementation
ERCOT Nodal ProtocolsSection 2: Definitions and
AcronymsAugust 1, 2006
(Effective Upon Texas Nodal Market Implementation
ERCOT Nodal ProtocolsSection 1: Overview
August 1, 2006(Effective Upon Texas Nodal Market
Implementation
ERCOT Nodal ProtocolsSection 5: Transmission Security Analysis and
RUCAugust 1, 2006
(Effective Upon Texas Nodal Market Implementation
ERCOT Nodal ProtocolsSection 6: Adjustment Period and Real-Time
Operations(Effective Upon Texas Nodal Market
Implementation
ERCOT Nodal ProtocolsSection 4: Day Ahead
OperationsAugust 1, 2006
(Effective Upon Texas Nodal Market Implementation
ERCOT Nodal ProtocolsSection 7: Congestion
Revenue Rights(Effective Upon Texas Nodal Market
Implementation
Triangle of Balance
Scope
Schedule Cost
Business Needs
Program Management Control System RUP Other ERCOT Policies and Procedures
Produce Influence Influence
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | Apr 19, 2023 |4
The Program Management Office Control System (PMOCS) supports program direction and increases productivity
Limited Experience
Program Performance
Highly Skilled
Low HighProgram Management Office System Structure
(PMOCS) Strength and Performance Level
Individual
Project Manager’s Skills
Probable Success
Probable Success
Highly Probable Success
Success Unlikely
Program Critical Path Plan Project Schedules Program Resource Plans and
Commitments Program Budget Project Estimates and
Forecasts Progress Reporting Scope and Change Controls Risk Management Plans Issues Tracking Communications Plans Deliverables Controls Quality Management Plan Technical Support Tools –
Version Control, Configuration Management, etc.
Other Program Management Tools and Techniques
Examples of PMOCS ContentThe
PMOCS involves fourteen
categories of support
Change Control
Commun
icatio
ns
Manage
ment
People Management
Supplier Management
u
Project/Program Definition St
akeh
olde
r
Enga
gem
ent
Organization
Managem
ent
Technical EnvironmentIs
sue
Man
agem
ent
Risk
Management
Deliverables Management
Tracking &
Control
Quality
Managem
ent
Work Plan
Management
PMOCS Elements PMOCS Impact
Triangle of Balance
Scope
Schedule Cost
Business Needs
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | Apr 19, 2023 |5
The IBM reviews consider PMOCS structure and use
This is the 3rd Review and it included: All fourteen categories of control with the PMOCS Assessment of the current state of the PMOCS Structure Evaluation of PMOCS Performance Consideration of a sub-set of the Nodal Program Projects:
• Project Management Office (PMO)• Enterprise Integration (EIP)• Energy Management System (EMS)• ERCOT Readiness and Transition (IRT)
Review of plans and other documents, interviews and other testing Provision of recommendations for improvement
Review based on PMOCS in place as of March 20, 2007
Progress Review Schedule includes five more reports:
July 2007 – Report 4 October 2007 – Report 5 January 2008 – Report 6 March 2008 – Report 7 Final 2009 – Report 8
Note: Program Management Control System (PMOCS)
Communicatio
ns
Management
People Management
Supplier Management
Change Control
u
Project/Program Definition S
take
hold
er
Eng
agem
ent
Organization
Managem
ent
Technical EnvironmentIs
sue
Man
agem
ent
Risk
Management
Deliverables Management
Tracking &
Control
Quality M
anagement
Work Plan
Management
PMOCS Areas: Fourteen Control
Categories
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | 19 Apr 2023 |6
Findings – The PMOCS is Operational, but certain areas of structure can be improved
Most components meet best practices standards
A few of the components in the PMOCS do not meet structural expectations:
Work plan integration and content Contractor extension procedures Organization target and decision framework Internal stakeholder communications Integrated testing phases
Source: March 2007 ERCOT PMOCS Structure Analysis
0
2
4
6
8
10
12
14
PMOCS Structure Assessment
Areas Needing Improvement
Areas Meeting or Exceeding Design
Standards
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | 19 Apr 2023 |7
Findings - The PMOCS is working in most areas, but certain areas of performance can be improved
Most components of the PMOCS are being used effectively
A few of the components in the PMOCS do not meet performance expectations:
Slow migration to single Risk and Issue Management tool usage and contingency planning
Undisciplined updating of Project Schedules Absence of re-estimate triggers Low compliance with Quality Management
Reviews as required by Nodal Program
Source: March 2007 ERCOT PMOCS Performance Analysis
0
2
4
6
8
10
12
14
PMOCS Performance Assessment
Areas Needing Improvement
Areas Meeting or Exceeding
Performance Expectations
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | 19 Apr 2023 |8
The PMOCS has been improved, but there are some areas that require emphasis and focus
Report 1 Report 2 Report 3
Control Dimension 10 Oct 2006 15 Jan 2007 24 Apr 2007
Program Definition
Stakeholder Engagement
Change Management
People management
Supplier Management
Risk Management
Issue Management
Communications Mgmt
Work Plan Mgmt
Tech Environment
Deliverables Mgmt
Organization Mgmt
Tracking and Control
Quality Management
IBM Review Reports
Control A
ctivity
Meets or Exceeds
Under
Perform
Structure
Structure and Performance Ratings
Significant Improvements Needed
Effective Controls
Combined Review Ratings
Minor Improvements Needed
Not Evaluated
Moderate Improvements Needed
Unsatisfactory
Trend
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | 19 Apr 2023 |9
Recommendation Impact Action
Integrate work plans Reduced PM administrative burden and improved information
Simply process with use of software to support project plan and task lists
Forecasts linked to productivity and deliverables in work scheduled
Ability to manage budget variance Establish and monitor Productivity Measure to trigger need for budget review
Update Communications Plan to ensure specific cascading communications
Directed communications to all organization levels
Provide more detail in existing plan to address internal communications actions
Accelerate transition process for single Risk and Issue Tool
Issues are not being tracked even if evaluated; risks are not completely evaluated and mitigation not documented
Review progress and make course adjustments
Determine ERCOT Organization target and components of employee transition
Consistent decision framework for ERCOT Transition
Leadership team define tenets of target
Simplify approval process for contractor extensions
Avoid work disruption and delays Consider alternative process for approved contractors for ongoing effort.
Accelerate implementation of and compliance with QA Policy
Avoid program delays due to absence of requirements traceability
Change behaviors by placing emphasis on the process
Provide manager responsible for all testing events including EDS
Improved integration, reduction of duplication of effort, better readiness measure
Assign ERCOT owner (s) of testing process – FAT to EDS
Recommendations for Improvement – Priority for Action
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | 19 Apr 2023 |10
Other Topics – Software Effort Estimation
Variances to estimated effort and cost will exist throughout a project
Variance (risk) declines as more is known about the work
Actions to manage variance are critical
Forecasting based on effort drivers identify factors to be managed
Track and manage to metrics to reduce variance from authorized spending
RequirementsAnalysis
SystemDesign
SystemBuild
SystemInstall
.25x
2x
1.5x
1.25x
4x
.8x
FeasibilityStudy
.7x
.5x
Ex
pe
cte
d r
an
ge
of
es
tim
ati
on
ac
cu
rac
y
Software development life-cycle
x
N1 2 3 4
Software project cost as a function of life-cycle phase 1,2
1 Software Engineering Economics, Barry W. Boehm, Prentice Hall PTR, 19812 “Software Size Estimation of Object-Oriented Systems,” IEE Transactions on Software Engineering, Vol. 16, No.5, May 1990
Business Requirements
TargetToday
Potential variance from estimate (degree of risk)
Budget Estimate
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | 19 Apr 2023 |11
Forecasting, tracking and quality assurance based on deliverables provides a common measurement of progress and success
Alarm Data Model Energy Management System (example deliverable) Compare effort (budget and actual) to produce deliverable Variance analysis yields insight into effort for other similar work Manage work around deliverables Provides more better variance control Supports more accurate forecasts Provides information for program direction and control focus
Other Topics – Software Effort Estimation
Global Business Services
© Copyright IBM Corporation 2007ERCOT and IBM Confidential | 19 Apr 2023 |12
ED
S 4
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Alarms Processing
FAT
EMS 2A
ED
S 1
ED
S 2
ED
S 3
FAT – Rel 3
EMS 2B EDW 1
FAT – Release 4
NMMS 1 NMMS 2 EIP 1
FAT – Release 5
REG 1 MMS 1
EMS 2C
EIP 2 S&B 1MIS 1EIP 3
EDW 2
FAT – Release 6
EMS 3EDW 3
EIP 4
FAT – Release 7
EDW 4CRR 1CMM 1 EIP 5
FAT – Release 8
MMS 2S&B 2
EDW 5 EIP 6
FAT – Release 9
MMS 2S&B 2
OS
NMMS 3 EMS 4EDW 6
EIP 7EMS 5 DIS
P2P Checkout
SE Verification
ITEST
RTM Go-Live
DAM Go-Live
EDS 2 Release 3: SE VerEMS- State Estimator & Tel Perf Stds,NSA, SPS/RAP, SCADA and DynRatings for Single Model
EDW– SE Statistics
EDS 2 Release 4: NM VerNMMS- Network Model, NOMCR
EMS- EMS /NMMS Interface
EIP
EDS 3 Release 5: LMPsMMS– SCED
EMS- NSA, RLC
COMS- S&B (FIP, FOP,Ver Costs)- Registration
EIPMISEDW
EDS 3 Release 6: LFC TestingEMS- LFC
EIPEDW
EDS 3 Release 7: CRRCRRCOMS- CMM
EIPEDW
EDS 4 Release 8: RT SettlementsMMS- RT Settlements InterfaceS&BEIP, EDWEDS 4 Release 9: DAM, RUCMMS- All Markets- OSEMS – Load Forecast, Out EvalNMMSS&BEIP, EDW
ITEST
ITEST
LegendIntegration TestingProject FAT Activities
Settlements Statements
Upload Bids
Test Settlements
Outages DAM, RUC, SASMFull Nodal Functionality
24*7 Operation 168 Hr Test
NOMCR Chk out Network Model Verification
Rel 1
Rel 2
EDS 1 Release 1:EMS COTS
EDS 1 Release 2:ICCP Upgrade
Rel 3
FAT
EDS
FAT
ITEST
EDS
FAT
ITEST
EDS
SCED TestingRegUpload Offers
Rel 4
Rel 5
Rel 7
Rel 6
6 month LMP Posting
CRR Data Validation Trial Ops of CRR
LFC Comms Testing
ITEST
EDS
FAT
FAT
EDS
ITEST
ED
S 4 Go-Live
Activities
EDS 1 EDS 2 EDS 3 EDS 4
MP Interface Spec
Start of EDS activities
FAT milestones
EDS Preparatory Milestones
Rel 8
Rel 9
LFC Testing
ReadinessDeclaration
Readiness Declarations
Early Start
The Early Delivery Software (EDS) releases - involve multiple iterations of design and build activities – albeit to a small degree
Results of test iterations result in changes to design and build of the systems
Estimate risk should decrease after each EDS
Estimate Risk will continue exist until all work is completed
Today
Other Topics – Software Effort Estimation