March 6, 2008 TAC Texas Nodal Market Implementation Program Update Jerry Sullivan.

March 6, 2008TAC

Texas Nodal Market Implementation

Program Update

Jerry Sullivan

2Texas Nodal Program UpdateMarch 6, 2008 2

AGENDA

• Program Status

• Key Risks

• IBM Review given to BoD in February

• Quality Results

• Schedule and the Program Level “Health Checks”

• Cost

The area of emphasis at today’s TAC Meeting


Program Status: Nodal Delivery

Nodal Delivery status remains amber; key checkpoints approaching

CostScheduleScope / Quality

• Revised budget approved by ERCOT BOD, however we will be RED until fee filing approved.

• Revised Fee Filing with the PUC this week.

• Need to decide on methodology for grayboxing and/or time-boxing the handling of:

current protocols,or

future NPRRs.

• Requirements for DAM in MMS are being revised.

• Key checkpoints in place to ensure ongoing viability of 12/1/08.

• Planning for the “168 Hour Test,” which is under 200 days away.

• EDS 2 is Red, but soon may be amber.

• EDS 4 is Red, will go amber when EMS 4/5 completed.

• Number of defects discovered in EDS were high, being resolved.

• Applying increased efforts to identify and resolve defects in pre-FAT and FAT

Green Amber RedAmber

Nodal Delivery

Legend

Red

Amber

Green Estimate at Complete = <$263m

Estimate at Complete = < $263m + 10%

Estimate at Complete = >$263m + 10%

Go-live = 12/1/08

Go-live = <30 days+

Go-live = >30 days+

Program aligned with current protocols

Program aligned to previous protocol version

Program not aligned to protocols

Current product quality is not sufficient for Nodal launch

Most Nodal products currently achieve quality standards

All Nodal products currently achieve quality standards


Program Status: ERCOT Readiness (3/15/2008)

ERCOT Readiness status remains amber, but a renewed awareness of the need for preparation is apparent

• Organization, Processes and Procedures, and Training show progress– Organization

• Completed review of Deep Dive presentations including staffing/ sizing department models for Post-Nodal implementation (2009).

• Draft Vision, Mission, Core Values and Strategic Plan completed. Transition sponsors are assigned.• January Executive Readiness Survey overall indicates Amber. Key Focus Areas to work on: adequate

staffing, procedures, transition planning.• Transition plans being documented for all key departments.

– Processes and Procedures• Business Process Model completed to procedure level and ready to leverage for testing and validation

during EDS and training.• Increased focus on the creation of procedures.

– Training• Training tracking and tailoring at the department/individual level.• Validation of current individual training plans

• Concerns Remain– Adequate staffing and resources for transition from project teams to employees

– Procedures still in flux until delivery of systems

– Completion of training in time for EDS

Priority for ERCOT Staff is to ensure readiness for the “168 hour test”


Program Status: Market Participants (2/28/2008)

• Registration and Qualification [green] - Resource Asset Registration quality, as measured by metric MP11, has improved. Two QSEs represent resources that are not current with submittals for resource registration:

• Barton Chapel Wind, Austin Energy

• Engagement continues to be main area of concern.

Engagement remains [amber] for all segments except QSEs with Resources.

Continuing to work with MPs to improve their engagement scores.

13 QSEs [red] - due to Engagement. The following remain red in the scorecard:

• Citadel, Clearview, Duke Energy Ohio, Econnergy, EnergyCo, JPMorgan Ventures Energy, Juice Energy, Kansas City Power and Light, Liberty Power, LPT, Mirant, Texpo, WCW International

Next survey to be released March 17. Negative responses will be treated as red in the next round.

• Training [green] – First pass of self reported training is green for all segments except LSEs

• Connectivity and Telemetry [green] – Connectivity metrics complete for most MPs. Outliers are new to the program and need to go through EDS process

Overall status* returns to amber

* Status current as of 2.28.08


Market Readiness

• Required by: ERCOT Nodal Transition Plan (5.4.4 (3))

• Purpose: Provide an opportunity for market participants to measure and improve the accuracy and stability of the Real Time Sequence (RTS).

• Accuracy: LMPs are reasonable given the input data, but the input data needs improvement. These are not “settlement grade” LMPs. ERCOT is limited in its capacity to handle all MP questions based on the quality and complexity of the data.

• Use: Should be used for trends and to assist the EDS team to improve the accuracy of the input data, but not for financial analysis.

• Actions: ERCOT is continuing to resolve a number of problems known to impact the SCED results: Working with QSE’s to resolve a number of SCED data input issues; similar exercise is in progress with TSP’s to improve the quality of the tele-metered data feeding the State Estimator.

Six month posting of LMPs initiated on February 15


Key Risks as 5 March 2008

These are the key, key risk

M

H

M

M

Likelihood

H

M

H

M

Impact on Go Live

ERCOT working with vendor to rectify defects/outstanding. issues Single entry model must be in place by July to verify go-live model

Single entry model could be lateNMMS

Prioritizing build out of environments and applications based on EDS schedule

Planning active / standby EDS environment (Austin / Taylor) to eliminate short-term ITest need

Building test environment in Blue Building server room Removing all non-essential equipment from Austin data center Compressing Nodal deployment footprint as much as possible

Working with 3 main vendors on multiple iterations of development and validation of CIM import/export

Investigating contingency plan to preserve April 1st start of DAM / RUC EDS trials

Building interfaces in advance of applications delivery based on available information expecting re-factoring

Testing application and integration in available, sub-optimal hardware / software environments

Mitigation

Limits speed and scale of environment build outs, impacts development projects, integration and EDS trials

Infrastructure constraints

(power, space, cooling)

Uncertainty over ability to integrate CIM models may impact data and system quality and testing

CIM

(Common Information

Model)

Lack of source systems and infrastructure limits ability to validate integration points, impacts re- factoring cycle times and costs

EIP (Enterprise Integration

Project)

DescriptionRisk


IBM Review - Dec 07 – Jan 08 - Some issues involve quality processes

Issue Nodal Team Comments

Resource Planning and Deployment. Attrition. More engagement needed from ERCOT.

Effort / Engagement will increase. Training necessarily follows delivery of software. Few key losses since October. We are relying on metrics and ownership which will drive behaviors.

Risk Management. Minor risks numerous, but duplicative by project. Few High program risks.

Relatively known risks. Risk log is comprehensive and quite exhaustive; Focus now is on the 5 or 6 top key risks rather than individual project perspectives.

Quality Reviews. Documentation lagging due to priority given to delivery.

Protocols covered. Priority given to fixing defects and trials.

Documentation of traceability is lagging behind delivery, we acknowledge documentation needs improvement

Timeline. Lagging measures in quality, need more Unit Testing visibility (before Pre FAT visibility.

High visibility given instead to Pre FAT at vendor site, to FAT at ERCOT sites, and to EDS. We view “Unit Testing” to be less reliable, and/or reporting delayed by most vendors.

PMO resources low (due to shifting resources to more critical projects)

Moved experienced personnel to other areas to fill needs; program leadership looks to optimize staffing in right places. Now adding staff to PMO and to Delivery Assurance Group (DAG)


Quality - Summary

• Successes– NMMS has fixed a number of high severity defects (2 fixed of 3)– CRR has reduced the overall number of defects; currently only 3 Severity 2

defects exist (No severity 1 defects currently)• Areas of Concern

– Defect resolution time– High percentage of EMS defects not detected till EDS

Mitigation Measures– Passive data center site with live data feed will allow for earlier defect detection

• Defect Classification

Quality has experienced recent successes but concerns remain

Severity 1 – Data Loss/Critical ErrorSeverity 2 – Loss of Functionality w/o workaroundSeverity 3 – Loss of functionality with workaroundSeverity 4 – Partial Loss of a feature setSeverity 5 – Cosmetic/Documentation ErrorSeverity 6 - Enhancement


Quality details: Nodal Test Results per Project

• Since December 2007, the Program undertook a significant documentation and tracking process.

• Significant visibility into quality status of each project:

• December 2007: 5 projects tracked

• March 2008: 14 projects tracked (others are not in the testing phase)

• Failure rates within acceptable limits, especially with Severity* 1 and 2 defects.

* Note: Severity 1 and 2 defects are the most significant of the 4 types.

December 17, 2007

March 4, 2008


Quality Details: Nodal FAT Active Defects by Severity by Project

(Note - Does not include Closed and Deferred)

Few severity 1 or 2 defects in the Market Management System (MMS)

No severity 1 defects in Energy Management System (EMS)

Commercial System Registration (COMS) is still early in development. In Pre FAT

The Network Model Management System (NMMS) has a few Sev 1 defects

Overall, active defects are being tracked by project and by classification.

The relative low number of Sev 1 and 2 defects is somewhat encouraging.


Quality Details: Nodal Active Defects by Severity Trend

Sev 3

Sev 4

Sev 1

Sev 2

• Severity 3 defect trend is rising due to focus on fixing higher priority severity 1 and 2 defects• Severity 2 trend is remaining steady which indicates “defect open rate” is only slightly higher than

defect closure rate


Quality Details: Average Days to Fix Defect by Vendor

The larger project vendors are doing fairly well in their turn around time for defect fixing. There are known and understood reasons for the other vendors.

What is not shown here is the turnaround by defect classification.

Overall, the time to fix defect in various categories gives us a good indication on:

how long the product will be in testing

what the next release will look like

how long will it take to get to market trials.

a good indication of ability to meet the 168 hour test.


Quality Details: Nodal Defects Reopened % (After Fixing by Vendor)

Indicates what percentage of fixes delivered by a vendor retested unsuccessfully.

ABB and Areva are fixing their products with a good low reopen rate.

Good indication that the Energy and the Market Management System are conducting good quality reviews at the vendor site prior to releases being given back to ERCOT and the market.

This metric is being tracked to determine trends.


CRREIPEMSIRTMMSMMS MM

Defect Summary Graph

Severity

1 D

ata

Loss

/Crit

ical

Err

or

2 Lo

ss o

f fun

ctio

nalit

y w

/o w

o..

3 Lo

ss o

f fun

ctio

nalit

y w

ith w

..

4 P

artia

l los

s of

a fe

atur

e se

..

5 C

osm

etic

/Doc

umen

tatio

n er

ror

6 E

nhan

cem

ent

Num

ber

of D

efec

ts

35

30

25

20

15

10

5

0

5

20

32

12

31

Quality Details: Defects found in the Early Delivery System

Severity 1 – Data Loss/Critical ErrorSeverity 2 – Loss of Functionality w/o workaroundSeverity 3 – Loss of functionality with workaroundSeverity 4 – Partial Loss of a feature setSeverity 5 – Cosmetic/Documentation ErrorSeverity 6 - Enhancement

We released EMS 2 through 3 too early, resulting in defects showing up in the Early Delivery System

No severity 1 defects in Energy Management System (EMS)

Commercial System Registration (COMS) is still early in development. In Pre FAT

The Network Model Management System (NMMS) has a few Sev 1 defects

Overall, active defects are being tracked by project and by classification.

The relative low number of Sev 1 and 2 defects is somewhat encouraging.

Severity

1

Severity

2

Severity

3

Severity

4

Severity

5

Severity

6


Schedule: “Health Checks”

Still targeting a 12/1/08 Go-Live date; critical path runs through EDS activities

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

EDS 1

EDS 2

EDS 3

EDS 4

Today

MP Trials & NM Verification

SCED & LMP Posting for 6 months

ICCP / P2P

SE Tuning

E1 R2

E2 R3E2 R4

E3 R5

E3 R6 LFC Testing

E3 R7 CRR Testing

E4 R9

CMM, DAM & RT Settlements, Verifiable Costs & Disputes

DAM, SASM & RUC execution, DAM Stability Test, EECP, DST, Compliance

E4 R8

Performance

Health Checks – measure of schedule

viability

Dictated by PRR 727: Market Readiness

Certification

168 Hr TestEDS 4 Go Live Sequence

CIMIntegration Defect

Go-Live (Go/No-Go)

Sept Oct Nov

TAC Meeting: 10/2

ERCOT BoD: 10/21

First Market Notice: 10/31 Second Market Notice: 11/21168 Hr Test Start

32 Days to complete Test3

2

1

Business Processes


Schedule: This is the metric status for each of the Early Delivery Systems

EDS activities are monitored and measured by 59 metrics

(xx) – Number of metrics associated with a particular Release

EDS1 – Complete

EDS2 – Overall Red resulting from ongoing issues with State Estimator

EDS3 – Yellow resulting from LMP performance and delays in LFC / ACE testing

EDS4 – Mostly unmeasured at this point, but inheriting Red status from recently moved Alarms Processing metric. Commercial Ops metrics being measured are Green

Overall Readiness – Yellow due to open issues on telemetry, modeling, and CIM

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

EDS 1

EDS 2

EDS 3

EDS 4

Today

MP Trials & NM Verification

SCED & LMP Posting for 6 months

ICCP / P2P

SE Tuning

E1 R2

E2 R3

E2 R4

E3 R5

E3 R6 LFC Testing

E3 R7 CRR Testing

E4 R9

CMM, DAM & RT Settlements, Verifiable Costs & Disputes

DAM, SASM & RUC execution, DAM Stability Test, EECP, DST, Compliance

E4 R8

168 Hr Test Go Live Sequence

(1 EDS, 2 MP)

(3 EDS)

(5 EDS, 1 MP)

(2 EDS, 1 MP)

(3 MP)

(2 EDS, 2 MP)

(12 EDS, 2 MP)

(5 EDS, 3 MP)

Overall Readiness: (15 metrics)


Outage Scheduler - Need for periodic duel entry of outages

• ERCOT will develop scripts to alleviate duel entry during most periods:– Between minor tests– Between the 168 hr. test and go-live

• There will be times, especially during the 168 hour test where zonal and nodal systems will require MPs to enter data into two systems

• Solid Bars below indicate duel entry requirement, estimated to be a total of 45 days

DecemberNovemberOctoberSeptemberAugust

Dual Entry to Support 168 Hour Test

Dual Entry just prior to Go – Live.

Minor tests

* (hashed bar if 168 hour test extends to the full month


Cost: Element Summary

EAC Revised Budget and Forecast remain constant; spending in January expected to be lower and should reverse much of the overrun through Dec.

Revised Var. (o)/u Revised Var. (o)/uCost Element Budget Actual Budget Budget Forecast Budget

Internal Labor $ 25.0 $ 25.3 $ (0.3) $ 47.9 $ 47.9 $ -

External Labor 56.6 59.8 (3.2) 94.9 94.9 -

Vendor 42.7 44.0 (1.3) 76.8 78.8 (2.0)

Hardware / Software 55.1 56.7 (1.6) 59.2 59.4 (0.2)

Taxes and Other 9.4 9.6 (0.2) 15.1 15.1 -

Subtotal $ 188.8 $ 195.4 $ (6.6) $ 293.9 $ 296.1 $ (2.2)

Contingency - - - 15.0 12.8 2.2

Total Nodal Program $ 188.8 $ 195.4 $ (6.6) $ 308.9 $ 308.9 $ -

Finance Charge 6.6 6.0 0.6 10.6 10.6 -

Total $ 195.4 $ 201.4 $ (6.0) $ 319.5 $ 319.5 $ -

2008 therefore bringing actual spending back in line with the Revised Budget NOTE: Overspending of the Revised Budget through December 2007 will be offset in January

Nodal ProgramCost Element Summary

($ Millions)

Through December 2007 Estimate at Complete (EAC)

January actuals are still unavailable as of 5 March

Date post:	12-Jan-2016
Category:	Documents
Upload:	daniela-banks
View:	218 times
Download:	0 times

March 6, 2008 TAC Texas Nodal Market Implementation Program Update Jerry Sullivan.

Documents