EPRI Procurement Forum Agenda August 23-25, 2011 ~ Minneapolis, MN Page 1 of 3 Tuesday, August 23, 2011 TIME TOPIC 7:00 a.m. Breakfast– Universe 8:00 a.m. Opening Remarks / Introductions – Planets 8:30 a.m. Information Notice 2011-01 - EPRI & NRC - Open Discussion 9:20 a.m. Break 9:35 a.m. Current Events and Concerns in your Organization - Open Discussion - Note to all attendees : Please bring examples of improvements or changes with positive impact, things your organization does well, and areas that challenge you. 10:50 a.m. Break 11:00 a.m. Back to Basics – Selecting Critical Characteristics - Marc Tannenbaum, EPRI - Case Study (Open Discussion) 12:00 p.m. Lunch – Universe 1:00 p.m. Cyber Security - Matt Bowman, ATC Nuclear Acceptance of Commercial Grade Software Guidance Update - Marc Tannenbaum, EPRI Current Regulatory Issues - JUTG Steering Committee 2:25 p.m. Break 2:40 p.m. Managing Procurement Engineering Work - Murray Kraemer, Bruce Power - Paul Saksvig 3:35 p.m. Break 3:50 p.m. Outsourcing Equivalency Evaluations at USA - Howard LeCompte, Detroit Edison Co. 4:15 p.m. Topics for Next Meeting 4:30 p.m. Procurement Engineering On-Line - Greg Keller, Nuclear Logistics, Incorporated 5:00 p.m. Adjourn 5:30 –7:30 p.m. EPRI Networking Reception & Supplier Information Exchange – Stars
Transcript
EPRI Procurement Forum Agenda August 23-25, 2011 ~ Minneapolis, MN
8:30 a.m. Information Notice 2011-01 - EPRI & NRC - Open Discussion
9:20 a.m. Break
9:35 a.m. Current Events and Concerns in your Organization - Open Discussion - Note to all attendees: Please bring examples of improvements or changes with positive
impact, things your organization does well, and areas that challenge you.
10:50 a.m. Break
11:00 a.m. Back to Basics – Selecting Critical Characteristics - Marc Tannenbaum, EPRI - Case Study (Open Discussion)
12:00 p.m. Lunch – Universe
1:00 p.m. Cyber Security - Matt Bowman, ATC Nuclear
Acceptance of Commercial Grade Software Guidance Update - Marc Tannenbaum, EPRI
Current Regulatory Issues - JUTG Steering Committee
2:25 p.m. Break
2:40 p.m. Managing Procurement Engineering Work - Murray Kraemer, Bruce Power - Paul Saksvig
3:35 p.m. Break
3:50 p.m. Outsourcing Equivalency Evaluations at USA - Howard LeCompte, Detroit Edison Co.
3:25 p.m. Nuclear Procurement Issues Committee (NUPIC) Update - Fawaz Jabali, Arizona Public Service
Regulatory Concerns - Open Discussion
4:35 p.m. Topic, Presenters & Locations for Future Meetings
5:00 p.m. Adjourn
EPRI Procurement Forum Agenda August 23-25, 2011 ~ Minneapolis, MN
Page 3 of 3
Thursday, August 25, 2011
TIME TOPIC
7:00 a.m. Breakfast– Universe
8:00 a.m. Identify Topics, Presenters & Locations for Future Meeting
8:25 a.m. New Supplier Entrance - Fawaz Jabali, Arizona Public Service - Cesar Mejia, ValvTechnologies - Rick Easterling, Kinectrics, Inc.
9:40 a.m. Break (Luggage/Checkout)
10:05 a.m. Ask the Regulator - Paul Prescott and Richard McIntyre, NRC - Open Discussion – Regulatory Concerns
10:50 a.m. Break (Luggage/Checkout)
11:00 a.m. Identify Topics, Presenters & Locations for Future Meeting
11:25 a.m. Adjourn
JUTG Procurement Forum Attendees ~ August 23‐25, 2011 ~ Minneapolis, MN
Full Name Company Email Address Work PhoneDave Bishop South Carolina Electric & Gas Co. [email protected] 803‐345‐4658Scott Borland Versatile Measuring Instruments [email protected] 800‐742‐1413Matthew Bowman ATC Nuclear [email protected] 865‐218‐8242Jose Camacho Cameron Measurement Systems jose.camacho@c‐a‐m.com 708‐717‐1572Milton Concepcion U.S. Nuclear Regulatory Commission [email protected] 301‐251‐7457Mark Coren Duke Energy Corp. mark.coren@duke‐energy.com 980‐875‐5887Christopher Crawford Babcock & Wilcox Co. [email protected] 865‐241‐9213Roy Croft Southern Nuclear Operating Co. [email protected] 205‐992‐7417Dan Dancer Xcel Energy Services, Inc. [email protected] 651‐388‐1121 x4492Scott Davis Constellation Energy [email protected] 410‐495‐2629Sam Davis Entergy Services, Inc. [email protected] 914‐272‐3307Spencer Daw Bechtel Corp. [email protected] 509‐371‐5849Rick Easterling Kinectrics, Inc. [email protected] 804‐550‐3109Shannon Echols Edgen Murray Corp. [email protected] 803‐554‐6108Chris Faraglia PKMJ Technical Services, Inc. [email protected] 412‐865‐3040Greg Galletti U.S. Nuclear Regulatory Commission [email protected] 301‐415‐1831P. Gallo AP Services, Inc. [email protected] 815‐529‐1001James Geer Cameron International jim.geer@c‐a‐m.com 562‐321‐9162George Gharabeigie Southern California Edison Co. [email protected] 949‐368‐3717Gary Gilmartin Gilmartin Engineering Works [email protected] 865‐360‐8689Jim Grant Duke Energy Corp. jim.grant@duke‐energy.com 803‐701‐3683Victor Hall U.S. Nuclear Regulatory Commission [email protected] 301‐415‐2915Brad Hemphill AP Services, Inc. [email protected] 724‐295‐6200Brian Hendel Energy Northwest bjhendel@energy‐northwest.com 509‐377‐4573Greg Hott ATC Nuclear [email protected] 865‐406‐1219Craig Irish Nuclear Logistics, Inc. [email protected] 978‐250‐1684Fawaz Jabali Arizona Public Service Co. [email protected] 623‐393‐6712Donald Jones FirstEnergy Nuclear Operating Co. [email protected] 419‐321‐8538Greg Keller Nuclear Logistics, Inc. [email protected] 702‐869‐6300Laura Knight ATC Nuclear [email protected] 423‐315‐1331Ralph Krause Nebraska Public Power District [email protected] 402‐825‐5662
Page 1 of 3 Note: Contact information omitted by request.
JUTG Procurement Forum Attendees ~ August 23‐25, 2011 ~ Minneapolis, MN
Full Name Company Email Address Work PhoneAlbert Lafleur Florida Power & Light Co. [email protected] 561‐845‐3343Robert Lane ATC Nuclear [email protected] 865‐964‐9864Howard LeCompte III Detroit Edison Co. [email protected] 734‐586‐5159Seth McClintock Omaha Public Power District [email protected] 402‐533‐6531Richard McIntyre U.S. Nuclear Regulatory Commission [email protected] 301‐415‐3215Beth McRimmon Electric Power Research Institute (EPRI) [email protected] 704‐595‐2536Cesar Mejia ValvTechnologies [email protected] 860‐961‐0823Matt Miles Edgen Murray Corp. [email protected] 704‐222‐3993Ujjal Mondal CANDU Owners Group [email protected] 416‐595‐1888 x116Michael Murdock Duke Energy Corp. MichaelW.Murdock@duke‐energy.com 980‐875‐4237Lisa Nolin Duke Energy Carolinas lisa.nolin@duke‐energy.com 980‐875‐5579Tyler Owen Lincoln Structural Solutions [email protected] 402‐290‐1481Bhavesh Patel Progress Energy, Inc. [email protected] 919‐362‐2566Cong Pham STP Nuclear Operating Company [email protected] 361‐972‐7996Daniel Philipps FirstEnergy Nuclear Operating Co. [email protected] 440‐604‐9796Dave Pirro Westinghouse Electric Co. [email protected] 724‐875‐0289Jeff Polacek NextEra Energy Point Beach, LLC [email protected] 920‐755‐6315Paul Prescott U.S. Nuclear Regulatory Commission [email protected] 301‐415‐3026Daryl Prisby Exelon Corporation [email protected] 815‐406‐3502Jerald Rogers Rolls‐Royce Corporation rogersj@ds‐s.com 256‐705‐2152Chuck Rogers Cameron International chuck.rogers@c‐a‐m.com 281‐582‐9507Paul Saksvig Dominion Generation [email protected] 804‐273‐2689Bob Salsky Cameron International bob.salsky@c‐a‐m.com 281‐582‐9500Russell Schaefer Dominion Energy Kewaunee, Inc. DEKI [email protected] 920‐388‐8725Arun Sharma Atomic Energy of Canada, Ltd. [email protected] 613‐584‐8811 x46359Jennifer Snead Duke Energy Carolinas jennifer.snead@duke‐energy.com 864‐885‐4113Garry Snyder Lincoln Structural Solutions [email protected] 678‐588‐8895Stan Stack Nutherm International, Inc. [email protected] 618‐244‐6000 x3011Keith Stall Cameron International keith.stall@c‐a‐m.com 361‐815‐9044Darlene Stimart Xcel Energy Services, Inc. [email protected] 612‐330‐6522Marc Tannenbaum Electric Power Research Institute (EPRI) [email protected] 704‐595‐2609
Page 2 of 3 Note: Contact information omitted by request.
JUTG Procurement Forum Attendees ~ August 23‐25, 2011 ~ Minneapolis, MN
Full Name Company Email Address Work PhonePaul Tobin PKMJ Technical Services, Inc. [email protected] 412‐865‐3040 x102Ararat Torosyan Enertech [email protected] 714‐528‐2301 x102Bill Van Wormer Westinghouse Electric Co. [email protected] 412‐716‐9753William Ware Southern Nuclear Operating Co. [email protected] 334‐814‐4566Stephen Werlick Scientech, Curtiss‐Wright Flow Control [email protected] 519‐938‐8084Robert Wieser Arizona Public Service Co. [email protected] 623‐393‐5715Gary Wiley Progress Energy, Inc. [email protected] 919‐362‐2955Edward Wynne Fluor Enterprises, Inc. [email protected] 864‐644‐8054Sam Yousif Dynamic Solutions USA, Inc. [email protected] 914‐456‐0227
Page 3 of 3 Note: Contact information omitted by request.
• Respect Participants and Presenters• Respect Participants and Presenters– Please return from breaks on time– Please refrain from side conversations during
presentations
• ConfidentialityAll i t ll i t t– All sessions are open to all registrants
February 21-23 2012February 21-23, 2012•San Antonio, Texas
Agenda items will be establishedduring this meeting
•• Bring up ideas and concerns during theBring up ideas and concerns during the•• Bring up ideas and concerns during the Bring up ideas and concerns during the meetingmeeting
•• Use the Open Discussion templateUse the Open Discussion template•• “Its your meeting”“Its your meeting”•• The Steering Committee welcomes The Steering Committee welcomes
Lack of Engineering Justification During the CGD Process
Take Away
1 CG1. CGD evaluations must be revised by engineering prior to changing dedication acceptance activities
2 Use equivalency evaluation and modification processes to2. Use equivalency evaluation and modification processes to address design changes, dedication is an acceptanceprocess
Concern/Clarification
Design changes need to be handled outside of the dedication process Dedication is an acceptance process that requiresprocess. . Dedication is an acceptance process that requires engineering involvement.
Interpret as: “Vendor made changes to the technical evaluation
1 A i l d t f li t l1. A commercial grade survey must focus on supplier controls related to the critical characteristics identified in the dedication technical evaluation
2. Surveys are necessary even if the supplier has an ISO 9000 Quality Assurance program
3. Limited scope audits are only appropriate when the supplier has a QA program that complies with 10CFR50, Appendix B
4 Pay attention to terminology4. Pay attention to terminology
• “Audit” refers to a look at all 18 Criteria of 10CFR50, Appendix B
S1. Sampling must provide reasonable assurance that items successfully tested or inspected conform to requirements.
2 It is important to document the basis for selection of2. It is important to document the basis for selection of sampling plans, particularly when heat traceability or lot/batch control can not be established.
GL 89-02 places restrictions on the use of both Methods 2 and 4 but onlyGL 89 02 places restrictions on the use of both Methods 2 and 4 but only restricts Method 4 from being used as the sole method of acceptance
Method 2 is restricted to situations in which:
(1) Th li h d t d ff ti t l d if li bl(1) The supplier has documented effective controls, and if applicable,
(2) When distributors are involved; the distributor must also be surveyed
Take AwayTake Away
A single survey cannot be used as a basis for accepting all subsequent purchases from the supplier surveyed unless:p pp y
a)Controls verified during the survey are related to critical characteristics
b)Controls verified during the survey are documented and imposed on the purchase order
c) Some type of incoming inspection or verification of certification is
performed for each shipment of items
Flow Chart
Concern/Clarification1 In addition to safety function/significance1. In addition to safety function/significance,
credible failure modes and design characteristics (included in EPRI NP-5652), the Document Results box suggests that design margin be documented in thedesign margin be documented in the technical evaluation.
Considerations involving design margin should be addressed as part of the design process. The design process must be complete prior to using CGD to accept an item. Although design margin may be considered during the modification or design change process, it is not evaluated during the commercial grade item dedication process.
1 Ensure provisions are included in procedures for reviewing the supplier’s1. Ensure provisions are included in procedures for reviewing the supplier s dedication plan when:
a) The supplier is not familiar with end-use application, safety function(s) etcfunction(s), etc.
b) Known issues exist with the supplier’s dedication program
2. Reinforce the importance of documenting decisions made relative to using M h d 2 l d f if i d i h h fMethod 2 alone and of verifying documentation that supports the use of Method 2 at receipt inspection for every purchase
3. Reinforce the importance of documenting the use of sampling plans (e.g. h t i l l t d d h t f t id d ( hwhy a certain plan was selected and what factors were considered (such
as lot homogeneity, complexity of items, safety significance of characteristics, etc.)
EPRI F ll 2011 JUTG M tiEPRI Fall 2011 JUTG Meeting
Current Events / Concerns in your PE O i tiOrganization
What concerns you most about or within PE – are there any hot
Current Events and Concerns in your Organizations y y
issues that you are dealing with?
Have you had recent assessments (Self assessments or Quality/Oversight assessments) that looked at your PE activities – what did they find – any areas of strength or any areas for improvement?
Are you aware of Industry issues affecting PE such as Software Dedication and NUPIC Findings relating to vendor dedication practices?practices?
How is your interface with your NUPIC representatives and are you notified of significant findings related to PE activities?y g g
How often do you provide the proper safety function of the item being procured on a PO?
•Understand the process for identifying and l ti iti l h t i ti f tselecting critical characteristics for acceptance
•Understand the NRC definition of critical characteristicscharacteristics
• Identify different types of critical characteristics
U d t d th f id tif i iti l•Understand the process for identifying critical characteristics for seismically sensitive items
•Understand the relationship between critical•Understand the relationship between critical characteristics for acceptance and the item’s safety function(s)
What properties or attributes provide reasonable assurance the item received will perform its intended safety function?
10CFR21 Definition
Acceptance
Background/basis
• Per NP-5652, critical characteristics (for acceptance) are:“identifiable and measurable attributes/variables of a commercial grade item“identifiable and measurable attributes/variables of a commercial grade item, which once selected to be verified, provide reasonable assurance that the item received is the item specified.”
1988
• Per NP-6406, critical characteristics for design are:“those properties or attributes which are essential for the item’s form, fit and functional performance. Critical characteristics for design are the identifiable
1989functional performance. Critical characteristics for design are the identifiable and/or measurable attributes of a replacement item which provide assurance that the replacement item will perform its design function.”
1989
• Per 10CFR21, Rev. 2, critical characteristics are:“those important design, material, and performance characteristics of a commercial grade item that, once verified, will provide reasonable assurance th t th it ill f it i t d d f t f ti ”
that the item will perform its intended safety function.”
Definitions are consistent and compatible because:
• Intent is to select critical characteristics that provide reasonable assurance the item will perform its safetyreasonable assurance the item will perform its safety functions
• Item’s design function(s) are important in determining critical characteristics for design (CCD)
• Item’s safety function(s) are important in determining critical characteristics for acceptance (CCA)characteristics for acceptance (CCA)
• Selection of CCAs is “based upon the complexity, intended safety-related function, and performance of the CGI using engineering judgment”
Now this looks reasonable, but what about part number?
Identification Attributes
• Strictly speaking, identification characteristics such as part number are not critical characteristics as defined in 10CFR21– They are not design, material, and performance characteristics that
once verified provide reasonable assurance that the item will perform its intended safety function
• Identification characteristics are still important and are typically verified• Identification characteristics are still important, and are typically verified during initial receipt inspection– In accordance with ANSI N45.2.2, Section 5.2 (currently NQA-1,
Subpart 2.2, Section 502.2)• Verification of identification characteristics is important
– Reviewed during technical evaluation to help determine if procurement is like-for-like or alternate item
– Identifies the item and links it to procurement documents that useIdentifies the item and links it to procurement documents that use identification characteristics (such as part number) to communicate with suppliers
– Provides some assurance that a change has not occurred– Confirms identify of the item prior to proceeding with dedication
Confirms identify of the item prior to proceeding with dedication acceptance methods to verify design, material, and performance characteristics
USNRC comments regarding the use of industry guidance (NRC IP 43004)
“The identified critical characteristics that are important for the item to perform its safety function as determined in
y g ( )
the item to perform its safety function, as determined in the technical evaluation, are to be verified.”
“Not all design requirements need to be considered criticalNot all design requirements need to be considered critical characteristics; however, licensees or applicants must assure the suitability of all parts, materials, and services for their intended safety related applications ”for their intended safety-related applications.”
“Reliance on part number verification and certification d t ti l i t i i ffi i t tdocumentation alone on receipt is insufficient to ensure the quality and suitability of commercially procured products.”
• Important in order to identify the appropriate criticalImportant in order to identify the appropriate critical characteristics
• Should describe how the item functions and how the item is designed
• This may include detailed information that may not be t i d f th f t ’ bli h dascertained from the manufacturer’s published
product information or from drawings of the item
• It may be necessary to closely examine the item’sIt may be necessary to closely examine the item s moving parts, and internal mechanical/electro-mechanical sub-assemblies
• Should be identified based upon the safety function of the parent equipment and the functional mode (i ethe parent equipment and the functional mode (i.e., active or passive) of the component in that application
• Seismic safety functions are those design functions of th it th t f th t tthe item that are necessary for the component to perform its safety functions during and after a seismic event
– Maintain calibration– Operate without spurious action
Seismic failure mechanisms
• Failures of each item should be postulated based upon its application and the accident conditions toupon its application and the accident conditions to which it might be exposed
• Defined as “the manner by which an item may fail, d di th it ’ bilit t f th t/degrading the item’s ability to perform the component/ system function under evaluation”
• Typical seismic failure mechanisms may include:yp y
– Structural failure– Chatter of contacts (inadvertent change of state
– Loss of calibration or setpoint– Inadvertent actuation
Critical seismic attributes
• Critical seismic attributes are defined as “those properties or attributes that are essential for the item's seismic performance”
• They should be selected to address both the seismicThey should be selected to address both the seismic safety function(s) and any postulated seismic failure mechanisms
E l f iti l i i tt ib t i l d• Examples of critical seismic attributes include:
• Seismic critical characteristics are “those important design, material, and performance characteristics of a g , , pCGI that once verified, will provide reasonable assurance that the item will perform its intended seismic safety function(s)”seismic safety function(s)
• Seismic critical characteristics that may affect the ability to perform during or after a seismic event i l dinclude:
– Mass and strength of the materials– Configuration and dimensions/tolerances of theConfiguration and dimensions/tolerances of the
• Each critical seismic attribute should have a related critical characteristic for acceptance
• Factors that can be considered in selecting related seismic critical characteristics for acceptance include:seismic critical characteristics for acceptance include:
• In some instances the related critical characteristic will• In some instances the related critical characteristic will be a direct measure of the critical seismic attribute (e.g., operating torque is a direct measure of torsion
d t i f )and return spring forces).
• In many cases it will be an indirect measure (may• In many cases it will be an indirect measure (may require verification in combination with other related critical characteristics to establish a clear link to the qualification basis)qualification basis)
Options for obtaining or recovering acceptance criteria
• Measurements taken from hardwareOption 1 Measure the tested item in conjunction with seismic testing– Option 1 - Measure the tested item in conjunction with seismic testing
– Option 2 - Measure a qualified item procured concurrently with seismic test specimen
– Option 3 - Measure an identical item procured subsequent to seismicOption 3 Measure an identical item procured subsequent to seismic testing but traceable through the seismic test report
• ANSI N45.2.13 allows the licensee various means by which selected critical characteristics may be verified:which selected critical characteristics may be verified:
– Special testing/inspection of the hardware received
– Performance-based evaluation of the manufacturer’s quality controls
– Source verification at the manufacturing facility
R li hi t i l f f th it d– Reliance on historical performance of the item and its supplier
• Special tests/inspections are the most common usedSpecial tests/inspections are the most common used method for dedicating CGIs
• The evaluations focus on those special functional t t d/ i ti th t ld d t l if
tests and/or inspections that would adequately verify the critical characteristics identified
Adequate acceptance plans
• If the seismic CGI acceptance plan is adequate, thenIf the seismic CGI acceptance plan is adequate, then it should supplement the overall CGI acceptance/dedication plan for the seismically sensitive itemsensitive item
• There may be other critical characteristics requiring y q gverification (in addition to those seismic critical characteristics) to reasonably assure that the item will perform other safety-related functions under normalperform other safety related functions under normal and accident conditions
• If the seismic CGI acceptance plan is not adequate, then the following actions should be considered:following actions should be considered:
– Procure an alternate item
– Procure the item as a basic component from a nuclear psupplier with a 10CFR50, Appendix B QA program.• The supplier performs qualification activities under their nuclear QA
program. In this case the supplier may be the original equipment p g pp y g q pmanufacturer (OEM) or a third-party supplier. The user should consider including the seismic critical characteristics in the performance-based supplier audit (PBSA) plan.)
V ifi ti f CCA ( t ) fi th t th• Verification of CCAs (acceptance) confirms that the item received conforms to design requirements
• Acceptance may also add assurance that the designAcceptance may also add assurance that the design of the item has not changed over time (i.e., acceptance assures that items received now are identical to those previously qualified)identical to those previously qualified)
Therefore, acceptance activities provide assurance that an item procured today remains “qualified”
Guideline for the Utilization of Commercial Grade Computer Programs in Nuclear Safety-Related ApplicationsApplications
Marc H. TannenbaumP j t M EPRIProject Manager, EPRI
Joint Utility Task Group MeetingAugust 23, 2011
Historical Perspectives on Acceptance of Commercial Grade Computer Programsp g
• Computer programs (not embedded in plant equipment) could be perceived as not meeting the definition of acould be perceived as not meeting the definition of a “commercial grade item” in 10CFR21
C t t i t• Computer programs use was not as prominent
– Regulatory documents mention software but do not address it in detail
• Acceptance performed by subject matter experts
• Licensees implement software quality assurance (SQA) programs under the umbrella of their 10CFR50, Appendix
Figure 3 – Overlaying dedication techniques to accept software
Application of Acceptance Methods
Inspection Attribute/Critical Characteristics
Acceptance Criteria Possible Method(s) of Acceptance
Software Revision Number Conforms to the procurement document Standard Receipt Inspection
Update (configuration) Control Current configuration remains suitable for the application
Method 2 (CG Survey)
Platform compatibility (operating system, etc.)
Computer program is compatible with the current operating system
Method 1 (Testing)
Hardware compatibility Computer program is compatible with the current hardware
Method 1 (Testing)
Built-in-Quality Appropriate in-process tests and inspections are performed
Method 2 (CG Survey)
Q lit f D i d I l t ti D i t l f d i M th d 2 (CG S )Quality of Design and Implementation Design controls are performed in accordance with SQA
Method 2 (CG Survey)
Functions/Applications Outputs are consistent and accurate for various applications
Method 1 (Testing), Method 2 (CG Survey)
Range (input variables, limits of Outputs are consistent and accurate Method 1 (Testing), application, etc.) over a range of inputs and applications Method 2 (CG Survey)
Accuracy Outputs are mathematically accurate Method 1 (Testing), Method 2 (CG Survey)
Consistency Repeatability Outputs are consistent and accurate Method 1 (Testing)
Consistency Repeatability Outputs are consistent and accurate over numerous times the computer program is used
Method 1 (Testing), Method 2 (CG Survey)
What should we be doing internally?
I iti t di l b t P t• Initiate dialog between Procurement Engineering, Information Technology, and your Vendor Audit organizationsVendor Audit organizations
–Review internal software acceptance methods
•Classification processClassification process
•Ensure safety function and failure modes are consideredare considered
–Establish a point of contact for requesting SME support for supplier assessments
Managing PE Work -Managing PE Work EPRI JUTG Presentation
Problem StatementProblem Statement
Currently, Procurement Engineering has processes in place to manage work thatprocesses in-place to manage work that are labour intensive, contain multiple data sources and does not provide a clear linesources and does not provide a clear line of sight on overall priorities within our core businessbusiness.
PassPort RoutingPassPort Routing
• Time Consuming• Time Consuming
• Difficult to Manage Resource Loading
• Inability to
Track HistoryTrack History
Cognos ReportingCognos Reporting
• Cognos is a reporting tool that queries data from PassPort• Cognos is a reporting tool that queries data from PassPort
• Bruce Power I.T owns and updates Cognos reports
LimitationsC i l t• Cognos is slow to run
• Requires months to update or to generate new reports
• Report data is limited in the number of rows that can be pulled
Ti i k i d h d l k• Time consuming to track metrics and schedule work
• Daily manual effort needed to run reports
• Only available for viewing in select formats
Procurement EngineeringProcurement EngineeringWork Management System
Mission StatementMission Statement
Our mission is to protect ProcurementOur mission is to protect Procurement Engineering’s core business of setting up CID’s for purchase in support of safe andCID s for purchase in support of safe and reliable plant operations at Bruce A and Bruce BBruce B.
PE Work Programs and PrioritiesPE Work Programs and Priorities
Goals:- Provide Clear Work Visibility And Improve Work Management - Reduce/Eliminate Wasted Resources And Identify Risks Early
• Required Date, TCD and Scope Changes allow prioritization and optimizationprioritization and optimization.
• Automation reduces required resources for• Automation reduces required resources for delegation, tracking and reporting.
• Consolidating several work sources into one to simplify and organize demand reports.simplify and organize demand reports.
Automated SchedulingAutomated Scheduling
Managing PE Work
Reporting Made EasyReporting Made Easy
• Dynamic reports allow Managers and Engineers to present specific/detailedEngineers to present specific/detailed information as required.
R t b i d i lti l t• Reports can be viewed in multiple ways to meet the needs.
A– Access
– Excel
Word document– Word document
Automated ReportingAutomated Reporting
Resource LoadingResource Loading
Risk AnalysisRisk Analysis
• System identifies changes not currently recorded by PassPort such as scheduledrecorded by PassPort such as scheduled dates and tracks the issues for Procurement to analyze.y
• Upcoming workload identification toUpcoming workload identification to provide early indicators of work load increases and decreases.
Optimize ResourcesOptimize Resources
• The system updates itself at a set time each day from data in PassPorteach day from data in PassPort.
• Work done in PassPort will be updated i t th tinto the system.
• No external changes to the company or any outside procedure required.
• Less effort, less noise, higher accuracy., , g y
CloseClose
Questions or Comments
Managing Procurement Engineering WorkEngineering Work
Paul Saksvig
Supervisor, Procurement Engineering
i iDominion
1
PurposePurpose
• How Dominion Procurement Engineers fit• How Dominion Procurement Engineers fit into the T-schedule.
PE i i i i f O li d• PE prioritizing of Outage, on-line and non-scheduled items
Material Discrepancies and Corrective Actions to SCM typically wind up withActions to SCM typically wind up with PE
•On average 100-150 CA/yr for VA alone
•Issues during quote process and upon•Issues during quote process and upon receipt – (mostly obsolescence or vendor supplying different part) take time to
4
supplying different part) take time to resolve
Dominion PE ResourcesDominion PE Resources
• Dominion Procurement Engineering• Dominion Procurement Engineering– 8 PEs and Supervisor at Millstone
5 PE d S i K– 5 PEs and Supervisor at Kewaunee
– 9 PEs and Supervisor at Surry/North Anna
• All PEs use common platform and procedures for job functions
5
Dominion PE ResourcesDominion PE Resources
• Each Site Manages day to day PE issues• Each Site Manages day to day PE issues
• Inter-site support for emergent needs
• Working to expand fleet sharing of resources
6
VA PE Support
Procurement Engineers in VA gsupport two sites so it becomes a challenge to track and meet thechallenge to track and meet the scheduling requirements of two
kwork groups.
7
Software PlatformsSoftware Platforms
• SAP for stock system and P O processing• SAP for stock system and P.O. processing– PE products are produced and stored in SAP
and linked to the material stock numberand linked to the material stock number
– Material discrepancies – notifications are generated in SAPgenerated in SAP
– Buyer’s communication to PE on parts issues are generated and stored in SAPare generated and stored in SAP
8
Software PlatformsSoftware Platforms
• Maximo for work control• Maximo for work control– Restraints created in Maximo for items that
require PE action Equivalencies QArequire PE action – Equivalencies, QA mismatch, other
– Restraints may be auto/manual created– Restraints may be auto/manual created
– PE can create/complete restraints
9
P i i i i W kPrioritizing Work• Until last year prioritizing work in PEs buckets was difficult
•Typically - the group that requested the loudest that got higher priority
• No consolidated list which made tracking work by the PE supervisors
10
very time consuming
Backlog Database Developed
•Improved tracking database has helped considerably with managing p y g gwork load
All k f i i i•All PE tasks from requisition processing to receipt notifications associated with an item or on theassociated with an item or on the list
11
Backlog Database Developed
•Groups common issues allowing PEs to know what tasks need to be completed before the issue is resolved.
•Puts in graphical form upcoming workwork
12
Priority Ranking
1 -Work Starting in next 16 weeks
2 W k S i 16 k2- Work Starting > 16 weeks
3- Critical Stock Reorders – 0 on hand
4- Critical Stock Reorders – some on handhand
5- Non-Critical/Non-Scheduled stock d
13
orders
T-Week Scheduling
Request for paperwork (IEE) completion is being pushed further outis being pushed further out
T 6 IEE must be approved or stationT-6 – IEE must be approved or station willing to carry at risk (not quite thereyet)yet)
14
Use of the ListUse of the List
• Database updated weekly Monday morning for use by PEs LIST
• Proving an effective tool for PEs to manage their work
• Visual tool to look ahead to upcoming IEEs CHARTCHART
15
ChallengesChallenges
The database has helped a lot but:
•PEs tend to focus on Priority 1 or 2 items and ignore low hanging fruit in other groups
•Daily issues tend to keep focus off of•Daily issues tend to keep focus off of scheduled work
16
•Other product due dates direct resources
ChallengesChallenges
Site SCM Coordinators/PE Sup Interface
•Have gotten better at screening non-priority issues and pushing back on items that are not really emergent
•PE Supervision needs to be actively•PE Supervision needs to be actively engaged in screening requests
17
Other Actions to Manage WorkOther Actions to Manage Work
•Procedure enhancement to minimize errors and standardize processesand standardize processes
•Leveraging vendors to provide the evaluation with the material
•Process improvements to better define “non•Process improvements to better define non-plant” work and non-critical changes
18
•IEE Short Form
ConclusionConclusion
• Procurement Eng support of the T-Schedule has improved significantly p g y
• Database and Chart tools are improving managing the work loadmanaging the work load
• Too many other issues still pull on PE resources and are not easily trackedand are not easily tracked
19
QUESTIONS?QUESTIONS?
20
Backlog Report by Priority
North Anna and Surry Power Stations
Procurement Engineering
Virginia Backlog Report
1a) Work Starting in the Next 16 WeeksStock# Description OnHandQA Ttl ResRestricted BlockedQI Man.Entry?
Requisition Backlog GOOD GOOD GOOD CAUTION! CAUTION!
(Caution: There was an increase in the last four weeks compared to the previous four weeks)
* Monthly Indicators
1
Requisition Backlog GOOD GOOD GOOD CAUTION! CAUTION!
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐(Caution: There was an increase in the last four weeks compared to the previous four weeks)
Level: NuclearLevel: NuclearGroup: PEGroup: PEFrequency: MonthlyFrequency: Monthly Back
More Detail
Open IEEsOpen IEEspp
171
155 156
700
160
180
Fleet Open IEEs
Combined Kewaunee Millstone North Anna Surry144 144
362 400
500
600
120
140
160
IEEs
Average A
ge (Days Good
Combined Kewaunee Millstone North Anna SurryCAUTION! CAUTION! CAUTION! CAUTION! GOOD
(Caution: Increase in numbers or average age)
319 326 324 313
362
200
300
400
60
80
100#
s)
0
100
0
20
40
Jan 11 Feb 11 Mar 11 Apr 11 May 11 Jun 11 Jul 11 Aug 11 Sep 11 Oct 11 Nov 11 Dec 11Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11
– Created to address common challenges associated to proactive
obsolescence solution development across the USA/STARS
(“the Alliance”)
– Leverages the strengths of engineering services (PKMJ) and
hardware supplier (CWFCN)
Engineered Obsolescence Solutions
What is EOS?g
• Purpose is to develop and implement solutions created by Obsolescence
– Engineering Solutions are performed under PKMJ’s
Appendix B program
– Hardware solutions by CWFC are provided under their
Appendix B program
Engineered Obsolescence Solutions
2010 Pilot Project Participantsg
• Ameren• Ameren • PG&E• PG&E• Arizona Public Service Co. • Detroit Edison Co. • Energy Northwest
• Arizona Public Service Co. • Detroit Edison Co. • Energy Northwest
• PPL Susquehanna• PSEG• Southern California Edison
• PPL Susquehanna• PSEG• Southern California EdisonEnergy Northwest
• Luminant• Nebraska Public Power District • Omaha Public Power District
Energy Northwest• Luminant• Nebraska Public Power District • Omaha Public Power District
• Southern California Edison • STP Nuclear Operating Co.• Wolf Creek Nuclear Operating
Co
• Southern California Edison • STP Nuclear Operating Co.• Wolf Creek Nuclear Operating
Co• Omaha Public Power District • Omaha Public Power District Co.Co.
Engineered Obsolescence Solutions
2010 Pilot Scopeg
• 8 Items were selected for solution development
– “Equity” for solution development was established across the
participating stations
– These 8 items yielded 59 site specific equivalency evaluations
– Of the 8 items, 1 was a reverse engineering solution, and will g g
move into the 2011 project for completion
Engineered Obsolescence Solutions
Challenges Resolved g
• Establishing a list of obsolete items that provided “equity” to all
t ti i l dstations involved
• Establishing access to site data systems using remote access (VPN)
• Assuring all QA requirements were aligned
• Issuing of Purchase Orders to PKMJ
– Challenge for sites to obtain funding needed for participation
• Acquiring site specific application informationq g p pp
Engineered Obsolescence Solutions
Lessons Learnedg
• Selecting list of evaluations to be pursued
• Specifying due dates for site requests
• Monthly Indicators streamlined to show better project summaries
and less details
• Engineering portion of project started late
– Evaluations started in June due to equity selections
– Late Purchase Orders resulted in shortened engineering windowg g
Engineered Obsolescence Solutions
Cost Savingsg
• Cost Savings on a per evaluation level (cont):
– 70% Cost savings, in man‐hours, is attained in the base
evaluation cost after the first evaluation is completed
– Cost savings achieved by leveraging “Economies of Scale” and
“Engineer Once, Implement Many” concepts
– In the pilot scope of this project, we resolved 2,355 obsolete
installed locations.
Engineered Obsolescence Solutions
Cost Savingsg
• Additional cost savings achieved of 6 man‐hours per evaluation by
utilizing CMIS Lite (software tool)
• Savings achieved by:
– Reducing the number of administrative errors found in
completing an equivalency evaluation
– Strict procedural compliance
– Replicationp
– Provides a communication portal between each participating
site and off‐site engineering resources to facilitate the reviewsite and off site engineering resources to facilitate the review
and approval cycle
Engineered Obsolescence Solutions
Cost Savingsg
• Cost Savings Indicator (as of 1/20/2011):
*1,223 of the remaining components are associated to a reverse engineer project to be completed in early 2011.
Engineered Obsolescence Solutions
2011 Current Statusg
• Alliance has Developed a 5 Year Plan Moving Forward with this Project
– 20112011
• Scope of items have been selected, and are being worked
• Process improvements identified in pilot year have been incorporated, p p y p ,
and are yielding positive results
• Process for leveraging OIRD solutions has been developed and
implemented
Engineered Obsolescence Solutions
2011 Project Participantsg
• Ameren• Ameren • PG&E• PG&E• Arizona Public Service Co. • Detroit Edison Co. • Energy Northwest
• Arizona Public Service Co. • Detroit Edison Co. • Energy Northwest
• PPL Susquehanna• PSEG• Southern California Edison
• PPL Susquehanna• PSEG• Southern California EdisonEnergy Northwest
• Luminant• Nebraska Public Power District* • Omaha Public Power District*
Energy Northwest• Luminant• Nebraska Public Power District* • Omaha Public Power District*
• Southern California Edison • STP Nuclear Operating Co.• Wolf Creek Nuclear Operating
Co
• Southern California Edison • STP Nuclear Operating Co.• Wolf Creek Nuclear Operating
Co• Omaha Public Power District* • Omaha Public Power District* Co.Co.
*Pending Purchase Orders*Pending Purchase Orders
Engineered Obsolescence Solutions
2011 Current Statusg
• 2011 Selection list
M f t M d l N b D i ti Sit I t dManufacturer Model Number Description Sites Impacted
Weston Aerospace 4303 Indicator, Instrument 4
United Brass Works 126S Valve 7
Leeds Northrup 708211 PH Analyzer 5
Leeds Northrup 7773 Sensor Element 6
Moore LTD 58S Pressure Relay 6y
Siemens Fire Safety DIS5B Smoke Detector 4
Fisher 4160KR PR Controller 0
R t N l 4915401 P S l 4Rosemount Nuclear 4915401 Power Supply 4
Masoneilan 3720721 AOV 2
Masoneilan 3820771 AOV 3
Masoneilan 3820721 AOV 3
Engineered Obsolescence Solutions
2011 and Beyondg
• Alliance has Developed a 5 Year Plan Moving Forward with this Project
– 20112011
• Refine process of item selection – reduce number of series challenges or
reduce total number of project items if focused on series challenges
• Continue to solve remaining challenges from pilot year (2010)
• Work with a pilot set of sites to develop a process to leverage Alliance
project into daily process and to develop backlog reduction plans
• Apply lessons learned from pilot year to a more streamlined process
• Continue to obtain VPN connections at each site
• Select a pilot set of sites to train PKMJ engineers to site process
This will reduce time and effort required for site’s involvement in the– This will reduce time and effort required for site’s involvement in the
long run
Engineered Obsolescence Solutions
Lesson Learnedg
• Start the Replacement Evaluation selection process 2nd Quarter of the
proceeding year to assure the final list is approved prior to the start of the p g y pp p
project.
• Develop Project Schedule and Milestones to assure the Project stays on track
• Bi‐weekly conference call between the site and EOS teams to assure each
organization is aligned and up to date on the progress.
• Site managements involvement to help eliminate obstacles associated with
Project completion.
• VPN access established at site to reduce impact on the engineering staff• VPN access established at site to reduce impact on the engineering staff.
• EOS Team qualified to the sites ESP program for Replacement Evaluations.
Engineered Obsolescence Solutions
Questionsg
Procurement Engineering and Supply Chain Updatepp y p
Marc H. TannenbaumP j t M EPRIProject Manager, EPRI
Joint Utility Task Group MeetingAugust 23, 2011
Topical
• Suspected counterfeit and fraudulent items
• Substandard item reporting
• Acceptance of commercial grade software for use as aAcceptance of commercial grade software for use as a basic component in safety-related applications
• Engineered item procurement and refurbishmentEngineered item procurement and refurbishment templates
• (1) Current reporting requirements only require the reporting of defects and failures to comply that could leadreporting of defects and failures to comply that could lead to a substantial safety hazard and significant events driven by equipment failures
– Basic components that are determined to be CFSls but do not constitute a substantial safety hazard or cause reportable event would not be required to be reportedepo tab e e e t ou d ot be equ ed to be epo ted
• (2) Lack of clarity whether CFSI constitutes a deviation, failure to comply, or condition adverse to quality as defined in existing rules and guidance
• (3) Current interpretation of Part 21 only applies to basic components (items that have passed receipt inspection orcomponents (items that have passed receipt inspection or completed the dedication processes)
– CFSI identified during receipt inspection and dedication gmay not be evaluated for reportability under Part 21
(4) 10 CFR 50 A di B C it i XV XVI d t• (4) 10 CFR 50, Appendix B, Criteria XV, XVI, and current guidance do not explicitly require licenses and vendors to enter CFSI occurrences identified during receipt inspection and dedication processes in their nonconformance or corrective action programs.
• (7) There is no requirement for a licensee facility or vendor to quarantine suspected CFSI materials for furthervendor to quarantine suspected CFSI materials for further analysis, regulatory/law enforcement purposes. Guidance on how suppliers of Critical Digital Assets implement the
li t l t i d i R l t G id 5 71supplier controls contained in Regulatory Guide 5.71 (Cyber Security)
• (9) The NRC does not currently perform procurement, Commercial-Grade Dedication, or Part 21 inspections at operating power plants to ensure licensees areoperating power plants to ensure licensees are adequately screening for CFSI during receipt inspection and commercial grade dedication activities.
• (10) CFSI is not specifically included in NRC inspection guidance [Response Working Group] i) NRC inspectorsguidance. [Response Working Group] i) NRC inspectors are not directed to look for the possibility of CFSI, or to look at a vendor/licensee's program for detecting and
ti CFSIpreventing CFSI
• (11) Guidance on how suppliers of Critical Digital Assets• (11) Guidance on how suppliers of Critical Digital Assets implement the supplier controls contained in Regulatory Guide 5.71
• (12) NRC inspection authority for non-safety related critical digital assets (Regulatory Guide 5.71)
• (14) There currently is no regulatory guidance or requirements for the authentication and testing ofrequirements for the authentication and testing of components necessary to identify a counterfeit or fraudulently identified item
• (15) There is no NRC guidance that specifically addresses the need for licensees or suppliers to implement programsthe need for licensees or suppliers to implement programs to identify fraudulent documentation
• (16) Current NRC requirements do not require licensees to pass down contractual requirements for supplier CFSI programs to identify and eliminate fraudulent goods
programs to identify and eliminate fraudulent goods obtained from subsuppliers.
Some gaps identified by NRC
• (19) 10 CFR 50 Appendix S, Criterion XVI does not require cause determination for conditions adverse torequire cause determination for conditions adverse to quality.
• (20) No regulatory guidance for implementing measures ( ) g y g gto prevent CFSI associated with RTNSS (Regulatory Treatment of Non-Safety Systems).
• (21) There is a lack of regulatory requirements associated• (21) There is a lack of regulatory requirements associated with the procurement of non-safety related critical infrastructure equipment.
• NRC believes changes to Part 21 and associated regulations are desirable– Many issues despite generic communications and outreach efforts
– Recent audits by Office of the Inspector General reiterated issues
• New plant construction is the appropriate time to• New plant construction is the appropriate time to consolidate guidance and clarify the NRC’s expectations
• Opportunity to clarify expectations for non-reactor facilities
“Even as the final rule was published on June 6 1977 the CommissionEven as the final rule was published on June 6, 1977, the Commission recognized that future experience and further information might warrant clarifying or other changes to the rule.” (NUREG-0302) 2
Right Time for Change (cont’d)
202020
10
15
1111
1515
1010
1414 1313
99
5
1010
33
8899
330
2007 2008 2009 2010 2011
Vendor Inspections With Part 21 Related Findings
(to date)
3
p g
Right Time for Change (cont’d)
95 - "Other" Quality Assurance
49 Part 21 Violations49 - Part 21 Violations
14 - Identification of Deviations
36 - Commercial-Grade Dedication
99 Total Part 21-related findings
Vendor Inspection Findings (2007 to 2011)
4
e do spect o d gs ( 00 to 0 )
Build on ExperienceBuild on Experience
EVALUATING AND REPORTINGRequires
RegulationChange
New Requirement
AdditionalGuidance
EndorseIndustry
Guidance1. Lack of Regulatory Guidance ✔
2. Quality Requirements in Procurement Documents ✔ ✔ ✔
3. Procurement language in definition of Basic Component for non-reactor facilities ✔ ✔
4. Clarify point of discovery ✔ ✔
5. Evaluating and Reporting Responsibility ✔ ✔5. Evaluating and Reporting Responsibility ✔ ✔
6. Deferral of Evaluation - 21.21(a) ✔ ✔ ✔
7. Use of Licensee Event Reporting (i.e. 50.72/73) ✔ ✔
8. Acceptable forms of written notification per 21.21(2) for non-reactor facilities ✔ ✔
9. 50.55e Redundancy ✔
10. Evaluation of Counterfeit Fraudulent and Suspect Item (CFSI) under Part 21 ✔
11. Clarify Deviation and delivery ✔ ✔
12. Contemporary posting requirements ✔
13. Training ✔
COMMERCIAL-GRADE DEDICATION
RequiresRegulation
ChangeNew
RequirementAdditionalGuidance
EndorseIndustry
GuidanceA. Lack of Regulatory Guidance ✔ ✔
B. Proper Place for Dedication ✔
C. Definition of Dedication ✔
D D fi iti f C i l G d ItD. Definition of Commercial-Grade Item ✔
E. Clarify that Dedication is a Safety-Related activity ✔
F. Dedication plans and the importance or safety function ✔ ✔ ✔ ✔
G. Sampling Requirements ✔ ✔
H. Use of Commercial Calibration (and Testing) Labs ✔ ✔
I. Software Dedication ✔ ✔
ADMINISTRATIVE CHANGESRequires
Regulation New AdditionalEndorseIndustry
5
ADMINISTRATIVE CHANGES Change Requirement Guidance Guidancei. Add “Part 52” to definition of Dedication ✔
ii. Definitions for Part 76 facilities (i.e. Basic Component, Substantial Safety Hazard) ✔
iii. Definition of Critical Characteristics for non-reactor facilities ✔
Build on Experience (cont’d)Build on Experience (cont d)
• Clarify the Regulations• Clarify the Regulations– Simplify when possible
– Revisit definitions in Part 21
– Must comply with Section 206 “Noncompliance”
of the Energy Reorganization Act of 1974
• Provide Updated Regulatory Guidance– Benefit from over 33 years of experience
– Opportunity to consolidate existing guidance
• Use stakeholder input to improve the Part 21 processUse stakeholder input to improve the Part 21 process
6
MilestonesMilestones
Offi f N R t b li i
212121• Office of New Reactors began preliminary
rulemaking activities in December 20091
• NRC agency-wide Part 21 working group formed in January 2011
1
y– Discussions on holistic approach to clarify Part 21
P li i di i t bli ti• Preliminary discussions at public meetings– 2011 RIC
– NUPIC Vendor Conference
– EPRI JUTG7
Milestones (cont’d)Milestones (cont d)212121
• August 1, 2011: Public Meeting for early stakeholder feedback
1• September 2011: Commission (SECY)
1
paper – Communicate the staff’s plan to develop the
regulatory basis for rulemakingregulatory basis for rulemaking
• Rulemaking kickoff meeting in Fiscal Year 2012• Rulemaking kickoff meeting in Fiscal Year 20128
Milestones (cont’d)Milestones (cont d)YOU YOU ARE ARE
Vendor Quality Program for Critical y gEquipment & Major Purchases
Business Need
This program is in response to the need for improved vendor quality with the repair and fabrication of critical equipment quality with the repair and fabrication of critical equipment and major purchases of plant equipment. The focus on equipment reliability and the economic risk of downtime equipment reliability and the economic risk of downtime associated with vendor work on critical equipment and major purchases have increased significantly due to:• Shortened outages and longer operating cycles (days vs. months)
• A greater business need for maximizing plant productiong g p p(goal of decile vs. quartile)
• Implementation of Major Projects and New Plant Development
2
Business NeedThis program is intended to address high risk critical equipment and major purchases. This program addresses:
• Off-site vendor critical equipment repair• Off-site vendor critical equipment fabrication and • Off-site vendor critical equipment fabrication, and • Major purchases (i.e. considered to be a cost greater than
$100K (cost per item) for new repair or rework of $100K (cost per item) for new, repair, or rework of spare/replacement equipment, Major Project equipment and key parts).y p )
• Repair/Fabrication of critical equipment and major purchases can have a significant impact to the site outage/operating schedule if nonconforming or unacceptable conditions are identified during the fabrication, receiving, pre-installation review or installation process.installation process.
3
Business Need
• Appendix B: QA Programs robust & abundant
T h i l ti dil il bl• Fewer vendors with App. B QA
Programs
That was then: 25/30 yrs. ago This is now:
• Technical expertise readily available• Growing market in Nuclear• Utility and AE presence at vendor
Programs• Eroding knowledge transfer (Retirements)
• Nuclear is a small niche marketUtility presence at vendor is limitedcommonplace
• Lead time flexibility• Detailed engineering reviews
• Utility presence at vendor is limited• Shorter outages, longer operating cycles• Less staff (on both sides) I’m sure you all
h h d “d ith l ” • Specifications reflected current technology• High volume of purchases• Equipment/Operating reliability standards
have heard “do more with less”. • Specifications out of date• Lower volume of purchases
The basic intent of this Program is to proactively schedule and determine quality assurance measures to be and determine quality assurance measures to be implemented prior to sending high risk critical equipment out for repair or acquiring new high risk critical equipment. This for repair or acquiring new high risk critical equipment. This helps ensure critical equipment and major purchases are repaired/purchased and received in a timely manner to support improved Equipment Reliability.
5
Program IntentEngagement and ownership of vendor quality by the plant Engineering and Maintenance organizations, Major Projects and New Plant Development organizations as well as Supply Chain is the most Development organizations, as well as Supply Chain is the most important factor for this program to be successful. The different elements of the program have “natural” lead organizations, but it is expected that all impacted organizations collaborate to lend the most effective support to plant operations.• Engineering is responsible to designate what equipment is critical, preparation of g g p g q p , p pCommon/Update Specifications, participate in completion of Major Purchases & Vendor Quality Checklists, and vendor shop surveillances as needed.• Maintenance is responsible to participate in completion of Quality Checklists, perform vendor shop surveillances as needed, and perform pre-/post-implementation inspection and testing of items. • Supply Chain is responsible to coordinate selection of preferred providers, assist in identification of high risk critical equipment or major purchases for application to this program identification of high risk critical equipment or major purchases for application to this program, and facilitate completion of Major Purchases & Vendor Quality Checklists.
6
Guiding Principles1. Clearly and proactively communicate what is expected of
internal organizations and the vendor (i e providing clear internal organizations and the vendor (i.e. providing clear and accurate specifications or requirements).
2 Ensure the vendor has internalized the expectations (i e 2. Ensure the vendor has internalized the expectations (i.e. the people actually performing the work understand the expectations as opposed to the sales rep.) i.e. their Engineering, QA and Shop personnel
3. Provide feedback and hold the vendor accountable if performance does not meet expectations i H d performance does not meet expectations. i.e. How can a vendor improve his quality, if no one tells him it is not up to your standards?
7
Program Participants• Component Engineers (these folks make good subject
matter expects)p )
• System Engineers• Responsible Engineers (Site Design Engineering, p g ( g g g,
Subject Matter Expects, really these folks work with this equipment everyday, therefore input should be very much valued)
8
Measures Applied to a Repair/Purchase1. Proactive scheduling of equipment repairs and purchases2. Clearly identify and communicate technical and quality
requirements• Repair/purchase specs or other requirements• Pre-repair/purchase submittals (these can be repair plans,
repair/fabrication related procedures, drawings, FME procedures )
• Vendor surveillance (get intrusive your paying the bill)• Vendor surveillance (get intrusive, your paying the bill)
• Vendor documentation (ask for it in writing and make sure you get all of it, you may not get a second chance, ensure the vendor understands that any deviation f th PO i t d t b id tifi d)from the PO requirements need to be identified)
• Acceptance testing (will it be during fabrication, per/post installation?)
• Receiving inspection (this inspection should be timely not 2 or 3 months after • Receiving inspection (this inspection should be timely, not 2 or 3 months after you received the material at your site – so that any non-compliances can be dealt with in a timely manner)
9
Measures Applied to a Repair/Purchase (cont.)
3. Update CAT ID and prepare requisition4. Perform vendor surveillance (try to use a subject matter expert, identify any
NDE/testing witness points, perform cleanliness and FME inspections (before the point of no-return where onsite inspections cannot see the FME), are there final performance tests to be witnessed at the vendors’ facility?)
5 R i i t d f i t i ti 5. Receive equipment and perform receipt inspection (consider using enhanced receiving inspection requirements by specifying critical acceptance criteria that can be examined at receipt )
6 P f P /P t i t ll ti i ti /t ti 6. Perform Pre-/Post installation inspection/testing (Maintenance inspection/testing of equipment (beyond what is required during the receipt inspection) should be performed promptly after receipt – where needed. This provides for early identification of potential quality concerns that may not be identified during the receiving identification of potential quality concerns that may not be identified during the receiving process)
10
Checklists• NGGM-PM-0020 is the governing program manual which
contains the following elements:• Major Purchases and Vendor Quality Checklist (this is the heart of
this program. Supply Chain Procurement Specialists will facilitate the completion of the Checklist with Engineering providing technical and quality guidance along with Maintenance providing their perspective. This is best accomplished in a face-to-face working meeting to complete the Checklist. The Checklist helps ensure all perspectives have been considered in determining what supplemental vendor quality measures are warranted in a given case This is really a very useful tool for both a seasoned warranted in a given case. This is really a very useful tool for both a seasoned professional, or a young person in their nuclear career, since it provides excellent questions to help facilitate a quality product.
• Risk Assessment Guide (this provides examples of equipment and scenarios Risk Assessment Guide (this provides examples of equipment and scenarios that should be considered for evaluation under this program)
11
• Is the program working? - - Good Question -p g g
Some success stories:• Some success stories: Due to intrusive vendor surveillance for a Dry Fuel Storage Project, one of our auditors
identified fraudulent material being used during the fabrication of the dry storage canisters. As a result the supplier wrote a Part 21 to the industry and further testing / evaluation was As a result the supplier wrote a Part 21 to the industry and further testing / evaluation was performed on other material that was supplied.
As part of a vendor / PGN Improvement plan an FME procedure was implemented by a As part of a vendor / PGN Improvement plan, an FME procedure was implemented by a vendor in all their business units.
We have seen a decreasing trend in Vendor Quality NCRs and Vendor Quality Receiving We have seen a decreasing trend in Vendor Quality NCRs and Vendor Quality Receiving Deficiency Rates. These trends have been monitored since 2005.
12
QUESTIONS ?QUESTIONS ?
13
EPRI JUTG Procurement ForumOff Si R i f L C i i l AOff-Site Repair of Large Critical Assets
August 24, 2011g ,
William Ware
1
Program Scope and Objectives• Off-Site Repair of Critical Components
Subset of Vendor Oversight Program activities– Subset of Vendor Oversight Program activities
• Vendor Oversight Program Objectives:• Vendor Oversight Program Objectives:To improve the quality of engineeredcomponents through p g
– a more rigorous approach to the performance of internal procurement activities, and
– a more structured and involved approach in the oversight of the vendor’s activities
–2
Definitions• Engineered Component
An item that has been specifically designed and– An item that has been specifically designed and manufactured to meet SNC technical requirements.
• Large MotorLarge Motor– Horsepower rating of 250 horsepower and greater
• Large PumpLarge Pump– Power plant pumps that are driven by large motors.
L t U St t ithLet Us Start with:No calling on earth is insignificant if itNo calling on earth is insignificant if it
is accomplished with pride and artistry. Do whatever your heart leads you to do but do it so well that those whodo, but do it so well that those who come to see you do it … will bring others to watch you do it again and
again & again. g gwww.dsusainc.com (24/7) Phone 914.456.0227 2
ANYONE KNOWS WHO ANYONE KNOWS WHO THESE PEOPLE ARE?THESE PEOPLE ARE?
www.dsusainc.com (24/7) Phone 914.456.0227 3
Th i i l f t lThey Are: A Purchasing Agent, a
The original manufacturer no longer exists,
Obsolete manufacturing processes or technology,
Procurement Engineer And / Or Maintenance
The original manufacturer of a product no longer produces a product,
Manufacturers unwilling to produce Planner, …
LOOKING FOR AN
g plimited or small quantities,
Unusual component / material requirements, including tight delivery schedules,
OBSOLETECOMPONENT!
,
There is inadequate documentation of the original design,
The original design documentation has been lost or never existed,been lost or never existed,
www.dsusainc.com (24/7) Phone 914.456.0227 4
WHY One Should Consider WHY One Should Consider Reverse Engineering as Reverse Engineering as S l i f Ob l N lS l i f Ob l N la Solution for Obsolete Nuclear a Solution for Obsolete Nuclear
SafetySafety--Related ItemsRelated Items
R i i Th t AGING PLANTS HAVE ONE1) Recognizing That our AGING PLANTS HAVE ONE COMMON ASPECT … THAT IS … OBSOLESCENCE
2) Plant’s Supply ChainSupply Chain Philosophy Changed FROM:2) Plant s Supply ChainSupply Chain Philosophy Changed FROM:Just-in-Case … to … Just-in-Time
3) Approach is supported by EPRI NP-107372 & INPO NX 1037
RECOMMENDATION TO SUPPLIERSRECOMMENDATION TO SUPPLIERSIn Order to Ensure Successes PLEASE GET YOUR NQAIn Order to Ensure Successes … PLEASE GET YOUR NQA, ENGINEERING, PURCHASING, & SUPPLIERS, INVOLVED
EARLY IF NOT DURING QUOTATION STAGE!IF NOT DURING QUOTATION STAGE!
www.dsusainc.com (24/7) Phone 914.456.0227 5
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
FROM THIS?Figure 1: 32 YEARS OLD OBS Expansion
Differential Detector – WHAT WE STARTED with
AND THIS: Figure 2: Back View Steel Base Geometry
www.dsusainc.com (24/7) Phone 914.456.0227 6
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
ANDTHIS:Figure 3: Top View - Steel Base Geometry
AND THIS: Figure 4: U Core Configuration
www.dsusainc.com (24/7) Phone 914.456.0227 7
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
FROMTHIS: FROMTHIS:Figure 5: U Core Coil ANALYSIS
AND THIS: Figure 6: U Core Coil ANALYSIS
www.dsusainc.com (24/7) Phone 914.456.0227 8
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
www.dsusainc.com (24/7) Phone 914.456.0227 9
–– VIRTUAL TOURVIRTUAL TOUR ––VIRTUAL TOURVIRTUAL TOURREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
Figure 7: MOLD - Bobbin For U-Core Figure 8: U Core Coil Configuration
www.dsusainc.com (24/7) Phone 914.456.0227 10
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
Figure 9: U Core Configuration & Layout Figure 10: U Core / Coil BobbinFigure 9: U Core Configuration & Layout Figure 10: U Core / Coil Bobbin Configuration
www.dsusainc.com (24/7) Phone 914.456.0227 11
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
Figure 11: RE Unit Wiring to Coils & External Cable Lay Figure 12: RE Unit POTTING PROCESSFigure 11: RE Unit Wiring to Coils & External Cable Lay Out
Figure 12: RE Unit POTTING PROCESS
www.dsusainc.com (24/7) Phone 914.456.0227 12
–– VIRTUAL TOURVIRTUAL TOUR ––VIRTUAL TOURVIRTUAL TOURREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYFigure 13: WHAT WE STARTED WITH FAILED UNIT Figure 14: REVERSE ENGINEERED TESTED UNITSFigure 13: WHAT WE STARTED WITH – FAILED UNIT Figure 14: REVERSE ENGINEERED TESTED UNITS -
FINAL PRODUCTS
www.dsusainc.com (24/7) Phone 914.456.0227 13
Dollar$ and $en$e REVERSE ENGINEERING SUCCESS STORY REVERSE ENGINEERING SUCCESS STORY
TANGIBLE RESULTSTANGIBLE RESULTS
OEM Special Run No Assurance of No
Changes From originally
Reverse Engineered By DS USA, Inc
Like-for- LikeChanges From originally Supplied
12 Month 12 Month Lead Time Minimum Order 12 Units
Like-for- Like Assurance Provided
4 Month 4 Month Lead Time One Prototype & No Minimum Order 12 Units
Unit Cost For Min. Order = $ 25,000.00T t l C t f Mi O d
One Prototype & No Minimum Order
Unit Cost = $ 11,000.00 Total $aving$ = 56 % Total Cost for Min. Order
= $ 300,000.00 Total $aving$ = 56 %
www.dsusainc.com (24/7) Phone 914.456.0227 14
EFFECTIVNESS OF DS USA, Inc Approach To EFFECTIVNESS OF DS USA, Inc Approach To , pp, ppREVERSE ENGINEERING Of Obsolete Components In REVERSE ENGINEERING Of Obsolete Components In
Nuclear Power PlantsNuclear Power Plants OUR PROGRAMOUR PROGRAM MATURED OVER THE LAST 9 YEARS TO OUR PROGRAMOUR PROGRAM MATURED OVER THE LAST 9 YEARS TO
FULL INTEGRATION Over 54 DevicesOver 54 Devices, all Installed & Functional Without any
Problems, OFFERED To PLANTS OFFERED To PLANTS THAT MOVED FROM:
“Just-in-Case Policy” To “Just-in-Time Policy” OUR Customers Reported Back SHORTER Lead Time & OUR Customers Reported Back SHORTER Lead Time & pp
40-60% Savings Over the Cost of New EquipmentWHAT WE DO NOT TELL YOU:
REGARDLESS OF THE ITEM’S CLASSIFICATION ALL WORKIS PERFORMED UNDER OUR FULLY IMPLEMENTED 10 CFR 50 App B NQA PROGRAM … TREATED AS SAFETY RELATED.
www.dsusainc.com (24/7) Phone 914.456.0227 15
DS USA, Inc Basis For Its Reverse DS USA, Inc Basis For Its Reverse ,,Engineering ApproachEngineering Approach
EPRI NP 107372 J l 1998 id EPRI NP 107372, July 1998 provides sufficient guides on Approach, Criteria, and Benefits of Reverse Engineering whenBenefits of Reverse Engineering when addressing OBS in Nuclear Safety and non-Safety applications, Sam A Yousif Sam A Yousif ––CoauthorCoauthorCoauthor, Coauthor,
INPO Procedure NX1037, Rev 1, November 2003 Recognizes Reverse Engineering as an2003 Recognizes Reverse Engineering as an OBS Solutions (This was based on a NUOG initiative), Sam A Yousif Sam A Yousif –– Technical Review. Technical Review.
www.dsusainc.com (24/7) Phone 914.456.0227 16
The Benefits To YOUR PLANT ByThe Benefits To YOUR PLANT ByThe Benefits To YOUR PLANT ByThe Benefits To YOUR PLANT ByUsing Our Reverse Engineering ApproachUsing Our Reverse Engineering Approach
C l t l D li t F Fit F ti Completely Duplicates Form, Fit, Function, Materials, and Interface/Performance of Original Part, Component, or Assembly,Original Part, Component, or Assembly,
Allows and provides a: “Like-For-Like / Equivalent Replacement unit” ,
No Prohibitive Modification Cost to your Plant,O l “SIMPLE” E i l P NP 5652 Only a “SIMPLE” Equivalency Per NP-5652 Is Required,
www.dsusainc.com (24/7) Phone 914.456.0227 17
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
HOW Do YOU GET
FROM THIS?Figure 1: 28 Years OLD OBS HUMISTAT
AND THIS: Figure 2: HUMISTATA Humidity Sensor Installation
Installed Humistat Humistat Sensor Installation
www.dsusainc.com (24/7) Phone 914.456.0227 18
Inside a Huffman BoxHumistat Sensor Installation
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
HOW Do YOU GET
TO THIS?Figure 3: Reverse Engineered Class 1E Qualified Like-
For-Like Replacement
AND THIS:Figure 4: RE Unit's Sensor INSIDE Humidity ChamberDuring ATP BEFOR & AFTER CLASS 1E Qualification
www.dsusainc.com (24/7) Phone 914.456.0227 19
–– VIRTUAL TOURVIRTUAL TOUR ––REVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORYREVERSE ENGINEERING SUCCESS STORY
HOW Do YOU GET
AND THIS TO THIS?Figure 5: Reverse Engineered Class 1E Qualified Like-
For-Like HUMISTAT PCB
AND THIS: Figure 6: Reverse Engineered Class 1E Qualified Like-
For-Like HUMISTAT
www.dsusainc.com (24/7) Phone 914.456.0227 20
DS USA, Inc REVERSE ENGINEEREDDS USA, Inc REVERSE ENGINEEREDQualified Class 1E HumistatQualified Class 1E HumistatQualified Class 1E HumistatQualified Class 1E HumistatSPECIFICATION SHEET
SPECIFICATION & FEATURES: DS USA, Inc Acceptance Test Procedure SPECIFICATION & FEATURES: Model No: DS6398 Power: 115 VAC, 10% Frequency: 50 / 60 Hz Dimensions: 4.1 x 4.1 x 4.1 in
, pDevelopment: ATP 2005-10
Class 1E NUCLEAR QUALIFICATION Seismic Qualification – IEEE-344 & IEEE-
3231) Qualification Procedure No. ER2005-01, Rev. B2) DS USA, Inc Report No. QR2005-01, Rev. B
Connections: Lug less Wire Terminal Block
Circuit Operating Temperature:(0 0 C to 70 C)
Circuit Storage Temperature:
) S US , c epo t o Q 005 0 , e EMI / RFI Qualification – EPRI TR-102323,
Rev. 21) Qualification Procedure No. ER2005-02, Rev. A2) DS USA, Inc Report No. QR2005-02, Rev. ADS USA, Inc Acceptance Test Procedure
Development: ATP 2005-10 (- 40 0 C to 85 C) Relay Type: Dry Contacts Rely Output: NO & NC, 8 A @ 115 VAC, 5 A @ 24 VDC Signal Output: 0 to 1 VDC, Proportional
Development: ATP 2005 10
Signal Output: 0 to 1 VDC, Proportional to Hygrosensor Dial Reading
Alarm Indication: Red LED indicates sensor reading above set point
Set Point Calibration:Hydrodynamics' Set Points
www.dsusainc.com (24/7) Phone 914.456.0227 21
y y
Class 1E Nuclear QualificationsClass 1E Nuclear Qualifications Overview – As ApplicableAs Applicable
Dedication of Commercial Grade ItemsDedication of Commercial Grade Items 100 %100 % Dedication of Commercial Grade Items Dedication of Commercial Grade Items –– 100 %100 %Governed By: 10 CFR PART 21, EPRI TR-106439, EPRI NP-5652, EPRI NP-6406, NRC Generic Letter 89-2 91 05 DS USA Inc KF13172, 91-05, DS USA, Inc KF1317
Over 54 DevicesOver 54 Devices, all Installed & , all Installed & F i l Wi h A P blF i l Wi h A P blFunctional Without Any Problem Functional Without Any Problem
DS USA Inc has successfully Reverse Engineered Fabricated andDS USA, Inc has successfully Reverse Engineered, Fabricated and Tested / Qualified a wide variety of components, parts, and assemblies, SOME ARE:
1)1) Printed Circuit BoardsPrinted Circuit Boards1)1) Printed Circuit Boards, Printed Circuit Boards, 2)2) AC and DC Motors, AC and DC Motors, 3)3) Power Supplies, Power Supplies,
R t t & T ittR t t & T itt4)4) Rotameter, & Transmitters,Rotameter, & Transmitters,5)5) ASME III, Class 3, Heat Exchanger ASME III, Class 3, Heat Exchanger 6)6) Expansion Detectors, & Turbine Parts, Expansion Detectors, & Turbine Parts, )) p , ,p , ,7)7) Mechanical Gears and Fittings, Mechanical Gears and Fittings, 8)8) Rotary Relays, Coils, Rotary Relays, Coils, 9)9) Fire Detection ComponentsFire Detection Components9)9) Fire Detection Components. Fire Detection Components.
www.dsusainc.com (24/7) Phone 914.456.0227 23
CONCLUSION No 1YOUR PLANT GREATLY CAN BENEFIT USINGYOUR PLANT GREATLY CAN BENEFIT USING
CORRECTLY Is BOTH LEGAL & ETHICALCORRECTLY Is BOTH LEGAL & ETHICAL
MidMid 1980 U S G t S d th1980 U S G t S d th MidMid--1980: U.S. Government Sponsored the 1980: U.S. Government Sponsored the Development of Large Scale Reverse Engineering Development of Large Scale Reverse Engineering Program .Program .gg
MIL-HDBK-115: DOE / US Air Force Launched a Reverse Engineering Program – 494 Revere Engineering Projects That
Yielded a $ 378 Million …
Average Return On Investment of 14:1 Average Return On Investment of 14:1 EPRI TR 107273 Chapter 3 & 4: Provides Guides On: EPRI TR 107273, Chapter 3 & 4: Provides Guides On:
Patents/Trademarks: Are Protected for 20 Years20 Years,
Trade Secrets: Are Not Protected If Obtained Legally (Agreement). (Agreement).
www.dsusainc.com (24/7) Phone 914.456.0227 25
Questions?
www.dsusainc.com (24/7) Phone 914.456.0227 26
Purpose Statement
Reverse Engineering maybe used to • identify an obsolescence solution,
• achieve cost savings by purchasing from an alternate supplier,
resolve lead time concerns or• resolve lead-time concerns or
• improve an item’s performance.
3
Site Roles and Responsibilities
Procurement Design EngineeringProcurement Engineering• Interface with the
Design Engineering• Provide technical
assistance as• Interface with the applicable vendor throughout the
assistance as required
gentire reverse engineering process
• Develop criteria & approve drawings
4
• Perform evaluation
DefinitionEPRI defines reverse engineering as the process g g p
of developing technical information sufficient to duplicate an item by • physically examining, measuring or testing existing
Refer to EPRI TR-107372 "Guidelines forRefer to EPRI TR 107372, Guidelines for Reverse Engineering at Nuclear Power Plants“.
5
Definition of Like-for-LikeExelon’s governance and oversight procedure g g p
defines a replacement item as like-for-like if there is a high level of confidence that there have been no changes in the design, material, or manufacturing process since the procurement of the item being replacedthe item being replaced.
6
Is Reverse Engineering Considered Is Reverse Engineering Considered Like-for-Like?
Based upon the nature of a reverse engineered item, both the design and manufacturing of the new item will inherently change.
7
Therefore, the item would not meet the definitionnot meet the definition of…
lik f liklike-for-like.
8
Reverse Engineering vs Item Reverse Engineering vs. Item Equivalency Evaluation
Reverse Engineering and Item Equivalency Evaluation processes are not the same.
Item Equivalency Evaluation may be necessary to document the replacement item is acceptable if h h b d th t t hchanges have been made that warrant such an
evaluation. (i.e.., material changes)
9
Who Performs Reverse Engineering?
Station • Follows the guidance in
EPRI TR-107372
Vendors• Interfaces with the
requesting station to ensureEPRI TR-107372
• Perform evaluations for items that pass the SM-AA-300 IEE Screening and are
requesting station to ensure that item meets the; technical aspects of the
original300 IEE Screening and are not prohibited by SM-AA-300-1001 Attachment C
original
design features that are unique to nuclear safety-related
10
Legal Issues
Reverse Engineering is a legal practice and, when performed ethically, is not a form of design infringement or theft.
Any legal concerns about Reverse Engineering h ld b dd d b b ishould be addressed on a case-by-case basis
with the legal department.
11
Proprietary Data
Technical data that embodies trade secrets developed at private expense, such as • design procedures or techniques
• chemical composition of materials or
f t i th d t t t• manufacturing methods, processes, or treatments, including minor modifications
12
Proprietary data (cont.)
Is not generally known or available from other sources
Has not been made available by the owner to others
And is not readily available to the public without obligation concerning confidentiality
13
Proprietary data (cont.)
OEM documents may not be given to vendors assisting in Reverse Engineering.
It is acceptable to transcribe design data / requirements from OEM documents and submit th h t itt l f d i i f tithrough a transmittal of design information.
14
Design Information
Written documents or graphical information which establish design, fabrication, construction, installation, erection, and operational or test requirements for a system, structure or componentcomponent.
15
Where to start
Identify the item’s end use applications and function.
Perform a failure mode effect analysis (FMEA).
Identify the item’s critical characteristics for design.
16
Reverse Engineering “To Do” List of Reverse Engineering To Do List of Activities
Collect / Review the Item’s Design Data.• This may include component / item drawings, system
drawings, vendor specification, vendor procedures, vendor manual information, station procedures and Exelon specifications.Exelon specifications.
17
Reverse Engineering “To Do” List of Reverse Engineering To Do List of Activities
Inspect, measure and test the original item.• This data should be recorded in the form of drawings
and / or reports by either the site or the vendor. It should be documented whether the item is new or used. If used, the service time and conditions are to beused. If used, the service time and conditions are to be identified. This information will be useful when evaluating critical characteristics.
18
Reverse Engineering “To Do” List of Reverse Engineering To Do List of Activities
Evaluate Plant / Component / Item Interfaces including Fits & Tolerances, Inputs / Outputs.• This includes mechanical and electrical interfaces. If a
vendor is assisting in the Reverse Engineering, interfacing item data or actual interfacing parts mayinterfacing item data or actual interfacing parts may need to be supplied to the vendor.
19
Reverse Engineering “To Do” List of Reverse Engineering To Do List of Activities
Review Item History. • This is to include a review of OPEX / EPIX and Exelon
condition reports. Historical failures may identify inadequate design characteristics that could be alleviated during the Reverse Engineering.alleviated during the Reverse Engineering.
20
Reverse Engineering Activities (cont.)
Establish Item’s Design.
Design drawings / requirements for the Reverse E i d it t b d l d b th itEngineered item are to be developed by the site or vendor.
All documents that are prepared by a vendor must beAll documents that are prepared by a vendor must be provided to the site for approval prior to manufacturing. Special tests / inspections / procedures that are required to demonstrate the replacement item will performto demonstrate the replacement item will perform acceptably are to be identified.
The manufacturing of a prototype may be required.
21
g p yp y q
Screening
Once it has been determined to proceed with the Reverse Engineered item, an Item Equivalency Screening must be performed.
The item may be approved as a Pre-Engineered Solution if it is manufactured to industry standards.
The item may be approved by a Safety Basis evaluation if dimensions and tolerances are the only unknowns. Justification is to be provided on why the dimensions and tolerances were determined to be acceptable.
An Item Equivalency Evaluation is required when there are changes to An Item Equivalency Evaluation is required when there are changes to fit, form or function. When enhancements are made to the item, an Item Equivalency Evaluation is required.
23
Configuration Control
“It is incumbent upon all engineering groups to maintain appropriate configuration control.”
24
Reverse Engineering Guidelines
link to Reverse Engineering Flowchart
25
In Summary
Reverse Engineering maybe used to identify an obsolescence solution, achieve cost savings by purchasing from an alternate supplier, resolve lead-time concerns or improve an item’s performanceperformance.
But………if performed incorrectly can result in legal ramifications as well as loss of configurationlegal ramifications as well as loss of configuration control.
26
27
Reverse Engineering Guidelines
Identify Item’s End Use Applications, Function and perform a FMEA
Determine Need for Reverse Engineering Obsolete, i.e. no longer available Cost Savings • Resolve Lead-Time Concerns Enhancement / Improvement
Identify the Item’s Critical Characteristics for Design Reference the critical characteristics / parameters section of the IEE template SM-AA-300-1001-F-01
Collect / Review Item’s Design Data OEM documents shall NOT be sent to vendors. OEM information may be transcribed Design data / requirements are to be transmitted to vendors using a TODI.
Inspect, Test and Measure Item On-Site Vendor
Evaluate Plant / Component / Item Interfaces including Fits & Tolerances, Inputs / Outputs Data or actual interfacing parts may need to be supplied to the vendor
Review Item History OPEX / EPIX review
Establish Item’s Design Create drawings and specifications Identify Special tests / inspections / procedures If a prototype is required, identify / perform required testing
NUPIC Audit Checklist Revision 15NUPIC Audit Checklist Revision 15became effective 08/01/2010: Added more attributes for 10CFR21 evaluation
Timeframes requirements for 10CFR21 records retention/maintenance (10 years)retention/maintenance (10 years)
Added attribute for overall effectiveness of internal audit process
Added attribute for overall effectiveness of corrective action process
R i i 16 C i SRevision 16 Coming Soon7
Performance BasedPerformance Based Supplier Audit (PBSA)
Audit Focus Upon Product Quality and Performance-Related Activities
Input from NUPIC users is crucial
Avoid Generic Programmatic Attributes Avoid Generic Programmatic Attributes that are Already Covered in the Checklist
I l d T h i l Ch t i ti d Include Technical Characteristics and Acceptance Criteria
Include Operating Experience8
NUPIC Ch llNUPIC Challenges
Software Dedication
A2LA/NVLAP A2LA/NVLAP
ILAC Accreditation Process
C b S it Cyber Security
Control of Safeguards
Safety Conscious Work Environment
Safety Culture Safety Culture
9
NUPIC Ch llNUPIC Challenges
Continuity of the NUPIC Organization
New Auditors and Training
Driving Performance Improvements in Closing Audits that Have Been Open for aClosing Audits that Have Been Open for a Considerable Length of Time
10
C ll b ti Eff tCollaborative Efforts
NUPIC & EPRI Working to develop New Guidance for the Dedication of Commercial Software
NUPIC & EPRI working to develop course NUPIC & EPRI working to develop course for NUPIC Audit Team Leader
NUPIC & NEI working to Develop a NUPIC & NEI working to Develop a Method of Acceptance for International Laboratory Accreditation CooperationLaboratory Accreditation Cooperation (ILAC) 11
Q ti ?Questions?
12
Becoming a Becoming a NUPIC SupplierNUPIC Supplier
EPRI Procurement Forum August 25 2011August 25, 2011
Evaluation of Vendor PerformanceDo not Certify Vendors like ISO/ASME
Perform Joint Utility Audits
Exchange Results of Individual Utility Auditsg y
Develop Methodologies to Monitor and Improve Vendor Performance
Provides a Forum to Constantly Improve the Utility Oversight Process of Vendors
3
NUPICNUPIC Provide a Forum for Discussion of
P t Q lit A IProcurement Quality Assurance Issues
Exchange Vendor Performance Information
Discuss Concerns Resulting from Quality g yAssurance Regulatory Issues
Assist Vendors in the Resolution of Member Concerns
4
NUPICNUPIC Provides an Interface for ProcurementProvides an Interface for Procurement
Quality Assurance ActivitiesMaintain Interface with NEI to CoordinateMaintain Interface with NEI to Coordinate
Communication/Dialogue with the NRC
Maintain Interface with NEI to Provide Input a ta te ace t to o de putand feedback on Industry Positions/Commitments Regarding Procurement Quality Assurance Issues
Maintain Interfaces with EPRI & JUTG
5
O ll POverall Process
One Utility Shared Utility NUPIC
One Utility At a time One Utility At a time
Good Work Brings More Work
Sh d Utilit A dit Shared Utility Audit
5 Utility Users NUPIC Supplier Audit
5/2006 (1) 5/2009 (3) 5/2011 (13)
6
Wh Y G C llWho You Gonna Call
Plant Users/Organizations (Eng, Projects)
Plant Needs Must Be Established Plant Needs Must Be Established
What Do I Need to DoWhat Do I Need to Do Have/Develop a QA Program which p g
Implements the Requirements of 10CFR50 Appendix B and 10CFR21pp
Market your QA Program as Well as the Safety Related Products or ServicesSafety Related Products or Services
Successfully Pass a Utility Audit per10CFR50 Appendix B and 10CFR21per10CFR50 Appendix B and 10CFR21
Contact NUPIC Rep. for Utility that Audited you to ensure NUPIC database isAudited you to ensure NUPIC database is Updated to list your Company as a User 8
S li A l PSupplier Approval ProcessUtilit O l A dit ($$$ + Ti C i ) Utility Only Audit ($$$ + Time Consuming)
Planning & Scheduling
Assemble Auditors & Technical Specialist
Perform Audit Perform Audit
Issue Report and Any Findings
A l T & S li N tifi ti Approval Type & Supplier Notification
Procurement May Now Commence
Update NUPIC Database9
Supplier Approval ProcessSupplier Approval Process Audit/Survey Exists ($ + Fairly Quick)y ( y )
Att ib t f T S liAttributes of Top SuppliersRobust Internal AuditRobust Internal Audit
Comprehensive System of Planned and Periodic Internal Audits
Qualified Audit Participants have No Direct Responsibility in the Areas Being Audited
Checklists/Procedures Used with Documented Objective Evidence
Intrusive Intrusive
Follow-up Action Taken Where Needed
13
Att ib t f T S liAttributes of Top Suppliers Robust Sub-Supplier Audits/Surveys Robust Sub-Supplier Audits/Surveys Effective Control and Release of Procurement
Documents Including Changes
Effective Evaluation, Selection and Assessment of Sub-suppliers with Clear Program Distinction Between Audit Vs Commercial Grade SurveyBetween Audit Vs. Commercial Grade Survey
Comprehensive and Instrusive Audits
Qualified Audit Participants have No direct Qualified Audit Participants have No direct Responsibility in the Areas Audited
Checklists/Procedures used with Documented Obj ti E idObjective Evidence
Follow-up Action is Taken Where Needed 14
Att ib t f T S liAttributes of Top Suppliers Strong Corrective Action SystemStrong Corrective Action System
Low Threshold to Identify Issues and Proper Classification Relative to Significance
Clear Connection between the Nonconformance and Corrective Action Processes and 10CFR21
Measures Established and Implemented to AssureMeasures Established and Implemented to Assure Conditions Adverse to Quality are Promptly Identified and Corrected
Adequate Actions Taken to Prevent Recurrence for any Significant Conditions Adverse to Quality
Deficiencies Identified by Customers are Adequately Deficiencies Identified by Customers are Adequately Assessed and Entered into the Nonconformance/Corrective Action Program 15
Q ti ?Questions?
16
Acquiring the ASME “N” Stamp; Development of the Required Programs and Implementation Development of the Required Programs and Implementation
a Suppliers Experience
Bill HenwoodVP & Director, NuclearJUTG Procurement ForumAugust 24, 2011
Who is VALVtechnologies?
Over 460 Employees Worldwide
Corporate Headquarters & Factory L tiLocations:
• Houston, Texas (2)• United Kingdom• China
Sales Offices: Direct Personnel • Dubai• China• China• Korea• France• Netherlands• South Africa• South America
N‐3328 N‐3329
The Global Leader in Absolute ZERO LEAKAGEMetal Seated Valve Technology
Who is VALVtechnologies?
Authorizations & Qualifications
• ASME III Class 1, 2 & 3 • 10CFR50 Appendix B• Part 21 Acceptance• ANSI N45.2• NQA‐1• ISO 9001 ‐ 2000• PED Compliant (Europe) • Lloyds EMEA Register SIL‐3 (Europe)• Nuclear Industry Assessment Committee
N‐3328 N‐3329
Primary Manufacturing Plant in Houstony g
Bingle Road 125,000 Square Ft
Bingle Road 1986
Railhead 40,000 Square Ft on 15 AcresPlanned 150,000 Square Ft Factory Site
Valves for Nuclear Power
“Application Solutions” for the Next 60 Years…”ppVTI technology did not exist during the Generation II and early Generation III
plant design phases (1960’s – 1980’s).
MSBV PSGV Control Valve Inline Check
Nuclear Future
VALVtechnologies NuclearManagement Commitment
“Nuclear is a significant contributor to the future of our company; perhaps the single largest area of growth for the next 10 years” perhaps the single largest area of growth for the next 10 years
‐ Kevin Hunt , CEO
Attracting New Nuclear Suppliersg ppThe Nuclear Energy Institute:
•Since 2008 the NEI has conducted 9 vendor workshops in strategic locations throughout the United States.
•The effort is intended to attract new and former nuclear suppliers to participate in the Nuclear Renaissance.
•Thousands of potential vendors have attended, most serious vendors have attended more than one.
• The abandonment of qualified nuclear suppliers in the post Three Mile Island era when most of the US domestic projects were halted and/or cancelled has caused a huge void.
•The effort is supported by the NSSS and EPC’s involved in the new build arena•The effort is supported by the NSSS and EPC s involved in the new build arena.
•The efforts are necessary and well intentioned… but no one talks about the costs!
Cost of Nuclear at VTI:Cost of Nuclear at VTI:
$4,773,000.004,773,Program development and implementation costs since 2005
Cost of Nuclear at VTI:Cost of Nuclear at VTI:
$4 773 000 00
•Commencing in early 2005 though September 2007 personnel initially assigned to develop and implement the nuclear program were not totally dedicated to the
$4,773,000.00
develop and implement the nuclear program were not totally dedicated to the process. Personnel were assigned to other Engineering and Quality related duties and thus distracted from the task at hand.
•Costs for the first 2 + years, for salaries, consultants, programmatic development resulting in a unusable program.
Approximately $1,600,000.00
•Investment in Dedicated Personnel, Consultants, Programmatic Development, Training, Supplier Audits, Implementation and ASME Audit since October 2007.
Approximately $3,173,000.00
State of the Industry
ASME “N” Stamp Holders
Nuclear Vessels, Pumps, Valves, Piping Systems, Storage Tanks, Core Support Structures, Concrete Containments, and Transport Packaging
ASME N Stamp Holders
• 213 Authorizations Worldwide• Held by 157 companies• VTI was Number 110 in March of 2009• 1008 ASME “N” Stamps in 1980p 9• 62 at the low point in the mid ‐ late 1990’s
Core Nuclear Team – 85 years of combined experience…
Noel MathenyNuclear Industry
Specialist
Cesar MejiaNuclear Engineering
SpecialistManager
Dave Coon Tim HowardDave CoonNuclear Quality
Manager
Tim HowardNuclear Territory
Manager
The VTI Nuclear TeamThe VTI Nuclear Team
Focused Key Personnel assigned to Team…
Erica ChavezProject Manager
Ken RickardMarketing Manager
Donna Moritz Peter MiddletonDonna MoritzPurchasing
Peter MiddletonEngineering
Hi t P D l tHistory ‐ Program Development
History ‐ Program Developmenty g p
Major Milestones in Program Development at VTI:
Efforts to develop and implement the VTI nuclear program commenced in 2005 as the ramp up to the “Nuclear Renaissance” was in full swing.
(O t b ) N l I d t Di t(October 2007) Nuclear Industry Director‐ Reports directly to the CEO with responsibility for the general oversight of all nuclear operations, sales and marketing efforts.
(November 2007) Nuclear Engineering Manager‐ Reports to the Director of the Nuclear Industry. His responsibilities include all aspects of engineering design. The NEM is the technical design authority, and supports all nuclear operations. He is also responsible for contract administration, project management.
(December 2007) Nuclear Quality Assurance Manager‐ Reports to the Director of (December 2007) Nuclear Quality Assurance Manager‐ Reports to the Director of Quality, responsible for the overall nuclear quality program, including maintaining and updating the NQAM’s, our nuclear procedures, the ASL; he is our Certified Lead Auditor, and interfaces with our ANIS.
(January 2008) Consultants Hired‐ Reported to the Director of the Nuclear Industry. They ( y ) p y yconducted a complete assessment of all existing quality programs within VTI, including ISO 9000, ASME, API and PED for consistency and incorporation in our nuclear program development.
History ‐ Program Developmenty g p
Major Milestones in Program Development at VTI:
(May 2008) 10CFR50 Appendix B ‐ Introduced for supply of Safety Related VTI Product to the existing Nuclear Plant Operators, NSSS and EPC’s for the New Build Industry
(October 2008) Nuclear Industry Assessment Committee ‐ NIAC Membership granted (October 2008) Nuclear Industry Assessment Committee NIAC Membership granted (NIAC) Organized in 1994 as a result of an industry initiative to share audits among members; it performs supplier qualification activities eliminating the redundancy and costs associated with duplicate audits
(January 19 21 2009)ASME “N” and “NPT” Audit ‐Successful audit with four (January 19‐21, 2009)ASME N and NPT Audit ‐Successful audit with four observations and ZERO findings. Certificate issuance in early March 2009
(May 2009) Appointed the VTI Nuclear Territory Manager ‐ Hired to cover the US existing fleet and manage our representative network Focuses on the Charlotte NC based NSSS & EPC’sexisting fleet, and manage our representative network. Focuses on the Charlotte NC based NSSS & EPC s
(November 2009) First NUPIC based audit was performed by Duke Energy – 4 day audit was a prelude to our first ever ASME Section III order. Zero findings or observations placed VTI in the Duke Energy Approved Suppliers List (ASL)gy pp pp ( )
History ‐ Program Development
Major Milestones in Program Development at VTI:(November 2009) First ASME “N” Stamped Order from Duke Energy – 12‐ 6” 300# 316SS MSBV with qualified gear operators and extended flexible drives. Acceptable PO received in mid December after 7 PO revisions.
Duke Oconee High Pressure Injection ValvesDuke Oconee High Pressure Injection Valves
(January 2010) DFT and VTI Sign License Agreement for Nuclear industry – Rounds out the VTI product offering and provides a path for DFT products to the Nuclear Industry
(March 2010) Successfully tested and delivered the first 4 ASME valves to Duke –(March 2010) Successfully tested and delivered the first 4 ASME valves to Duke 14 week delivery. 8 valves remained. 4 more tested and delivered in August 2010. Surveillance for the remaining required by March 2011 has been lifted.
Challenges –Moving Forward
Challenges ‐Moving Forward
f d d h l d d h f
g g
•Acceptance of our products and technology. Our products require a paradigm shift in an industry that thrives on obsolescence and is reluctant to change
•Placement on utilities Approved Suppliers List is key to the success. As a result of the Duke placement VTI is now on the ASL for the following Utilities:
• Progress Energy• Bruce Power (Canada)• EDF Energy (formerly British Energy)EDF Energy (formerly British Energy)• Nuklearna Elektrarna Krsko (Slovenia)
•ASL status as a commercial supplier will not allow for automatic acceptance for SR or ASME applications once your programs and authorizations are in placeor ASME applications once your programs and authorizations are in place.
Challenges ‐Moving Forwardg g
•Nuclear Procurement Issues Committee (NUPIC) was formed in 1989 by a
The Holy Grail‐NUPIC acceptance
partnership involving all domestic and several international nuclear utilities. The NUPIC program evaluates suppliers furnishing Safety‐Related components and services to nuclear utilities in accordance with 10CFR50 Appendix B.
•NUPIC based audits are not driven by suppliers with an ASME Section III programs as it is not a requirement.
• Audits are not automatic regardless of an existing Appendix B program or • Audits are not automatic regardless of an existing Appendix B program or NIAC membership; they are expensive, time consuming and will not occur unless a utility requires your product or service.
•Full NUPIC acceptance and automatic placement on the utility ASL requires five independent audits or audit results accepted by an auditing utility and 4 additional member utilities. A full NUPIC audit will be scheduled to review your 10CFR50 A di B thi d i l d i th t tiAppendix B program once this occurs and your company is placed in the rotation.
Challenges ‐Moving Forwardg g
•We continue have ASL issues at most nuclear utilities where NUPIC acceptance is a i W hi i h R i h i NUPIC di d H i requirement. We counter this with Representation who is NUPIC audited. However it
can add cost and loss of any competitive edge.
•Keeping representatives motivated…p g p
•New construction activity has slowed to a crawl.•NRC approval, political and economic pressures along with the new found ab ndance of nat ral gas are hindering the progression of ne n clear abundance of natural gas are hindering the progression of new nuclear construction.
•Shift in focus in existing fleet applications through major projects such as Extended Power Up rates and Steam Generator replacement are the new focusPower Up ‐rates and Steam Generator replacement are the new focus.
•Up to the equivalent of 8 new 1000MW plants will be added to the grid without building a single plant by 2015
•Return on the investment …
Lessons Learned
Lessons Learned
•Be well financed… whatever you think it going to cost… double it!
•Have the commitment and patience at the highest levels of management; it is paramount.
•Focus… once you decide don’t look back… don’t distract.
•Find and hire the best and brightest Core Team. (Quality & Engineering)
•Don’t go it alone; the right consultants can make all the difference.
•Carefully select your ANIS – they are your advocate.
•Educate your team members – dispel the fear of nuclear.
Lessons Learned
•Establish realistic deadlines and hold the team accountable.
•Contact the ASME as early as possible to schedule your audit; do all you can to hold to it.
•Focus on training; document it… and do it again.
•Be prepared to demonstrate and defend your processes. Don’t be adversarial p p y p
•Be organized, be prepared to make real‐time manual changes; have your documentation and procedures available and ready for review.
•Live the process… a nuclear program can improve your overall product quality, reduce rework, hold sub‐suppliers more accountable, advance customer satisfaction and increase profitability.p y
About KinectricsAbout Kinectrics Formerly the Technical Division of
Ontario Hydro, one of North America’s largest, most reliable utilities
Mechanical TestingLab High Voltage/
High Current Lab
Comprehensive facilities & advanced specialized laboratories near Toronto, ON Canada
300,000 SQFT on 15+ Acres Almost 100 years of advanced technical
expertise & experience In business as Kinectrics since 2001 Increased Business, Revenue and
Employees every year since 2001p y y y Over 400 scientists, engineers &
professional staff ~$100 000 000 Annual Revenue$100,000,000 Annual Revenue US Office (Cincinnati, OH)
Quality Quality Management and AuditsManagement and AuditsQuality Programs 10 CFR 50 Appendix B (US)/NQA-1CSA Z299.2CSA Z299.2CSA N285/NCA 4000ISO 9001-2000ISO 17025•Analytical & Environmental Chemistry Testing •MetrologyMetrology•High Voltage / High Current Labs
Numerous Audits:Exelon Audit per NUPIC Checklist (August 2008)MOX Project Audit (August 2010)NIAC Member and Approved Audits (January 2011)pp ( y )NUPIC Audit Completed (April 2011)Numerous other OEM audits (SI Audit May 2011)
TopicsTopics• About Kinectrics
• How to Get An Audit?• How to Get An Audit?
• Politics
• The First Sponsor
• Hard Work• Hard Work
• Patience
• The Audit
• ResolutionResolution
• Reflections
Politics•KI Chairman
S li iti t•Soliciting support
•Solve a business issue dor need
The Sponsor
•Need a willing utility sponsor:
T i t i dit ( NUPIC Ch kli t)• To invest in an audit (per NUPIC Checklist)
• To share the audit report
• To order something
• To keep you on the ASL
Hard Work
•Add as many utilities, leveraging the audit.
U bili i ibl•Use as many capabilities as possible to perform safety related work:
• Forensic/failure analysis
• Special analysis (oil and refrigerant)
• Small dedication projects
• Field testing services
•Be creative as well
• BeggingBegging
• Quid pro quo projects
Patience is a Virtue
•Patience comes from Latin root “patior: toLatin root patior: to suffer”
•Actual time fromActual time from focused effort to audit completion:
4 Y !4 Years!
Finally, the Audit
•Extensive reviews and preparationpreparation
•Self-assessment
Ki t i dit•Kinectrics audit was more complex:
L d b S h• Led by Southern
• Included Exelon, SCE&G, OPG, Bruce personnel
• NRC Observers (2)
• CNSC Observed NRC on behalf of IAEAbehalf of IAEA
Conditional Entropy
Audit Resolution
•Audit completed April 2011
3 Fi di f A di•3 Findings from Audit
•Audit findings closed in June 2011
•Kinectrics audit report now available in NUPIC databaseNUPIC database
Reflections/Summary
•Be committed to the industry you seek toindustry you seek to serve
•Use political andUse political and product leverage where possible
•Be prepared for a significant investment gof time and resources
