PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org www.build4scale.org
SustainingQualityandWarrantyModule7BPreparingfortheProduc1onRamp
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
Mo2va2on
2
o Scalingac1vi1escanaffectproductqualityandreliability,butquickiden1fica1onofproblemskeepscorrec1veac1oncostslow
o Scalingtheprocessfrombetaprototypebuildsintoactualdeliveredpilotandproduc1onbuildswiththefollowinggoals:— Catchproductreliabilityproblemsquicklyandearlyonintheproduc1onprocess
— Ensurethatwarrantyandservicestrategiesareappropriateandcosteffec1ve
— Priori1zerisk-mi1ga1oneffortsbasedonimpactstototalcostonthebillofmaterials,customersa1sfac1on,and1metomarket
Whyisthismoduleimportant?
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
3
PilotAndScalingWheredoesthisfitintothedevelopmentcycle?
Conceptand
FeasibilityDefini2on
ProductandProcess
Design
Implementand
Validate
Produc2onandStock
Launchand
Closure
Pre-alpha Alpha Beta Pilot Ramp Scale
PHAS
ES
BUILDS
EngineeringValida9on
DesignValida9on
ProcessValida-on
ConceptValida9on
PRODUCTDESIGNOBJECTIVESMarketResearch
DesignResearchFeasibility
DevelopmentQualifica2on
FieldReadinessLaunch
ManufacturingCapacity
MANUFACTURINGREADINESSLEVELSLevels: 109987654321
ManufacturingDevelopmentManufacturingResearch
Mfg.Capability
Produc2on
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
ModuleOutline
4
o Learningobjec1veso Reliabilityoverview— Keydriversandlifecyclereview— Scale-upobjec1ves
o Produc1onscale-upandriskmi1ga1ono Transi1onto“screening”testmethodso Ini1a1onofdatamonitoringandfeedbackloopprocedures
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
LearningObjec2ves
5
o LO1.Understandac1vi1esandop1onsinvolvedwithscale-upforsustainingquality,warranty,service,andrepair
o LO2.Understandhowtousefailuremodeandeffectsanalysis(FMEA)outputstopriori1zeac1vi1esanddecisions
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
6
ReliabilityOverviewKeydriversandlifecyclereview
ConceptandFeasibility Defini2on
ProductandProcessDesign
ImplementandValidate
Produc2onandStock
LaunchandClosure
EstablishPreliminaryReliabilityPlans
PHAS
ES
RELIAB
ILITY Setthe
ReliabilityGoalsfortheProduct(s)
DesignandImplement
theReliabilityCasefortheProduct(s)to
Support
VerifytheProductReliabilityfromIni2alProductBuilds.
RefinetheReliability
Case
ValidateProductReliabilityfromIni2alProduc2on
ValidateProductReliability
fromProduc2on
andCustomerFeedback
o Reliabilityandqualityac1vi1esshouldoccuratallstagesoftheproductlifecycle
o Reliabilitypredic1onsandFMEAoutputsdrivepriori1za1onofac1vi1es:— E.g.,impactson1metomarket,totalcost,andcustomersa1sfac1on
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
ReliabilityOverview
7
Reliabilitytes2ngobjec2vesbybuildversion:o Pre-alphates2ng:establishreliabilitygoalso Alphates2ng:testandrefinementofproductdesigno Betates2ng:testandrefinementofmanufacturingprocesso Pilottes2ng:validatedesignandmanufacturingpriortoproduc1on
o Produc2onramp:validatewarrantyandserviceimpacts
Scale-upbuildobjec9ves
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
Produc2onScale-Up
8
Ensurethatoverallprogramimplementa2onmeetsgoalsfor2metomarket,cost,andcustomersa2sfac2onwithunifica2onacrossthefollowingteams:o Marke1ng/saleso Quality/reliabilityo Productdesigno Newproductintroduc1on(NPI)o Manufacturingo Serviceandrepairo Technicalpublica1onso Supplychain
Cross-func9onalteamengagement
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
Produc2onScale-Up
9
Betatopilottoramptransi9ons
Tip:NPIandotherteamsmayneedtostayinvolvedinthepilotandproduc1onrampto
maintainaccountabilitytothesegoals
o Usereliabilitygoalsandconfidencemetricsasgates
Decisiontradeoffswhengoalsarenotmet:o Costtoachievehigherreliabilityorconfidence
o Costofdelayedlauncho Costofwarrantyandserviceimplica1ons
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
Produc2onScale-Up
10
o Transi1onfrombeta“usesimula1on”tofinalproduct“screening”testmethods:— Allpilotbuildsandbeyondareintendedasdeliverableproducts
o Ini1ateappropriatetest,datacollec1on,andfeedbackloopprocedures
o Iden1fypilotcustomersanddevelopafeedbackplano Setuseagreements,warranty,andserviceplansinmo1ono Releasesupportdocumenta1on
Produc9onrampac9vi9es
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
Produc2onScale-Up
11
KeyStepstoRiskManagement:o Iden1fyo Analyzeo Controlo Transfero Reduceo Assess
Ac9vitypriori9za9onandriskmi9ga9on
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
StepsforEffectAnalysis:o FMEAoutputsbecomeinputsforqualityandwarrantyscale-updecisions
3-DimensionalViewofRisks:o Detectability,Do Probability,Po Severity,So Setascoreof1–3foreachdimension
12
Ac9vitypriori9za9onandriskmi9ga9on(cont.)Produc2onScale-Up
Tip:assigningappropriateFMEAriskscoresof1–3forD,P,andSandmul1plyingthemtogetheriden1fiesthehighest-risk,highest-valueitemsforpriori1za1on
(Severity)
(Probability)
(Detectability)
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
FailureModeAndEffectsAnalysis
13
Mi9ga9onac9vi9esshouldnotaffectsubsequentriskassessmentscoresunlesssignificantrisksgetlostinrepriori9za9on
Example
Risk D P S Risk Mi2ga2onAc2vity
UselifeforcomponentXislessthantherequiredproductuselife 2 3 2 12
Planforservicestrategytoreplacepartifmorecosteffec1vethanincreasingcomponent
performance
Productdown1mehassignificantimpactoncustomeropera1ons 3 2 1 6
Planforfast-respondingrepairserviceorachievesufficientreliabilityconfidencethroughaddi1onal
testanddatacollec1on
Highinfantmortalityratewilldamagereputa1onandsales 3 1 3 9 Testreliabilitypredic1onsduringscale-upwith
feedbacklooptocorrectimportantrootcauses
Detectability(D)xProbability(P)xSeverity(S)=Risk
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
14
o Inpreviousbetates1ngstages,usesimula1on(i.e.,acceleratedlifetes2ng[ALT])isconductedwithappropriateseveritytofailtheproductandiden1fypoten1alproblems(adesignqualityexercise)
o TheknowledgegainedfromALTcanbeusedtoreducetheseverityoffailurecondi1onsfordevelopingappropriateproduc1on“screening”tests(amanufacturingqualityexercise)
Reminder:ALTisates1ngstrategywherebytheengineerextrapolatesaproduct’sfailurebehavioratnormalcondi1onsfromlifedataobtainedatacceleratedstresslevels.Sinceproductsfailmorequicklyathigherstresslevels,thisstrategyallowstheengineertoobtainreliabilityinforma1onaboutaproductinashorter1meframe(i.e.,meanlifeexpectancyandprobabilityoffailureataspecific1me)
“Screening”testmethodsTransi2ontoProduc2on
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
StressScreening
15
o Highlyacceleratedstressscreening(HASS)usesacceleratedstressesbeyondaproduct’sspecifica1onstodetectproductdefectsduringmanufacturingandproduc1on
o Non-destruc2veinspec2onsincludeoutofboxaudit(OBA),func1onaltes1ng,x-ray,andvisualmethods
o Destruc2vetestexamplesincludeongoingreliabilitytests(ORT),teardown,andbreakopen
Anyscreeningtestimplementa9onshouldaddvaluetotheproductbasedonpriorrisk-mi9ga9onorcost-mi9ga9ondecisions.Otherdesignfactorsorservicingplansmightbe
alterna9vestoscreentes9ng.
Highlyacceleratedstressscreening
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
DataCollec2onandFeedbackLoops
16
Theprimaryobjec2veisensureearlyfailureratesmatchreliabilitypredic2onsandgoals;availabletoolsincludethefollowing:o HASStes1ng,OBAs,andproduc1onlineinspec1ons:— Lastprocessinproduc1onflowbeforeshippingaproducttoensurethatallmanufacturingandtestprocesseshavebeencompleted
— Dataaboutthemanufacturedproductacquiredpriortodelivery
— Couldbeconductedasahigh-percentagecheckearlyinproduc1onandalow-percentagechecklaterinproduc1onramp
Ini9a9onandprocedures
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
DataCollec2onandFeedbackLoops
17
o ORTs— Periodiccheckpointsanddirec1onresets— Predefinednumberofproductsinthemanufacturinglinearerandomlypickedeachdayofproduc1onandsubjectedtoenvironmentalstressestoinducedamage
o Cumula2vedistribu2onfunc2onanalysis— Actualfielddataforpredic1ngunexpectedfailuretrends
o Rootcauseanalysis(detailedonnextpage)— Analysisforunderstandinganddeterminingcorrec1veac1on
Ini9a9onandprocedures(cont.)
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
o Rootcauseanalysisisusedtoiden1fytherootcausesoffaultsorproblems:— e.g.,asking‘why’five1mes
o Afactorisconsideredarootcauseifitsremovalfromtheproblem-faultsequencepreventsthefaultfromrecurring
o Acausalfactorisonethataffectsanevent'soutcome,butitisnotarootcause.Removingacausalfactorcanbenefitanoutcome,butitdoesnotpreventitsrecurrencewithcertainty
18
RootCauseAnalysis
Allowsforproperdetermina9onoffailure
source,impacttoreliability,andcorrec9veac9onop9ons
PreparingfortheProduc1onRampU.S. DEPARTMENT OF ENERGY • OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY www.build4scale.org
PilotCustomers
19
o Considernon-disclosureagreementsifproduct-launchschedulewillbeprolongedtoavoidprematurereleaseofsensi1veinforma1ontothepublic
o Encouragethecustomertoprovidefeedbacko Definearangeofcustomerusecases(e.g.,industryorgloballoca1on)togeneratethemostusefulfeedback
o Seekcustomerswhoarewillingtohostonsiteinstalla1onandusageac1vi1es
o Considerdiscountedpricingforearlyadop1onandaccesstodata
Bestprac9ces
Glossary
Module 7B
Ongoing Reliability test (ORT) is a hardware test process usually used in manufacturing to
ensure that quality of the products is still of the same specifications as the day it first went to
production or general availability.
Process Validation is the analysis of data gathered throughout the design and manufacturing of
a product in order to confirm that the process can reliably output products of a determined
standard.
Bill of Materials (BOM) is a list of the raw materials, sub-assemblies, intermediate assemblies,
sub-components, parts and the quantities of each needed to manufacture an end product.
Failure Mode and Effects Analysis (FMEA) was one of the first highly structured, systematic
techniques for failure analysis. It was developed by reliability engineers in the late 1950s to
study problems that might arise from malfunctions of military systems. An FMEA is often the
first step of a system reliability study.
Pre-alpha Testing refers to testing associated with concept validation. This validation could be
customer discovery based of could require documented evidence that establishes a high degree
of certainty that a particular product or process will consistently meet established criteria for
concept success and reliability goals.
Alpha Testing is in-house testing of a pre-production model or version, to locate or estimate
design flaws or deficiencies.
Beta Testing is second level, external pilot-test of a product (usually a software) before
commercial quantity production. At the beta test stage, the product has already passed through
the first-level, internal pilot-test (alpha test) and glaring defects have been removed.
Pilot Testing is a small-scale campaign, survey, or test-plant commissioned or initiated to check
the conditions and operational details before full scale launch.
Production Ramp-Up is the start or increase in production ahead of anticipated increases in
product demand and also in an effort to confirm all production assumptions.
Detectability is the ability to detect a failure before it causes harm. The purpose of
considering detection in any scenario is to ensure that potential or actual failures can be
identified with enough time to take action before the user is adversely affected.
Probability is the likelihood of the hazard occurring and is ranked on a point scale.
Severity is the amount of damage or harm a hazard could create and is usually ranked on point
scale.
Accelerated Life Testing (ALT) is the process of testing a product by subjecting it to conditions
(stress, strain, temperatures, voltage, vibration rate, pressure etc.) in excess of its normal
service parameters in an effort to uncover faults and potential modes of failure in a short
amount of time
Highly Accelerated Stress Screening (HASS) uses accelerated stresses beyond a product’s
specifications to detect product defects during manufacturing and production
Non-Destructive Inspections or non-destructive testing (NDT) is a wide group of analysis
techniques used to evaluate the properties of a material, component or system without causing
damage.
Destructive Tests on-going reliability tests (ORT), tear down, break-open, etc. the product to
test for its compliance to specification.
Root Cause Analysis is a method of problem solving used for identifying the root causes of
faults or problems
Cumulative Distribution Function (CDF) is the cumulative distribution function (CDF) of a real-
valued random variable X, or just distribution function of X, evaluated at x, is the probability
that X will take a value less than or equal to x.
Root Cause is an initiating cause of either a condition or a causal chain that leads to an outcome
or effect of interest.
Causal Factor is those elements that lead to variation in the quality of the finished product. The
Events and Causal Factors (ECF) chart depicts the necessary and sufficient events and causal
factors for accident occurrence.
Non-Disclosure Agreements - is a legal contract between at least two parties that outlines
confidential material, knowledge, or information that the parties wish to share with one
another for certain purposes, but wish to restrict access to or by third parties.