Utilizing current test strategies to drive diagnostics development, deployment and

support through software tools

NDIA October 2006

Answer on a student’s Geometry test

So Why Is it So Hard To Understand or Do we Just Make It Hard?

NDIA October 2006

Using The Learning Organization Process to Improve

External/Internal Disturbances

Understanding

Behavior

Data

Physics

Act

Analyze

Measure

Predict

Decide

LearnKnowledge

Results

Object of Interest

TheoryExperiment

NDIA October 2006

So Why Do We Test…….

- To Measure the Health of the Process- To Reduce Risk of Variability

- To Identify ProblemsRisk

Design ManufacturingValidation Support

Verify Design

Ship Product that Works

Warranty Cost

Meet Spec

NDIA October 2006

The Problem

NDIA October 2006

How Much is Really at Risk ?

75%

80%

85%

90%

95%

100%

0.0% 2.0% 4.0% 6.0% 8.0% 10.0% 12.0% 14.0%

Shipped Defect Rate Exposure

Yiel

d at

Tes

t Sta

ge

50% Coverage 60% Coverage 70% Coverage80% Coverage 90% coverage

1.0%

NDIA October 2006

EffectivenessEffectiveness Effectiveness

Faul

t Sp

ectr

um

Typical Test Strategy

Characteristics:Broad coverage at each testHigh level of redundant testLeakage of early defects caught later by testFault coverage is unpredictable

SubAssy

Top Level Assy

Ship

Repair

TestSMT

Repair Repair

Test Test

Faul

t Sp

ectr

um

Faul

t Sp

ectr

um

Effectiveness

Faul

t Sp

ectr

um

NDIA October 2006

Process Test Coverage Redundancy

• Insufficient• Excess• Cold Solder• Marginal Joints• Voids

• Shorts• Open

• Missing• Gross Shorts• Lifted Leads• Bent Leads•Excess•Bridging

•Tombstone•Misalignment

•Polarity, Extra Part,•Non-Elec. parts•Bypass Caps, L’s•Lifted Power/Gnd

• Inverted•Polarity

• Dead Part• Wrong Part• Bad Part• Short/Open on PCB• Functionally Bad

Solder

Correctness

Electrical

AOI

ICT X-Ray

Functional

NDIA October 2006

Functional

5.5%Unique Coverage3.9% (FA)

Correctness

AOI

5.0% Unique Coverage2.8% (FA)

Solder 66.6%Unique Coverage33.5% (FA)

X-Ray

~10% NOT COVERED

Example Design CoverageICT → AXI → AOI → Functional

Electrical12.2%Unique Coverage51.1% (FA)

ICT

NDIA October 2006

ICT

Electrical

0.9% UniqueCoverage2.9% (FA)

Solder X-Ray44.0%Unique Coverage

Correctness AOI

3.2% Unique Coverage

~10% NOT COVERED

Example of CoverageFunctional → AXI → AOI → ICT

Functional41.2% Unique Coverage

NDIA October 2006

What is Needed

NDIA October 2006

Coverage Mapping and What We Know

Process Test Coverage (Static) -Functional Coverage (Dynamic) -

A

B

C

Input

Fail

Pass

Pass

X +/- %

Y +/- %

Z +/- %

Output

C1

T1

IC3HY1

IC1 T1

C1

IC2

NDIA October 2006

X +/- %

Y +/- %

Z +/- %

Output

C1

T1 IC2

IC3HY1

IC1 T1

C1

Process Test Coverage (Static) -Functional Coverage (Dynamic) -

A

B

C

Input

Fail

Pass

Pass

Coverage Mapping and What We May Know

??

?? ??

??

NDIA October 2006

Process Test Coverage (Static) -Functional Coverage (Dynamic) -

A

B

C

Input

X +/- %

Y +/- %

Z +/- %

OutputT1 IC2IC1

IC3HY1

Fail

Pass

Pass

80%

20%

Coverage Mapping and What We Want to Know

C1

Possible NTF

NDIA October 2006

The Answer

NDIA October 2006

Desired Production Test Process

Characteristics:Test in right placeMinimize redundant testEffectively catch defects at originFault coverage is predictable

SubAssy

Top Level Assy

Ship

Repair

TestSMT

Repair Repair

Test Test

EffectivenessEffectiveness

Effectiveness

Faul

t Sp

ectr

um

Faul

t Sp

ectr

um

Faul

t Sp

ectr

um

Effectiveness

Faul

t Sp

ectr

um

NDIA October 2006

What is Needed For Effective Test Strategies•A common Framework for describing Test Strategies.•Standardized documentation for test.•Metrics to make test tradeoffs between test strategies (AXI, AOI, ICT, Functional, etc) for gaps and overlaps.•Ability to simulate a Test Strategy as the product is being developed to feedback DFX input.•A consistent, repeatable process throughout the product life cycle.

What is not needed is…….•An automated way to develop test plans•A tool to eliminate the need for test developers

NDIA October 2006

HIGH98%HIGH98%HIGH98%HIGH98%HIGH98%4Analog LIBMED90%MED90%MED90%MED90%HIGH90%4Other LIBHIGH100%HIGH100%HIGH100%HIGH100%HIGH100%28Other LIBLOW82%LOW82%LOW82%POOR81%HIGH81%6Other LIBLOW88%MED91%LOW88%LOW88%HIGH88%5Analog LIBHIGH98%HIGH98%HIGH98%HIGH98%HIGH98%24Other LIBHIGH98%HIGH98%HIGH98%HIGH98%HIGH98%6Mixed LIBHIGH98%HIGH98%HIGH98%HIGH98%HIGH98%6Digital LIBHIGH100%HIGH100%HIGH100%HIGH100%HIGH100%4Other LIBHIGH98%HIGH98%HIGH98%HIGH98%HIGH98%28Other LIBMED95%MED95%MED95%MED95%HIGH95%4Other LIBLOW83%LOW83%LOW83%LOW83%HIGH83%6Mixed LIBLOW82%LOW82%LOW82%POOR76%LOW38%80Other LIBHIGH95%HIGH95%LOW82%POOR76%NO0%6Digital LIBHIGH95%HIGH95%LOW82%POOR76%NO0%6Digital LIBLOW82%HIGH95%LOW82%POOR76%NO0%6Digital LIBHIGH95%HIGH95%LOW83%LOW83%HIGH83%8Digital LIBMED95%MED95%MED95%MED95%HIGH95%24Other LIBHIGH100%HIGH100%LOW86%LOW86%HIGH86%48Digital LIBHIGH98%HIGH98%HIGH98%HIGH98%HIGH98%6Other LIBMED95%MED95%MED95%MED95%HIGH95%28Mixed LIBLOW83%LOW82%LOW83%POOR79%NO0%1Single PinLOW82%LOW82%LOW82%POOR76%NO0%5Other LIBLOW82%LOW82%LOW82%POOR76%NO0%2ConnectorHIGH100%HIGH100%HIGH100%HIGH100%HIGH100%10Other LIBMED93%MED93%MED93%MED93%HIGH93%6Analog LIBMED95%MED95%MED95%MED95%HIGH95%5Analog LIBHIGH100%HIGH100%HIGH100%HIGH100%HIGH100%2ResistorHIGH98%HIGH98%HIGH98%HIGH98%HIGH98%2ResistorLOW83%MED94%LOW83%POOR79%NO0%2ResistorLOW83%MED94%LOW83%POOR79%NO0%2ResistorMED95%MED95%MED95%MED95%HIGH95%2ResistorMED94%MED94%LOW83%POOR79%NO0%2Resistor

GradeScoreGradeScoreGradeScoreGradeScoreGradeScorePinsType

FT+AXI+AOI+ICTCurrent Access

FT+AXI+AOI+ICTFull Access

FT + AXI + AOIFT + AXIFunctionalCoverage

Component

Sample of Product Coverage Mapping

NDIA October 2006

Test Optimization Software: Knowledge Transfer

NDIA October 2006

•Multiple design teams, multiple locations, using common design rules, common design tools for uniform test strategies.

•Transfer product designs, processes, test and repair knowledge in a predictable, repeatable manner to anyone anywhere.

•Model, simulate and predict test performance, quality, and cost drivers.

•Deliver manufacturing specifications without flying engineers everywhere.

•Have a common language for communicating product/process functionality.

•Consistent repair and diagnostic process with no variability, any time, any where and by anybody.

Elements for a true Product Life Cycle Test Strategy

NDIA October 2006

Example of Change

NDIA October 2006

Using The Learning Organization Process to Improve

Act

Decide

Knowledge

External/Internal Disturbances

Learn

Effectiveness

Faul

t Sp

ectr

um

Effectiveness

Faul

t Spe

ctru

m

HIGH98%HIGH98%HIGH100%HIGH100%LOW82%POOR76%LOW82%POOR76%MED90%MED90%LOW82%POOR76%LOW82%POOR76%

Measure

Analyze

Understanding

Predict

X-Ray ICT

AOI Func

Object of Interest

FailPass

Pass

?? ??

????

Behavior

Results

80%

20%

FailPass

Pass

NDIA October 2006

Original Manufacturing Flow

Yield= 67% Yield= 86% Yield= 96%Yield= 0% Yield= 78%

Pristine Yield= 43%

DIAGNOSE

REPAIR

RE-T

EST

6Days

DIAGNOSE

REPAIR

RE-T

EST

30Mins

DIAGNOSE

REPAIR

RE-T

EST

1Day

DIAGNOSE

REPAIR

RE-T

EST

4Days

DIAGNOSE

REPAIR

RE-T

EST

2Days

DT

DT

Thru-HoleSMT MDAI I

SRU Assembly

Cells

SRU Cell Test

LRUAssembly

Cell

LRU CellTest

Final System Assembly and Configuration

Final Assembly

System TestShipESS

LRU CellTest

Repair Diagnostics

Repair Diagnostics Repair Diagnostics Repair DiagnosticsRepair Diagnostics

RepairRepair

Pristine Yield= 0%

0% 20% 40% 60% 80% 100%

Solder/Workmanship

Wrong/Bad Parts

Placement

Other

LRU

SRU

Distribution of Defects by Symptom

Complexity and Variability at Every Step

NDIA October 2006

Redesigned Manufacturing Flow

Thru-HoleSMT MDA

SRU Assembly

Cells

SRU Cell Test

LRUAssembly

Cell

Final System Assembly and Configuration

Final Assembly

System TestShipESS

LRU CellTest

Repair DiagnosticsRepair Diagnostics

DT

XRAY

Repair Repair Diagnostics

DIAGNOSE

REPAIR

RE-T

EST

15Mins

DIAGNOSE

REPAIR

RE-T

EST

30Mins

DIAGNOSE

REPAIR

RE-T

EST

60Mins

DIAGNOSE

REPAIR

RE-T

EST

60Mins

Yield= 95% Yield= 98% Yield= 98%Yield= 80% Yield= 98%

Pristine Yield= 72%

NDIA October 2006

What is the impact of Efficient Test and Effective Diagnostics?

Increase in quality and reduction in diagnostic and repair of products

$.67 Million

Increase productivity of production lines due to increased test capacity (20% * 200 units/yr * $20K/unit)

$.8 Million

Savings in elimination of new tests and testers at SRU (at $500K) and LRU (at $1,000K) test cells* (($500K * 4 + $1,000 * 4) * 25%)Reduction in total cycle time due to increased quality and re-alignment of processes ( 5days * $240day * 960 units/year )

$1.15 Million

$1.50 Million

5 year ROI

Financial Impact of Redesign Flow

$2.62 Million First Year

$16.3 Million

NDIA October 2006

Conclusion

NDIA October 2006

Leverage Points in the Product Life Cycle

80-90% of manufacturing cost determined before QUAL90% of achievable first pass yield determined by QUAL80% of potential profitability fixed at DEV80% of MTBR determined before DEV

DevelopmentConcept DefinitionInvestigation FirstProduction

Shipment SupportLifeHW Qualification

STRTSTRT DEFDEF DEVDEV QUALQUAL SHPSHP SUPSUP CLOCLO

SW Qualification

Current tools will allow you to leverage these advantages by:Eliminating Complexity and VariabilityEnsuring Efficient and Effective Test Strategies before releaseDeveloping Automated Diagnostics with little additional

resources during developmentUsing Test and Diagnostics as part of a learning organization

throughout the total product life cycle to leverage Results!!

NDIA October 2006

Questions???