Leveraging Designed Experiments for Success · Leveraging Designed Experiments for Success #1: Get...

Leveraging Designed

Experiments for Success

Scott C. Sterbenz, P.E.

Six Sigma Master Black Belt, Ford Motor Company

Technical Advisor, United States Bowling Congress

Presentation Outline

I. Introduction

A. Ford Motor Company – The One Ford Plan

B. United States Bowling Congress – Governance of the Sport of Bowling

II. Lessons Learned in Successful DOE Application

A. Get Creative with Your Response:

Premature Bulb Failures – Ford Motor Company

B. The Response Needn’t be Continuous Data:

Carpet Quality – Ford Motor Company

C. Interactions Matter:

Carpet Quality – Ford Motor Company

D. Expand the Range of Your Factors:

Static Weight Study – United States Bowling Congress

E. Don’t Forget About Center Points:

Static Weight Study – United States Bowling Congress

III. Questions & Discussion

2

Ford Motor Company

The One Ford Plan:

• Lays a foundation for business success

• Focuses on working together to achieve profitable growth for all

• Facilitates leadership in four pillars for customer satisfaction and value

3

The United States Bowling Congress:

• Vision

Create lifelong bowlers

• Mission

Provide benefits and programs to enhance the bowling experience

Equipment Specifications and Certification Department:

• Vision

Uphold the credibility of bowling

Leading source of technical information

• Mission

Bring science, technology and bowling together

Solve problems, answer questions, and implement specifications

Expert technical services and sound statistical analyses

4

United States Bowling Congress

Leveraging Designed Experiments for Success

#1: Get Creative With Your Response

5

Get Creative With Your Response

Training materials for designed experiments teach that the response should be:

• The KPOV of the process

• Directly related to the customer CTQ

• Selected from the C&E matrix, fishbone diagram, or y=f(x) cascade

Generally, these guidelines are true, but sometimes yield a nonmeaningfulmeasure.

6


Practical Problem: Premature Bulb Failures

Background:

• Warranty costs in 2005 were $2.7M, and increasing every model year

• Single highest warranty cost in Ford Motor Company

• Disagreement between vehicle subsystems about root cause

Over-voltage

Vehicle vibration

Supplier quality

7



Plan:

• 25 full factorial DOE:

Vibration input

Voltage input

Bulb supplier

Filament orientation angle

Filament centering

• Bench test:

Typical customer usage cycle

Twenty bulbs (replicates)

8

What should the response be?• Average time to failure

• Variance in time to failure

• Signal-to-noise ratio

Is there something better?



Selected Responses:• Reliability shape• Reliability scale (B63 Life)

Analysis:

9

Reliability Plots Constructed for

Main Effects and Interactions

Probability Plot of Effects to

Illustrate Significance



Results:• Cumulative savings since implementation (2008MY) = $5.6M• Design rules for voltage regulation at incandescent lamps

10

Implementation


#2: The Response Needn’t Be Continuous Data

11

The Response Needn’t Be Continuous Data

Training materials for designed experiments teach that the response must be continuous data—not attribute data.

This guideline is true, but sometimes measurements are not possible to be continuous.

12

Ordinal or Nominal Data

(NO)

Ratio or Interval Data

(YES)

Practical Problem: Fiesta Carpet Quality

Background:

• Critical vehicle launch for Ford Motor Company

Largest threat to a quality launch

Anticipated customer satisfaction concerns

• Ford and supplier at odds

Competing responses—brush marking and softness

Cost versus quality

Promises versus deliverables

13


Brush Marks


Plan:

• 26-1 fractional factorial DOE:

Six factors

Two center points

Two replicates

• Evaluations:

Five evaluators

Brush marking and softness

Likert scale

14


Individual collected responses are attribute—What can be done?

Leverage replicates / multiple evaluators

• Transforms ordinal Likert scale

Increases resolution from units digit to tenths digit

Mimics continuous data


Selected Responses:• Average softness rating• Average brush marking rating

Analysis:

15

Abbreviated DOE Matrix Shows

Transformation of Likert Scale (Ordinal Data)


Pareto Chart of Effects Illustrates

Standard DOE Analysis


#3: Interactions Matter

16

Interactions Matter

Training materials for designed experiments teach that three-way and higher interactions are rare (Sparsity of Effects Principle).

Generally, this is correct. However, there are some cases where three-way interactions are not only present, but also very strong:

• Complex manufacturing processes

• Chemistry

• Psychology

17


Selected Responses:• Average softness rating• Average brush marking rating

Analysis:

18

Pareto Chart Shows Significant and

Strong Three-Way Interactions

Interactions Matter

Minitab “Display Available Designs”

Details Resolution of Designs


Results:

• Full extent of interactions understood

Fosters technical excellence

Replication of knowledge

• Multi-Response Optimization

Softness and brush marking

19

Interactions Matter

Minitab Optimization Plot Illustrates

Balance of Multiple ResponsesElimination of Brush Marking;

Softness Better Than Baseline


#4: Expand the Range of Your Factors

20

Expand the Range of Your Factors

Training materials for designed experiments teach that factor levels should:

• Be wide enough to create a desired change in the response

• Go beyond typical limits in the process

• Not create unsafe or impossible conditions

This guideline is absolutely correct:

• Don’t be afraid to make bad parts

• Challenge the limits of the tools

21

Practical Problem: Static Weight Study

Background:

• 2007 study determined factors that affect ball motion on a lane

High Influence – coverstock

Moderate Influence - core

Low Influence – static weights

22



Background:

• 2007 study evaluated static weights within current specifications

• Bowling ball manufacturers requested removal of specification

• USBC concerned static weights were influential outside current specifications

23


Definition of Static Weights

Finger

Thumb

Positive

Side

Negative

Side

Top (Grip Side)

Bottom (Not Visible)

(±1 oz.)

(±1 oz.)

(±3 oz.)


Plan:

• 26-1 fractional factorial DOE:

Six factors

Three static weights

Core shape (intermediate diff.)

Ball speed

Rate of revolution

Eight center points

24


How wide should the levels be?

USBC investigating completely removing static weight specification:

• Static weights set at maximum possible values

• Core shape is significant to ball motion

• Ball speeds and rates of revolution cover all bowling styles


Selected Responses & Analysis:

• 19 measures characterize ball motion

Collected from CATS (computer-aided tracking system)

Transition points between phases

Lengths of the phases

Shape of the phases

25


23 Lane Sensors

Track Ball Motion

Regression Techniques Used to

Characterize Ball Motion Mathematically



Results:

• Anomalies discovered before the DOE was analyzed

Residuals analysis from regression

Undesirable 4th phase of ball motion discovered

26

Initial Results from Ball Motion Algorithm

Residuals Analysis in Roll Phase

Shows Missed Quadratic Term Correction Shows 4th Phase of Ball Motion



Results:• 4th phase is unpredictable

Unfair advantage Athlete dissatisfaction

27

Angle into Pins DiminishedAngle into Pins Augmented



Results:• 4th phase is unpredictable

Unfair advantage Athlete dissatisfaction

28

Normal / Expected Ball Motion 4th Phase / Unexpected Ball Motion


#5: Don’t Forget About Center Points

29

Don’t Forget About Center Points

Training materials for designed experiments teach to include center points:

• Increases power of the experiment

• Helps eliminate saturation

• Evaluates linearity of the response

This guideline is absolutely correct:

• Can lead to use of Response Surface design

• More accurate modeling

30


Analysis:• DOE analyzed without responses affected by 4th phase• Curvature was significant in 18 of 19 responses

31


Typical Result Indicating

Significance of Curvature

Analysis of Variance for A-Score (coded units)

Source DF Seq SS Adj SS Adj MS F P

Main Effects 6 0.0099790 0.00997901 0.00166317 2544.74 0.000

2-Way Interactions 15 0.0005627 0.00056270 0.00003751 57.40 0.000

3-Way Interactions 10 0.0000638 0.00006383 0.00000638 9.77 0.003

Curvature 1 0.0002570 0.00025705 0.00025705 393.30 0.000

Residual Error 7 0.0000046 0.00000458 0.00000065

Pure Error 7 0.0000046 0.00000457 0.00000065

Total 39 0.0108672



Analysis:

• Central Composite Design

Reduced static weight levels:

o Attempt elimination of 4th phase

o Widen specification, not elimination

32



Results:

• Non-linear effects confirmed

• 4th phase of ball motion still present; direction and occurrence not predictable

• Effects of static weights within current specifications insignificant

• Static weight specification not changed

33

Contour Plot—Static Weights Within Specs Are InsignificantPareto of Effects—Order of Effects and Non-Linearity

Presentation Summary34

Topic Lessons Learned

Get Creative with Your Response 1. Think beyond mean and standard deviation

The Response Needn’t Be Continuous Data1. Leverage replicates

2. Convert attribute data

Interactions Matter1. Select proper design resolution

2. Improve process optimization

Expand the Range of Your Factors1. Discover what happens outside the typical

inference space


1. Check linearity assumptions

2. Achieve greater knowledge with response

surface methods


Questions & Discussion

35

Date post:	24-Jul-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

Leveraging Designed Experiments for Success · Leveraging Designed Experiments for Success #1: Get...

Documents