2013 SFTE/SETP Flight Test Symposium
DOE Used to Investigate
an F/A-18 E/F Strafing
Anomaly
Lt Col Timothy R. Jorris, USAF, PhD
412 OG/Hypersonic CTF
661-277-8924 [email protected]
I n t e g r i t y - S e r v i c e - E x c e l l e n c e
War-Winning Capabilities … On Time, On Cost
Maj Zach McCarley, USMC
Experimental Test Pilot
Naval Air Weapons Station
Approved for public release, distribution is unlimited. 412TW-PA-13416
Introduction
• Gun (Vulcan Cannon)
• Problem
• DOE (theory)
• Test Execution (reality)
• Analysis
• Lessons Learned
F-15 Eagle, F-16, F/A-18, and F-22
Primary Mission
Predictive Gun Sight
11
Problem: Fleet reporting valid attacks
on soft targets with enemy walking
away unscathed.
Should be a hit ...
but NOT!
12
Problem (cont.)
• Rumors of AESA problems
• Lots of anecdotal evidence
• TOPGUN conducted TAC T&E
– 3 mil long
– 2 mils right
• (8 mil gun dispersion)
• Is the software CCIP prediction good or not?
• Is it Super Hornet wide or restricted to AESA?
• What other factors might affect solution?
CCIP – Constantly Computed Impact Point AESA – Active Electronically Scanned Array (Radar)
Is it good or not?
• TOPGUN said no (3 mils / 2 mils)
• What can we offer to the problem?
–Scientific method?
–Disciplined flight test?
–Statistical significance?
• If so, then how do I get results with
statistical significance?
Factors
• Ammo – M56 / M55
– PGU-28 / PGU-27
• Software – Legacy / Assembly Language / X-Build / 25X
– HOL / C++ / H8
• BAAT – Radar (AGR)
– DTED
• Radar – APG-73
– APG-79
16
Example:
250 ft / 200 ft = 1.25
How many runs?
Jet type runs with ammo type
PGU-28 M56 PGU-27 all ammo
25X APG-73 AGR (no Legacy AESA) 5 5 5 15
25X APG-73 AGR (no Legacy AESA) 5 5 5 15
25X APG-73 AGR (no Legacy AESA) 5 5 5 15
25X APG-73 DTED (no Legacy AESA) 5 5 5 15
25X APG-73 DTED (no Legacy AESA) 5 5 5 15
25X APG-73 DTED (no Legacy AESA) 5 5 5 15
H8 APG-73 AGR 5 5 5 15
H8 APG-73 AGR 5 5 5 15
H8 APG-73 AGR 5 5 5 15
H8 APG-73 DTED 5 5 5 15
H8 APG-73 DTED 5 5 5 15
H8 APG-73 DTED 5 5 5 15
H8 AESA AGR 5 5 5 15
H8 AESA AGR 5 5 5 15
H8 AESA AGR 5 5 5 15
H8 AESA DTED 5 5 5 15
H8 AESA DTED 5 5 5 15
H8 AESA DTED 5 5 5 15
total runs 90 90 90
sorties 18 18 18
rounds per sorties 250 250 250
total rounds 4500 4500 4500
Run 1/2 AGR and 1/2 DTED -> 12 runs each way -> stat sig both ways
∑ = 54
sorties?
Config:
A
B
C
One Factor At a Time
• Y = A (ammo) + B (BAAT) + RS + A·B + A·RS + B·RS
RS -Radar/SW
B-B
AA
T
80 Runs
Statapult!
• Statapult = DOE
• Boomer
can explain?
DOE – Design of Experiments (theory)
• Principles
– Randomization
– Replication
– Blocking
Edwards Stats Office
• Need pic of Dr J and crowd
Libi in stats office
Factorial Design
General Factorial Design
with Interactions
Y = A (ammo) + B (BAAT) + RS + A·B + A·RS + B·RS
B-B
AA
T
RS-Radar/SW
54 Sorties 10 Sorties 6 Sorties!
95% Confidence
80% Power
Nuisance Factors
• winds
• temperature
• pressure
• pilot
• aircraft
• cannon
• ammunition lot
• slant range
• dive angle
• airspeed
Measure
Standardize
Test Methodology
• Ammo
• Profile
• Data requirements
– Hits
– Aim Point
• Range
• Bullet Count
• Wind
Ammo
• Combat
–M56 High explosive incendiary (HEI)
–PGU-28 Semi-armor piercing high
explosive incendiary (SAPHEI)
• Training
–M55
–PGU-27
Modified TOPGUN Profile
Data Requirements
• Hits
• Aim Point
IRSSS Description Improved Remote Strafe Scoring System
• IRSSS is an acoustic-based scoring system for supersonic
projectiles
• Typical system consists of: • A microphone array in front of the strafing target(s)
• A downrange equipment rack hardwired to the target microphones
• An outdoor-mounted weather station adjacent to the downrange rack
• A scoring console remote from the range
Mach Angle = sin-1(c / Vp)
Lateral Velocity VL = Vp * tan() = c / cos()
c
Vp
VL
IRSSS Scoring Algorithm
Supersonic Shock Wave Cone
IRSSS Scoring Algorithm
Hyperbolic Line Intersections
0
5
10
15
20
25
30
He
ig
ht A
bo
ve
T
ra
ns
du
ce
rs
(ft)
-20 -15 -10 -5 0 5 10 15 20 Dist. From Center of Transducers (ft)
T1 T2 T3
T1-T2 T2-T3 T1-T3
Projectile Passing Thru Point 2
Point 2
(-12 .5, 12 .99)
Problem – No Target in HUD
• Air Force Solution
• Marine Solution
Fewer hits than expected?
Wind Limits?
Wind limits?
Operational
Significant
Operational
Significant
Analysis
• Instrumentation
• SW Tools
– Measuring hits
– Measuring aim point
– Stats – Design Expert
• Results
Manual Hits
49
-8.5
11
Manual Hits
50
± 0.5
Grabit.m
Also Applicable to HUD Images
Design Expert
53
Design Expert
54
significant
significant
significant
not significant
Miss X vs. BAAT & RS
not significant
RS
Miss Y vs. Long Track Wind
•NAVAIR public release
SPR-2013-303
Op
era
tio
nal S
ign
ific
an
ce
significant
Miss X vs X Track Wind
Op
era
tio
nal
Sig
nif
ican
ce
significant
Wind Effects
•NAVAIR public release
SPR-2013-303
Pointed Traveling
Wind Effects
Action
• Statistically and Operational Significant
• Boeing explained wind compensation
• Modification in Work!
DOE + Statistics = Action
Lessons Learned
• DOE is not just a buzz word
– It is applicable to real world flight test
– Concepts understandable enough for a monkey
– Resources are available to help (keep in touch with stats
instructors)
– Look for Interactions
– Defensible results
• Design of Experiments ≠ Experimental Design
– See the target
– Extracting HUD video data
– Analysis between flights (gun malfunction)
• Statistical Significance may ≠ Operational Significance
Statistical Evidence supports Decisions
Questions?
