THEN AND NOW
Dr. Paul E Garrison
Systems Engineer Principal
THE ARCHITECTURE OF THE F-35 LIGHTNING II MISSION SYSTEM
INTEGRATION LAB
© 2019 Lockheed Martin Corporation. All Rights Reserved
BIOGRAPHY SLIDE
Dr. Paul E. GarrisonSystems Engineer PrincipalF-35 Lightning II Mission Systems Integration Labs
Lockheed Martin AeronauticsEmail: [email protected]: 817-763-1494
• Key Programs• Air Force Electronic Warfare Evaluation Simulator• Classified IRAD and CRAD programs• F-22 Raptor Laboratory Simulation• F-22 Raptor Avionics• F-35 Lightning II Mission Systems Integration Laboratory
• PhD in Electrical Engineering• Over 40 years in Hardware in the Loop labs• F-35 HITL Lab Chief Engineer
© 2019 Lockheed Martin Corporation. All Rights Reserved
F-35 MISSION SYSTEMS INTEGRATION LABORATORIES
INTEGRATION LAB CAPABILITY EVOLUTION
• F-35 Variants
• F-35 Avionics Architecture• F-35 Hardware in the Loop (HITL) Laboratories
• Vehicle Systems• Mission Systems
• Stimulation System Integration Station• System integration and test process – then and now
• T&E Evolution• One big lesson learned!• Multi-ship Virtual development Lab (MVDL)
• Success Stories• Lessons confirmed
• Lessons Learned• The future
© 2019 Lockheed Martin Corporation. All Rights Reserved
THE F-35 IS PRODUCED IN THREE VARIANTS – COMMON AVIONICS
STOVL
CV CTOL
Lift FanRoll Nozzle
3-BearingSwivel Duct
© 2019 Lockheed Martin Corporation. All Rights Reserved
By mid-2019: over 370 aircraft, over 200,000 flight hours, more than 840 pilotsand 8,040 maintainers operating from 19 bases around the world
CV – Carrier VariantCTOL – Conventional Takeoff and LandingSTOVL – Short Takeoff/Vertical Landing
SITUATIONAL AWARENESS IS ACCOMPLISHED THROUGH SENSOR DATA FUSION
© 2019 Lockheed Martin Corporation. All Rights Reserved
RADAR
EW
CNI
EOTS
DAS
Image Proc.
Signal Proc.
Fusion Proc.
FireControl& Stores
External Comm.
Data Collect.
Prognostic Health Mgmt.
Mission
Pilot Vehicle
Interface
Integrated Core Processor (ICP)
Sens
or A
pert
ures
Sensors Vehicle SystemsVehicle Mgmt.
Computer A
Vehicle Mgmt.
Computer B
Vehicle Mgmt.
Computer C
VMS
Subs
yste
ms
VMS
Subs
yste
ms
VMS
Subs
yste
ms
Mission Systems HMD
Reports, Tracks, Images
Reports, Tracks, Images
F-35 Lightning II Avionics Architecture
EW – Electronic Warfare, CNI, Communication, Navigation and IFF, DAS – Distributed Aperture System, EOTS – Electro-Optical Targeting System
F-35 LAB PRODUCT DEVELOPMENT “V”
© 2019 Lockheed Martin Corporation. All Rights Reserved
System Definition System andSubsystem Design Implementation Subsystem
TestingSystem Integration
and Test
Detailed Design
Software Development
Software Test
Operational ConceptDevelopment
System Requirements
Preliminary Design
Detailed Design Unit Test
Integrated System I&T
Verification Testing
Desktop Based Simulation (DBS)
Vehicle System Processing/Flight Control
System Integration Facility (VIF)
Ownship Sensors (OSS)Fire Control & Stores
Flight Test
CSCI Integration
Reprogramming Labs
88% of Anomalies are found prior to flight test
F-35 VEHICLE SYSTEMS INTEGRATION LABS
© 2019 Lockheed Martin Corporation. All Rights Reserved
VIF-Fixed Base (FB) Simulator• Purpose: Flight Control Systems
Integration and Test• Retired after SDD program
VIF-Motion Base (MB) Simulator• Purpose: Flight Control Systems,
Handling Qualities and Control Law evaluation Integration and Test
VSIF• Purpose: Integration of Vehicle
Systems: Environmental Control System, Propulsion, Electrical Power, Hydraulics, Failure Modes & Effects Test, VS SW test and integration
Vehicle Systems Labs are Transitioning to a Sustainment role
F-35 MISSION SYSTEMS INTEGRATION LABS
© 2019 Lockheed Martin Corporation. All Rights Reserved
Desktop Based Simulation (DBS)Description: PC-based Software Development StationOperational Flight Program (OFP) Development
Display Development Station (DDS) for Display Software Development
Open-Air System Integration Station (OASIS)Description: All mission systems avionics operating through aircraft apertures, Cockpit controls and displays in lab area.Installed end-to-end system integration and testing.Pole Model retired; Open-Air test capability retained
Simulation System Integration Station (SimSIS)Description: Aircraft ICP integrated with simulated sensorsSupports System Builds and software integration
SimSIS+ adds VS and MS hardware for air-worthiness testing
Stimulation System Integration Station (StimSIS)Description: All Mission Systems Avionics minus the sensor apertures, ICP, Cockpit Controls and DisplaysSystem Integration and Test; Sensor fusion
Sensor Stimulators
RADARCNI
DAS
EW
F-35 STIMULATION SYSTEM INTEGRATION STATION - STIMSIS
© 2019 Lockheed Martin Corporation. All Rights Reserved
Sensors are stimulated at the fidelity required for sensor data fusion testing.
T&E EVOLUTION
SI&T EVOLVED TO EMPHASIZE SHIFT-LEFT
TESTING
• What was retired:• Air System Integration Facility (ASIF) – Limited use of
integrated MS & VS labs due to security constraints• Vehicle Systems Fixed Base Simulator – Capability
retained in the Motion-Based Simulator• Sensor Integration Platform (OASIS Pole Model) –
Open air capability retained using roof-top apertures• Cooperative Avionics Test Bed (CATB) – retired.
• What was re-purposed: Simulated System Integration Stations (SimSIS)
• Air-Vehicle-Air Worthiness Labs: AVOIDS & Sim+• Partner Weapons integration lab• Filled the gap created by the limited utility of the ASIF
• What was added:• A second StimSIS• Multi-ship Virtual Development Lab (MVDL)• Virtual Wingman
© 2019 Lockheed Martin Corporation. All Rights Reserved
AVOIDS - Air Vehicle Optimal Integration Development System
SimSISSimSIS
F-35 MISSION SYSTEMS INTEGRATION – THEN AND NOW
© 2019 Lockheed Martin Corporation. All Rights Reserved
DIS SimSIS
StimSIS
ASIF FlightTest
Domain Integration
Station
Stimulator System
Integration Station
Air System Integration
FacilityOASIS
Open-Air System
Integration Station
Supplier Labs
MVDL SimSIS
StimSIS
FlightTest
Multi-ship Virtual
Development Laboratory
Simulation System
Integration Station OASIS
Supplier Labs
Sim+
AVOIDS - Air Vehicle Optimal Integration Development System
AVOIDS
Air Vehicle Integration
Labs
Simulation System
Integration Station
StimSIS2
ONE BIG LESSON LEARNED – SHIFT LEFT TESTING!
© 2019 Lockheed Martin Corporation. All Rights Reserved
System Definition System andSubsystem Design Implementation Subsystem
TestingSystem Integration
and Test
Detailed Design
Software Development
Software Test
Operational ConceptDevelopment
System Requirements
Preliminary Design
Detailed Design Unit Test
Integrated System I&T
Verification Testing
Desktop Based Simulation (DBS)
Vehicle System Processing/Flight Control
System Integration Facility (VIF)
Ownship Sensors (OSS)Fire Control & Stores
Flight Test
CSCI Integration
Reprogramming Labs
88% of Anomalies are found prior to flight test
MVDL is a key enabler for shift-left testing.
MULTI-SHIP VIRTUAL DEVELOPMENT LABS (MVDL)
A GREAT EXAMPLE OF SHIFT-LEFT TESTING!
• Derived from the Verification Simulation (VSim)• Composed of:
• Simplified Cockpit, controls, and displays• Server-based computer system
• Virtual environment• Capable of simulating eight-ship mission• Mission systems sensor M&S• Vehicle systems M&S• Rehosted ICP, DMC, and AMS
• As M&S became more available and matured, the MVDL was used for Mission Systems I&T
• Software delivered to MVDL• Rapidly tested using automated test tools• Tested prior to introduction to large complex HITL
venues (StimSIS)
A second MVDL has been included as part of an agile development environment
© 2019 Lockheed Martin Corporation. All Rights Reserved
SUCCESS STORIES• Stimulator Labs (StimSIS)
• Initially the value was questioned.• The integration all of the stims/PME together was a great success• The users requested a 2nd StimSIS.• Both StimSIS 1 and StimSIS 2 have the highest utilization of all of the Mission
Systems Labs.• INS/GPS testing. In the lab, we flew the F-35:
• Across the equator• Across the prime meridian• Across the international date line• Over the north and south poles
• 88% of anomalies are found prior to flight test• MVDL was not in our original plan for Mission Systems I&T
• MVDL has proved invaluable for early defect detection• This will be part of OFP development & test as we move forward.
• Vehicle Systems Labs – Electrical Power Systems (EPS) lab F-35 RADAR in-rush current testing
© 2019 Lockheed Martin Corporation. All Rights Reserved
WE CONFIRMED A FEW THINGS
What were we thinking? What are we thinking? Lesson ConfirmedLegacy program had difficulty collecting and analyzing avionics data
F-35 HITL labs were designed with data acquisition in mind & tools were developed
Yes – Some tools are used from software development through System I&T
Legacy program realized that severe anomaly resolution required integrated testing in the HITL lab
Integrated testing – even if SPAR resolution for a single sensor
Yes – Integrated testing
Legacy program had no way to perform stability testing
Labs have the ability to run and stimulate multiple sensors for extended periods of time
Yes – stability testing is required in an integrated lab environment
Single sensor testing is adequate Re-learned the lesson that single sensor testing is just the beginning
Integrated testing – Test Like You Fly (TLYF)
Legacy Program labs were scattered geographically limiting the ability to perform full system integration
Program labs are collocated in Ft. Worth Yes – Mission Systems Labs, Vehicle systems labs Man-in-the-loop labs in FW
© 2019 Lockheed Martin Corporation. All Rights Reserved
WE LEARNED A FEW THINGS
© 2019 Lockheed Martin Corporation. All Rights Reserved
What were we thinking? What are we thinking? Lesson LearnedSeparate Virtual and HITL labs Integrated Virtual and HITL labs Use virtual test environments to mature
software before HITL testing
Deliver Models and Simulations after the capability has been delivered
Deliver models and simulations as soon as possible before capability is delivered
Use virtual test environments to mature software before HITL testing
Integrate separate MS and VS labs for Air-Vehicle testing
Need a single integrated MS & VS lab Integrated systems need to be tested in an integrated environment
Mission Systems subsystems can be tested separately
Mission Systems subsystems cannot be tested separately
Integrated systems need to be tested in an integrated environment
We can save money by deleting capabilityfrom the test lab
You will not save money removing capability from a lab
In the long run you will not save money byreducing lab capability
Labs can be used 24 X 7 Labs cannot be used 24 X 7 Unless the lab configuration is stable, upgrade and maintenance shifts are needed: 25% of prime time shifts.
Technology Refresh is planned for the end product but not for the labs
Labs also require tech refresh Lab systems are subject to diminishing Manufacturing sources and out-of-production parts – same as the end product
Labs will be scheduled 24 X 7 only for short periods of time (surge capacity).
Labs that are run 24 X 7 (over-utilized) suffer from reduced efficiency.
Lab cannot be used 24 X 7 for extended periods of time.
THE FUTURE…
INCREASE THE ROLE OF VIRTUAL SIMULATION IN THE
SOFTWARE DEVELOPMENT AND TEST PROCESS
• Shift Testing Left• Identify tooling and process changes that allow OFP
testing to naturally occur sooner in the development lifecycle
• Shift defect detection left out of lab and flight testing.• Increase maturity of SDLs before lab and flight testing
• Improve OFP Software Development Tools• Identify tooling and process changes that lower cost by
making each lab user more efficient and effective• Requirements force us to assess ‘high impact changes’
– not minor tweaks
• Improve Overall Lab Efficiency• Identify lab product and lab process improvements
that will reduce the overall cost while maintaining or increasing product quality
• Demonstrate capability maturity in lab testing and reduce the number of times a function is tested
• Make common automated test tools available throughout the software development process
• Integrate virtual and hardware in the loop development and test environments – F-35 Virtual Wingman.
© 2019 Lockheed Martin Corporation. All Rights Reserved
F-35 INTEGRATION LAB EVOLUTION
NEW TEST AND EVALUATION LESSONS WERE LEARNED
AND OLD LESSONS CONFIRMED.
• The F-35 Lightning II Mission Systems Integration Labs have evolved since they were described in 2003.
• Many of the development, test, and evaluation venues were repurposed due to the late availability of models and simulations.
• The multi-ship virtual development lab (MVDL) was developed late in SDD.
• It is a key enabler of shift-left testing.• It relies on models and simulations for software
development in an agile environment.• It is used from software development through
software integration and test. • The MVDL is the future of F-35 Lightning II
mission systems integration and test.
© 2019 Lockheed Martin Corporation. All Rights Reserved
WHAT QUESTIONS DO YOU HAVE?
© 2019 Lockheed Martin Corporation. All Rights Reserved
© 2019 Lockheed Martin Corporation. All Rights Reserved