Australian Government Department of Defence Defence Materiel Organisation
FUTURE SUBMARINES PROGRAM
ESSYS-622-79-02
MOTS COMBA"( SYSTEM
FOi 294/15/16 Item 1
RFI EVALUATION METHODOLOGY (MCSREM)
(ESSYS-622-79-01-10)
VERSION 1.0
31 AUGUST 2012
Prepared by
Engineering and Scientific Systems Pty Limited
Trading as Essys
Unit 8, 33 Meroo Street
Bomaderry NSW 2541
ABN 90 008 635 185
PO Box2433
Bomaderry NSW 2541
This material Is delivered by Engineering and Scientific Systems Pty Limited (Essys) under the tenns and conditions of OMO Contract 031015 ES-4 dated the 7 March 2007 and DMOSS Purchase Order4500849365 dated 4 Aprll 2012.
Annex A - Response Blanks Report .... ~ .................................................................... 12 Annex B - Evaluation Exceptions Report .................................................................. 13 Annex C - T earn Evaluation Report ................................................. ; ........................ 14 Annex D - Roll-Up Exceptions Report ...................... ................................................ 15 Annex E - Evaluation Count Roll-Up Report ............................................................. 16 Annex F - .MCSRE IPT RFI Evaluation Terms Of Reference .................................... 17
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31 August 2012) Page 3 of 17
Refer Distribution Marking on Document Cover Sheet
COMMERCIAL•IN-CONFIDENCE
UNCLASSIFIED
1 SCOPE
UNCLASSIFIED
GOMMERGIAL•IN•GONFIDENGE
1.1 Identification
This document is the evaluation methodology for the Future Submarines Program (FSP) MOTS Combat System RFI (MCSREM). This project is identified as Essys-622-79-01 .
1.2 Alm
The evaluation methodology details the process by which the evaluation of the responses to the RFI SOW will be undertaken covering:
1. Responsibilities for each team 2. Scope of evaluation to be undertaken by each team 3. Means of evaluation of responses to questions 4. Roll-Up mechanism utilised by each team 5. Terms of Reference for the Evaluation Workshop
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31August2012) Page 4 of 17
Refer Distribution Marking on Document Cover Sheet
COMMERCIAL•IN•CONFIBENCE
UNCLASSIFIED
UNCLASSIFIED
GOMMERGIAL IN GONFIDENGE
1.3 Evaluation Flow
Evaluation Sheets
Project Team
Passl Import
Pass2 Import
RFI RESPONSES
Issues
Guidi nee Prioritisation
EVALUATION REPORT
lnltlal Pass~ Payment Authorisation~
1.4 Clarlflcatlon Requests
The Initial pass is the stage at which any clarification required from respondents should be identified. These may be either:
1. Short falls in information supplied with respect to the requirements of the SOW that may delay payment; or
2. Discrepancies in, or clarification of, information supplied that need not delay payment
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31August2012) Page 5 of 17
Refer Distribution Marking on Document Cover Sheet
COMMERCl:AL-IN•CONFIBENCE
UNCLASSIFIED
UNCLASSIFIED
COMMERCIAL•IN•CONFIDENCE
There may be situations where issues in the evaluation at Pass 1, once considered in the workshop, may require further clarification from respondent(s).
In all cases the requests are to be forwarded to the Project Team for collation and transmission to the relevant respondent(s).
2 Team Roles
The evaluation teams have been constructed to assess different aspects of the responses, with at least two teams allocated to each section of the SOW with the exception of the commercial se,ction.
2.1 Project The Project Team will oversee the overall evaluation and provide quality assurance and consistency checks of other team assessments.
The Project team will also perform the roll up of the evaluation into the key criteria and constructing the top level traffic light report.
The Project team is responsible for writing the Evaluation Report.
2.2 Strategic
ihe Strategic Team will assess interoperability, alliances and specific existing inter-government or inter-defence-force agreements.
Due to the nature of strategic matters, the evaluation by the strategic stakeholders will be mainly subjective.
2.3 Operational The Operational Team will assess the respondent's Combat System functions capability to achieve the full range of the future submarine missions.
The evaluation will be objective at each lower level of response; however, subjective analysis will be required to roll up the evaluation to the desired level.
A separate sheet is provided to allow the operational team to evaluate each response against each mission requirement.
2.4 Engineering The Engineering Team will assess the ability of the respondent's Combat System design and construction to support the full range of the future submarine missions over the life of the submarine.
In particular, an assessment of the architectural design's ability to support the full life cycle of the future submarine including obsolescence mitigation, technical upgrades and capability upgrades.
The evaluation will be objective at each lower level of response; however, subjective analysis will be required to roll up the evaluation to the desired level.
2.5 Logistic The Logistic Team will assess the respondents logistics support capability to enable the through life support of the Combat System including training, spares, obsolescence monitoring, repairs and maintenance.
The evaluation will be objective at each lower level of response; however, subjective analysis will be required to roll up the evaluation to the desired level.
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31August2012) Page 6 of 17
Refer Distribution Marking on Document Cover Sheet
COMMERCIAL•IN•CONFIDENCE
UNCLASSIFIED
2.6 Certification
UNCLASSIFIED
GOMMERGIAL=IN=GONFIDENGE
The Certification Team will assess the respondent's ability to achieve the certification of the Combat System within the RAN Regulatory Framework including the standards and testing required.
The evaluation will be objective at each lower level of response; however, subjective analysis will be required to roll up the evaluation to the desired level.
2. 7 Commercial/Business/Legal
Commercial/Business/Legal Team will assess the respondent's business capacity to deliver the required Combat System including the costs, ability to provide a complete system and the engineering and commercial relationships with suppliers and shipbuilders to achieve effective integration.
The evaluation will be objective at each lower level of response; however, subjective analysis will be required to roll up the evaluation to the highest level.
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31August2012) Page 7 of 17
Refer Distribution Marking on Document Cover Sheet
GOMMERCIAL•IN•CONFIDENCE
UNCLASSIFIED
3 Methodology
3.1 Evaluation Sheets
UNCLASSIFIED
GOMMERGIAL=IN=GONFIDENGE
Evaluation sheets for each Respondent will be generated for each team once the responses have been received. In generation of the evaluation sheets, a check will be made to highlight sections of each response that may be deficient.
The evaluation sheet contains the response information, the Collins Reference Model information, header criteria and a mechanism to roll individual evaluations to the header levels.
The following items are highlighted in the example evaluation sheet (with dummy response data) pictured below:
1. The respondent name is fixed in each sheet
2. The team name is fixed in each sheet
3. Both baselines are provided on each sheet
4. Header level criteria are provided to assist in assessing the roll-up
5. The assessment value is provided as a list selection (1 -5 only used)
6. A control button is provided to perform the roll-up to header levels
7. The header level roll-up will identify if any subordinate evaluations have not been completed. The same list control ls used for setting header level evaluation.
8. Items not allocated to a .Team will have the evaluation preset to NIA
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31August2012) Page 8 of 17
Refer Distribution Marking on Document Cover Sheet
COMMERCIAL•IN-CONFIDENGE
UNCLASSIFIED
UNCLASSIFIED
COMMERCIAL•IN•GONFIDENGE
3.2 Response Evaluation by Teams Each evaluation team will assess the responses for sections of the SOW allocated to them. The breakdown of allocated sections to team members is at the discretion of the Team Leader. The Team Leader is responsible for validation of the evaluation scores and the justifications provided. The team leader is also responsible for the SOW Header Roll-Up for allocated sections and the justification of the roll-up determination.
3.2.1 Detailed Questions
Detailed questions are characterised by short, discrete objective answers that have corresponding reference values that can be directly compared to. Use of additional material supplied by the respondent required only to validate claimed figures or to gain an appreciation of functionality provided.
The results of the comparison will result on one of the following evaluations:
1. Unacceptable - provides less capability than the reference value(s) and requires an excessive change for any eventual procurement
2. Acceotab!e with Change- provides less capability than the reference value(s) and does require change for any eventual procurement and change is not excessive (e.g. redesign of a sonar array would be excessive while modification to sonar processing may not be excessive)
3. Acceotable Lower Capabllitv - provides less capability than the reference value(s) but does not require change for any eventual procurement
4. Acceptable Capability- meets the reference value(s) 5. Acceotable Higher Caoabilitv - exceeds the reference value(s)
Evaluations of 1-3 will require a justification of reason for the evaluation.
Where a question does not require an answer, the evaluation should be set to MN/A~ .
3.2.2 Summary Questions
Summary questions are characterised by long descriptive answers or diagrams will have corresponding reference descriptions that can be used to guide the evaluation. Use of additional material supplied by the respondent will be required to extract information to assess against the header level criteria.
The results of the comparison will result on one of the following evaluations: 1. Unacceotable - supports a capability that does not fully meet the reference
descriptions and requires an excessive change for any eventual procurement 2. Acceptable with Change - supports a capability that does not fully meet the reference
descriptions but does require change for any eventual procurement and change is not excessive (e.g. redesign of entire system architecture to achieve satisfactory security domains would be excessive while modification to network filtering may not be excessive)
3. Acceptable Lower Capabilitv - supports a capability that does not fully meet the reference descriptions but does not require change for any eventual procurement
4. Acceptable Caoabilitv- supports a capability that meets the reference descriptions 5. Acceotable Higher Capability - supports a capability that exceeds the reference
descriptions
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31 August 2012) Page 9 of 17
Refer Distribution Marking on Document Cover Sheet
GOMMERGIAL=IN·GONFIDENGE
UNCLASSIFIED
UNCLASSIFIED
COMMERCIAL=IN-CONFIDENGE
Evaluations of 1-3 will require a justification of reason for the evaluation.
Where a question does not require an answer, the evaluation should be set to "N/A".
3.2.3 Data Modelling
3.2.4 SOW Heading Roll-Up
The SOW Heading Roll-Up should be initially set to the minimum evaluation score of all questions In that section including subsection headers by using the "Roll Up Evaluation• function on each sheet. The Team Leader will have the option of changing the roll-up by either:
1. Decreasing the roll-up value (from values of 5, 4, 3 or 2) 2. Changing the evaluation score for questions within the section/subsection.
In either case, a justification must be provided including the initial evaluation score if it has changed.
The Roll Up Evaluation• function can be used at any time and it will request confirmation if any override will occur.
3.3 Evaluatlon Team Roll-Up
The Project Team will undertake the roll up of all team's evaluation into a single evaluation of each respondent. The SOW Heading ~oll-Up will be initially set to the minimum evaluation score of all team evaluations. The Project Team will have the option of changing the roll-up by either:
1. Decreasing the roll-up value (from values of 5, 4, 3 or 2) 2. Requesting a relevant team to re-assess particular section(s).
In either case, a justification must be provided induding the initial evaluation score that was changed.
The evaluation team roll-up will produce a number of reports to assist in the assessment process: 1. Header level team assessment exception status showing the number of evaluations
not completed and evaluations less than "4. Acceptable Capability (see Annex B) 2. Detailed header level assessment status showing evaluations of each team's roll-up
of the detailed assessment. (See Annex C) 3. Detailed header level team assessment exception status showing evaluations
differing from the roll-up of the detailed assessment and the justifications provided. (See Annex D)
4. Header level team assessment status showing count of each evaluation level within each section summed up to successive header levels. (See Annex E)
At the completion of the Team Roll-Up evaluation, the traffic light reports will be generated.
3.4 Response Evaluatlon The Team Roll-Up evaluation will also provide a report that collates all justifications at each header level to the key criteria which will be used as basis for undertaking evaluation against the key criteria.
The evaluation will entail constructing the key criteria summation for the evaluation report as the justification against the key criteria.
At the completion of the Response Evaluation, the Top Level traffic light reports will be generated.
Essys-622-79-01 · 10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31 August 2012) Page 10 of 17
Refer Distribution Marking on Document Cover Sheet
COMMERCIAL=IN=CONFIBENCE
UNCLASSIFIED
UNCLASSIFIED
COMMERCIAL•IN..CONFIDENCE
3.5 Evaluation Workshop
At a minimum the evaluation workshop will include Team Leads, Project management, COG management and key personnel from the teams as identified by the leads.
A draft evaluation report will be promulgated prior to the workshop for review.
The evaluation workshop will:
• Present final roll up to Key Criteria for each respondent and issues encountered;
• Resolve any remaining evaluation issues and feed back into evaluation matrix;
• Re-cast Combat System options if necessary;
• Present final roll up to Key Criteria (Report Basis) against each option;
• Present and resolve any issues with Final Draft Evaluation Report; and
• Minute changes required to Final Report.
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31August2012) Page 11 of 17
Refer Distribution Marking on Document Cover Sheet
GOMMERGIAL IN GONFIDENGE
UNCLASSIFIED
UNCLASSIFIED
COMMERCIAL•INaCONFIDENCE
ANNEX A - RESPONSE BLANKS REPORT
Response Blanks Report
S« Header Qs Rl R2 R3 R4
lll.1 Below WaterSltna1Ul'e 70 2
ll.2.2 Below Water Environment 43 9
U .6 Contlla Ma,,..ement u U .14 Onboard Tratnt,. 35
L2.S ()per9tlOMI certlfiQtlon 12 6
1.3.1.1 Leclslatlve eontonn.nce 4 4
1.3.3 System 22 s 1.3.4.9 Nevic•tlon 38 7
U .lo.8 Ship Control System I~ s U .11 Third Party lntelfatlon 14
1.45.4 Re-<ertif"IClltlon 7
2.1.1.1 Command Team Trainer 53
2.ll5 ~lntalner Trainers 13
Pagelofl
RS R6 R7 R8
1
s 2
1
1
1
Essys-622-79-01-10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31 August 2012) Page 12 of 17
Refer Distribution Marking on Document Cover Sheet
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31 August 2012) Page 16 of 17
Refer Distribution Marking on Document Cover Sheet
COMMERCIAL IN·GONFIDENOE
UNCLASSIFIED
UNCLASSIFIED
COMMl!RCIAL-IN=CONFIBl!Nee
ANNEX F - MCSRE IPT RFI EVALUATION TERMS OF REFERENCE
Objective
To inform the Government decision making process, a comparison of the Collins Class Combat System capability [The Collins Reference Model (CRM)] against existing commercial Combat System implementations is required. The evaluation process culminates in an evaluation report which
. provides the Future Submarines Program advice into the Government's decision making process. The report needs to arrive at a number of options that expose total cost of ownership versus capability and the costs associated with expanding the capability through life to meet changing strategic needs.
Roles
Notwithstanding the general roles of stakeholders under the FSP MCSRE IPT TOR, this DSD requires the following specific responsibilities to be undertaken in support of the RFI evaluation and report preparation:
a. FSP to provide technical and commercial data as required by the IPT, chair a weekly MCSRE IPT meeting/teleconference, and report to the FSP/Essys on IPT activities;
b. FSP to provide RFI supplier response data to the IPT;
c. COG to provide operational and functional advice on MOTS Combat System capability and design;
d. DSME to provide technical advice on the proposed MOTS Combat System solutions, certification and platform integration; and assist with the RFI evaluation and contribute to the RFI Evaluation Report;
e. Essys to provide management and quality control of the RFI evaluation process and workshops ·in accordance with the MCSRE Evaluation Plan and develop the draft ahd final RFI Evaluation Reports;
f. FSP to provide platform impact advice of the proposed MOTS Combat System solutions;
g. DSTO to provide technical data and advice on MOTS Combat System capability, design, integration, evolution; and assist with the RFI evaluation and contribute to the RFI Evaluation Report.
Outputs from the IPT
The following initial, outputs will be required in support of this DSD:
a. Analysis of MOTS Combat System RFI response data; and
b. Development of draft and final MOTS Combat System RFI Evaluation Reports.
Essys-622-79-01 -10 MOTS COMBAT SYSTEM RFI EVALUATION METHODOLOGY (MCSREM) (v1 .0 31August2012) Page 17 of 17
Refer Distribution Marking on Document Cover Sheet
COMMERGIAL•IN•GONFIDENGE
UNCLASSIFIED
•,
COMtVmRCIAL=II<l'-COI<l'PIOl!NCl'!
SEAlOOO Future Submarines Program fllVY
MOTS Combat System RFI Comparison Report
FOi 294/15/16 Item 2
Title SEA1000 - MOTS Combat System RFI Comparison Report
Prepared by
Security Classification
Document No
Version Number
Total Number of Pages
Future Submarines Program Office
RFSTRICTED
SEA1000/ AB12487861
1.0
47
'R~lUC'ffiD
COMMERCIAL-R••COt~IDEt4CE
RtiS'fRICflID COMMHRClAl:rlNaCONFIDENCE
REVISION HISTORY
Revision Date Version Revision description
14Jun 13 1.0
Consultations This document was preipan!d
Endorsements This document is endorsed
Name
CORE Bronko Ogrizek
CORE Rob Elliott
Dr David Kershaw
Notes:
Initial Issue
DSTO SEA 1000
DSTO A/HSMCS
Appointment
OOFSP
OOMD
OSTO SEA1000 S&TAdvisor
Approved by
B.S. OORIZBK
--
The master copy of this document is electronically maintained in Objective. Controlled paper copies of this document will be stamped as CONTROLLED COPY by a Data Management Officer in Red colour. All other paper copies of this document are UNCONTROLLBD and will not be automatically updated.
llliSIMCI £15 @81Uftill!ll81215 fl I @81 I Fib DI IM
IUis'FRIC:RsEl @8J:Htll!R@k'ds It f @81 fFl~Ef f@El
Executive Summary
As part of Phase 0 activities, the SEA1000 Project released follow-on Request For Information (RPI) packages, to prospective platform designers in early 2012 to assess MOTS Platforms, and to Combat System Providers in August 2012 to assess MOTS Combat System suitability for Future Submarine (FSM).
This report discusses the responses to the 2012 MOTS Combat System RPI and is intended to ascertain the viability of, and the risks associated with the selection of a MOTS Combat System for the Future Submarine. The assessment of each of the respondent's proposals is based on the written responses received.
Following delivery of the RPI. to the potential Combat System suppliers, SEA1000 staff conducted UNCLASSIFIED briefings to clarify any potential misunderstandings about the intent and purpose of the RPI and to receive any briefings from the commercial entities.
The assessment undertaken was a comparison of a Collins Reference Model (CRM), based on CSOS.02 in service with the Royal Australian Navy, against supplier's assurances as to the functional capabilities of their proposed systems. The level of responses allowed the analysis of a general level of compatibility or gross deficiencies in the systems offered.
None of the proposed combat systems would provide an off the shelf solution that would match the existing Collins Class submarine combat system capability or requirements. Significant customisation and modifications would be required for any MOTS Combat System to meet Australian requirements.
RilOfiilil I Wliili COJoOotriRClAls IN COtWl'8EtJCE
The RFI respondents all have capability shortfalls in the weapons subsystem, sonar subsystem, optical subsystem, external communications subsystems and shore facilities that require changes to meet Australian requirements.
A key capability required is the ability to support the insertion of 3rd party technology into the Combat System. There are two parts to fully support this capability, while also ensuring the protection of the 3rd party intellectual property. The first part is the implementation of an information assurance framework required to prevent the unauthorised leakage of information across defined boundaries. The second is the ability of the Commonwealth or its agents to insert the 3rd party technology without the assistance of the Combat System Provider so as to maintain National Sovereignty and protect National Security information that may not be releasable to the Provider. None of the Combat System respondents were able to fully meet both of these requirements.
Further studies will be required to establish the true through-life cost of ownership of the Combat System that includes all Parent Navy support facilities and infrastructure and any associated commercial aspects.
Interviews and tours of Combat System Respondent Facilities have identified that the procurement of a land-based integration and test facility, that transitions to a through-life support facility, will provide significant combat system technical and capability risk mitigation. This facility should be independent of a training system.
Acknowledgements
The completion of this report would not have been possible without th~ assistance and key contributions from Subject Matter Experts from DSTO, DSME and the RAN.
Many thanks for the contributions of the following people who all provided key inputs as Subject Matter Experts into the process for completing all aspects of this RFI:
• Mr Ross Susie, Mr Adam Sbrana, Mr Warren Smith, Mr Stephen Hoefs, Mr John Stobie, Dr Richard Ellem, Dr Samuel Huf and Dr David Kershaw from DSTO's Maritime Operations Division,
• Dr Gerald Bolding from DSTO' s C3I Division, • Mr Steve Impett from DSME, • LCDR Steven Wren, CMDR Neil Carson, CMDR Ian Jimmieson from the RAN, • Mr Anthony Irving, Mr Michael Kim and Mr Damien McGinnes from OMO, and • Mr Ted Vanderhoek and Mr Peter Garrard from ESSYS.
Without their assistance, this report would not have been able to be completed.
Rl98'ffttC'fE8 •QJ:H1MSR'1h\k R J '19t JBlil!lt iCJJ!1
" -USTRY SUPPORT 27 4.6 SCHEDULED DELIVERY PRIOR TO COLLINS WITHDRAW AL 28 4. 7 TECHNICAL RISK 28 4.7.1 CAPABILITY SHORTFALL 28 4.7.2 THROUGH LIFE SUPPORT 29 4. 7 .3 THROUGH LIFE EVOLUTION 30 4.7.4 SUPPORT INFRASTRUCTURE 30 4.7.5 PLATFORM IMPACTS 31 4.7.6 CERTIFICATION RISKS 32 4.8 COMMERCIAL RISK 33
S. CONCLUSION JS
6. REFERENCES 37
Appendix A Detailed Traffic Light Report Appendix B Risk Analysis Report
Rffi'flUC'ff:B t;QUt At:R•I Us IN •QNl*llisll!i:ti€i
I
38 44
RESIRICJED CSIJhhdtCL iE II 4 66141 lbd 4 LE
1. Background .
The Future Submarine project (SEAlOOO) is tasked with providing Australia with an advanced submarine capability beyond 2025. This submarine force will have a superior ability to deny an adversary effective deployment of its maritime forces (through AntiSubmarine Warfare and Anti-Surface Warfare operations), to utilise stealth to achieve assured projection of Australian force (through employment of strike weapons) and to build the situational awareness of the Australian Defence Force in deployed areas.
To maximise the benefit of procuring this advanced submarine capability, the Future Submarine (FSM) program will need to establish a strategy to maintain this world class capability throughout the life of the platform. This will require careful and early consideration of the FSM strategy to achieve an appropriate technology and capability refresh capacity to maintain an RAN capability advantage, as well as the framework to ensure that key enabling infrastructure is realised.
1.1 The SEA1000 Context
In late 2008, the SEAlOOO Project released an initial Request For Information (RFI), as part of the Phase 0 activities, to potential submarine designers as the first stage of a proposed procurement process for the acquisition of the replacement submarine capability.
As part of follow-on Phase 0 activities, the SEAlOOO Project released follow-on RFI packages, to prospective platform designers in early 2012 to assess Military Off-The-Shelf (MOTS) Platforms, and again to Combat System Providers in August 2012 to assess MOTS Combat System suitability for FSM.
For the purpose of this study, MOTS has been defined to be systems that are available for purchase, with no modifications, except for changes that are required to meet Australian regulatory constraints.
The Phase 0 activities lead to the release of a formal DSTO Technical Report DSTO-TR-2333, authored by Dr T. Mansell (DSTO) and Mr R. Oark (OMO) 11s1, which studied each potential candidate option. Each option was benchmarked against key drivers as identified in the 2009 Defence White Paper (131.
The 2012 Combat System RFI respondent information was delivered in the form of presentation materials and technical discussions with the SEAlOOO Combat System RFI team; lead by LCDR S. Wren and supported by Mr R. Susie (DSTO) during a visit in September and October 2012. The completed spreadsheets containing answers to the questionnaire that formed the RFI were delivered to the SEAlOOO Office in early December 2012 to allow analysis to commence.
The responses to the 2012 MOTS Combat System RFI are intended to ascertain the viability of, and the risks associated with the selection of a MOTS Combat System for the Future Submarine.
M!M7ftlCPf!e @@hBetM:@l!'tls BI @@I WHJI!l f@l!l
fl&lltlCIES COMMERCII'te IN CONFIDENCE
2. Scope
2.1 Identification
This document is the comparison report for the FSP MOTS Combat System RFI and is known as the MOTS Combat System RPI Comparison Report (MCSRCR).
2.2 Aim
The aim of the report is to conduct a survey of commercial Combat System suppliers that reports capabilities and costs over the breadth of the Combat System as defined by the following context diagram (Figure 1 below) for the Future Submarine.
The comparison is informed by the key capabilities available for the Combat System as determined by the Collins Reference Model (CRM). These key discri.minators include the technical capability demonstrated, the ability to integrate and protect third-party devices, management of obsolescence issues and the management of capability upgrades. Indicative life support costs and models were also sourced to compare all the Combat System options .
The CRM was derived from the current Collins CS requirements baseline (CSOS.02) as defined by References 131 through 191.
Questions related to functionality derived from submarine capability statements in the 2009 Defence White Paper that are not supported by the CRM were also included in the questionnaire and are analysed and assessed separately.
For the purposes of this report, the Combat System is defined as the elements of the submarine that contribute to the navigation, sensing, detection, classification, localisation, tactical picture compilation, planning, communication, tactical information distribution, onboard training, problem monitoring and localisation, countermeasure engagement, weapon firing, and weapon control.
These combat system functions are typically realised through the provision and integration of the following core systems: • Navigation sensors and processing system; • Acoustic sensors; • Acoustic sensor processing; • Above water sensors (including periscopes, electro-optics, ESM, and radar); • Above water sensor processing (including visual, ESM, and radar); • Combat Management System (CMS); • Communications system (internal and external tactical communication); • Countermeasure system; • Weapon handling system; • Torpedo weapon system; and • Anti-ship missile system.
This definition extends the traditional view of the scope of a combat system and reflects the increasing levels of integration of sensors, navigation and tactical information, and the increased reliance on external communications to support network-centric war fighting.
The Combat System RFI questionnaire provides the respondents with descriptions of the subnlarlne missions and examples of scenarios that exercise those missions. The missions can be grouped as: • Intelligence Surveillance and Reconnaissance (ISR); • Anti Submarine Warfare (ASW); • Anti-Surface Warfare (ASuW); • Land Strike; and • Special Forces Support.
3.2 Analysis Methodology
The MOTS Combat System RFI followed standard OMO commercial format with a questionnaire comprising an extensive set of questionsl related to: • Functional Capability; • Operation; • Design and Construction; • Engineering Management; • Support Systems; • Commercial Considerations; and
1 Over1800
W 'flilt@WJ8 @@hllill!!R@li its II f @@I ffl8'1I f@fg
COiottctEREIADllG-WIGI IDEIGCE
• Cost.
Each area contained questions for the respondent to fill out and in the majority of cases required a short answer. Most questions had a baseline answer from the CRM for the evaluation team to use for comparison. Questions were also included for known extensions to the CRM based on anticipated FSM requirements or known deficiencies in the CRM. The questionnaire included the ability for references to supporting information to be supplied as part of responses to the MCSRCR.
The questionnaire provided the respondents the ability to respond against two baselines: 1. Current Baseline - The most recent baseline in service by a customer Navy that best
satisfies the questionnaire, and intended to confirm supplier and sub-system supplier interactions; and
2. Proposed Baseline - Under development for delivery by 2015-16 for a customer scaled to meet the scenarios provided in the MCS RFI, and intended to identify the suppliers' leading edge capability.
The assessments throughout this report are based on the capabilities offered by each respondent's Proposed Baseline in a direct comparison against the capabilities offered by the CRM. The Current Baseline has been used to confirm viability of the Proposed Baseline, in terms of system supplier and sub-system supplier extant commercial and technical relationships.
The detailed analysis was conducted by the following teams: a. Operational; b. Engineering; c. Logistic; d. Certification; e. Strategic; f. Project; and g. Commercial.
The analysis process, detailed in the MOTS Combat System RFI Assessment and Analysis Plan 111, provided scoring of the individual questions and a roll up at each header level with comments to support the scoring and roll up.
This report is derived from the analysis by reporting the rolled up scores and utilising the Subject Matter Experts (SME) comments to generate the bulk of textual content, and as a basis for forming the report's conclusions.
3.2.1 SEAlOOO Industry Visits
Following delivery of the questionnaire to the potential Combat System suppliers, SEA1000 staff conducted site visits with each of the potential suppliers. LCDR S. Wren and Mr R. Susie visited each site to conduct UNCLASSIFIED briefings to clarify any potential misunderstandings about the intent and purpose of the RFI and to receive any briefings from the commercial entities.
Rl38'fftl@ilXl!B @eteflel!!Il@!M!S HQ MI ff'I15!!I f et!
RWIRICIED
3.2.2 Clarification Requests
Clarification requests were formulated and transmitted to the MOTS Combat System RFI respondents wherever clarification was required to complete analysis. Where clarification was needed but the clarification would not affect the analysis outcome, clarification was not sought from the respondent.
3.3 Analysis Key Criteria
Key analysis criteria in the assessment of the respondents' submissions consisted of: • Capability Vs CRM; • Capability Vs FSM New Capability; • Cost-Effective Submarine Capability;
• • Australia Based Industry Support; • Scheduled Delivery Prior to Collins Withdrawal; and • Technical Risk.
3.3.1 Capability Vs CRM
The analysis assessed each question for each of the two baselines as either: • (Acceptable Capability} - Meets/Exceeds CRM capability; • {Acceptable Lower CapabilityJ - Less than CRM capability but acceptable; • (Acceptable Capability after Upgrade I - Upgrade required for capability to be
acceptable; • {Unacceptable Capability} - Does not meet CRM capability and is incapable of upgrade; • {Capability Not Provided\ - No functionality is provided to support this capability and
functionality may be additional to the CRM; or • !Not Assessed} - Information available from the RFI was insufficient to make an
Upgrade in this context means that an existing capability can be improved by a. Software change; or b. Choosing other options available for an equipment; or c. Replacement of an existing component with a compatible component.
Where an assessment of !Not Assessed} or !Capability Not Provided I has been made, further work may need to be undertaken with the respondent to determine what changes to the respondent's system or other deliverables are required (or if any are possible).
A summary of the analysis is presented at a suitable rolled up header level and is attached as the Detailed Traffic Light Report (Appendix A).
3.3.2 Capability Vs FSM New Capabilities
New FSM capabilities derived from the Defence White Paper 2009 1131 include: • Strike Capability; • Very Secure, Covert Real-Time Communications; • Support for Special Forces; and • Support for Off Board Sensors.
The analysis assesses the ability of the respondent to meet these potential requirements.
The assessment is summarised against the specific question header levels that pertain to these FSM capabilities and in the corresponding risk assessment.
For capabilities that the Combat System respondents have provided no response, or have indicated that the particular capability is not av~able, the responses related to the ability to conduct integr,ation of third-party products and the protection of Intellectual Property is used as an additional discriminator. The protection of Intellectual Property and the Integration of Third-party products are assessed as part of the Cost-Effective Submarine Capability discriminator.
3.3.3 Cost Effective Submarine Capability Edge
The challenge for a MOTS Combat System is to provide the full Combat System capability at a level that provides an edge over regional threats for the life of the Future Submarine, which is required for the Future Submarine to fulfil all its proposed roles.
Achieving and maintaining a capability edge will require augmentation through third party products and through evolution of the supplier's baseline.
RES I RIC I ED WIOllOIERLMDJG-L&GPIBEIGCE
RES I RIC I ED COMMERCffiCii<J-Wi<JFIDEt<JCE
The analysis assesses the ability to support Third Party Products and the level of integration that is achievable for: • Indigenous Research and Development Trials; • Indigenous Combat System Products; • Third-Party Combat System products; and • Protection of Third-Party Intellectual Property.
The analysis addresses the following areas: • In built Security and Privacy controls (Information Assurance); and • Certification impacts especially the ability of an Australian based testing capability that
is able to isolate RAN only capability from the supplier's non-Australian workforce.
The analysis assesses the capacity of the supplier to undertake evolution of the Combat System through life based on: • Any current supplier funded upgrade program; • A request for response to a hypothetical upgrade in terms of overheads (re-certification,
logistic support (ll.S) updates, re-installation, configuration management); • The ability and mechanism by which RAN directed capability change is able to be
isolated from the other supplier's customers; and • The ability of the supplier to manage multiple baselines (Configuration Management,
Il.5, Facilities and workforce).
While the analysis cannot provide complete through life costs, it should provide the basis for a comparison between the potential M01S suppliers, however the analysis will not provide sufficient basis for comparison between the required Australian capability and the proposed M01S capabilities. This activity will need to be addressed through the engagement of the M01S Parent Navies to understand the support activities and throughlife cost implications.
The assessment is summarised against the specific question header levels that pertain to the ability to support maintenance of the capability edge and in the corresponding risk assessment.
'9Q)u0uHsA€llds It 1 €'et I~l!t IQ!
M!:jl'Jtf@ff!1' C!OIVHOIEltCIA£&11QZCOiU IOEIQCE
3.3.5 Australia Based Industry Support
The analysis assesses the respondent's capacity to undertake the installation and through life support of the Combat System using Australian based assets. This may be a mix of: • Original Equipment Manufacturer's (OEM's) Australian location(s); and • . Australian industry as agent for the OEM.
The analysis assesses the respondent's capacity to undertake the management of upgrades to the baseline of the Combat System using Australian based assets for: • Capability Insertion; • Obsolesce~ce Monitoring; • Technology refresh; and • Certification of an Australian variant.
3.3.6 Scheduled Delivery Prior to Collins Withdrawal
The analysis assesses the respondent's capacity to meet delivery deadlines of the Combat System within the current project schedule. The assessment includes: • Upgrades required to meet FSM required capability; • Integration of Third Party products needed to meet FSM required capability; and • Impact and cost of transition of crews.
3.3.7 Technical RiSk
The technical risk assessment seeks to broadly determine the likely technical risks associated with either the installation of the Combat System as a MOTS item, with the exception of required legislative and regulatory changes, and also the installation of additional capability currently sourced through co-operative programs with the US.
Thus, the risk assessment considers two broad scenarios: 1. The respondent's proposed system is accepted in full with no change to weapon types;
and • • • • • 2. Th d t' d t ble of integrating extant RAN weapons and
All risks considered through the analysis process are relative between respondents and not absolute. These risks have been identified through the engagement of Subject Matter Experts and are indicators of expected technical risk. Any risks assessed at this point of the project would, almost by definition, be automatically high. The following risks have been considered:
3.3. 7.1 Capability Shortfall The analysis assesses risk based on the respondent's system capabilities and the amount of change required to bring the system up to a FSM level for each scenario.
M8t'Rl@'itl~8
iii It liill ii 'at II f iii ffllHH Hiil
IC& I me I Lb @@lilhlElt@H iL II d €6141 WEI d CE
3.3.7.2 Through Life Support The analysis assesses risk based on the ability of the respondent's capabilities to be able to · support the Combat System through life. The risk is characterised by the ability to support maintenance, defect rectification, tolerance to failure, adequate supply lines and clear strategies for the management of obsolescence and is considered for each scenario.
3.3.7.3 Through Life Evolution The analysis assesses risk based on the respondent's capabilities to be able to evolve the Combat System to higher levels of capability through life. The risk is characterised by the ability of the Combat System architecture to support the periodic change, the ability to support multi-vendor components and the ability to support sensitive RAN and other allies' information and components and is considered for each scenario.
3.3.7.4 Support Systems The analysis assesses risk based on the respondent's offered support systems and engineering capabilities and the amount of change required to provide effective facilities to support training and integration, development and test at a FSM level for each scenario.
3.3.7.5 Platform Impacts The analysis assesses the respondent's hotel services load against a defined platform Combat System allocation. The allocation should represent the maximum that can be provided to the Combat System without impact to the platform dived endurance.
The assessment includes margins for: • A capability upgrade path; and • Third Party products if required to complete the baseline system.
The analysis assesses the risks associated with the respondent's platform integration requirements.
3.3.7.6 Certification Risks The analysis assesses the certification risk by comparison of the respondent's standards to standards that have been used for the certification baseline of Collins CSOS.02.
The assessment also includes the capacity of the respondent to complete Certification of the Combat System at each certification stage (CAT 3 - Shore Based Testing & Certification, CAT 4 - Harbour Acceptance Testing and CAT 5 - Sea Acceptance Testing).
The assessment ranks the risk according to the perceived re-work involved to meet the minimum standard for each scenario.
3.3.8 Commercial
Corporate information was sought for: • Corporate structure including related bodies; • Corporate mechanism for provision of the Combat System (prime, consortium, joint
venture); • Commercial Arrangements with parent Navy, subcontractors and partners; • Governance Arrangements with parent Navy, subcontractors and partners; • Corporate capacity and resources;
l\£::"1'ftl@! I E:e WNliOIERLME-hG-CONFIO&GCh
RM7HH@'ff!8 CQMMl!RC&\J:s ItJ CQHFI8EHCE
• Australian based capacity and resources; • Corporate financial position; • Any known projects (Parent Navy or Export) that would conflict with the Australian
Future Submarine build program for resources; and • Any known restrictions on working with the candidate ship builders.
Commercial information was sought for • Insurance arrangements typically used for delivery of a Combat System; • The owner of significant intellectual property in the Combat System and licences
attachedtothatlP;and • Details of any export controls associated with the Combat System and the ability for any
such grant being approved.
The analysis assesses each respondent in its capacity to: • Provide the majority of the Combat System with minimal GFM; • Undertake a long build program with deliveries as required and incorporate technical
refresh and/ or capability upgrades; and • Undertake initial and future capability upgrades with adequate protections for
Australian IP and capability.
The cost analysis is based on a cost model with the following top level breakdown: • Capital Cost including boat systems, boat installation, llS products, facilities and
certification; • Capability Sustainment Cost including maintain sustainment capability, systems
support, obsolescence management and training support; and • Capability Insertion Cost including capability development, llS product update, system
re-certification and update of systems.
3.4 Conduct of Analysis
The analysis of the MOTS Combat System RPI responses was conducted in accordance with the approved Assessment and Analysis Plan (Reference Ill). In particular: • Probity briefings were conducted for all staff involved in analysis of RFI data. All data is
stored on access-controlled Defence systems (t6J.
• Subject Matter Experts were used to assess all responses, conduct initial analysis, and seek clarification where required.
• A MOTS Combat System RFI Analysis Workshop was conducted in Adelaide between the 4th and 7th February 2013, to seek SME agreement 01' the analysis of all respondent sections. This activity completed the analysis of the respondent data, including SME narratives for sections of analysis, resulting in the development of DMOFSP-RFI0230-2012 MCSRSOW A-C Analysis Spreadsheet 1121 as the completed product.
• The roll-up process is intended to encapsulate the capacity of the Combat System Respondent to meet or exceed the capability as described by the CRM. ·
RESllGCIZB @SlbfhlEICCL i£ ii 4 CCI Ci MEI t EE
Capability vs FSM New R uirements
Cost Effective Submarine Capability Edge
Australian Based Industry Support
Scheduled Delivery Prior to Collins Withdrawal
Technical Risk
Through Life Cost Obsolescence Cost Ca bili Insertion Cost
k&lkiCIEB
4. Analysis Summary
Capability Edge can be maintained without extra burden
Capable of proactive sustainment in Australia Capable of meeting deliv schedule Acceptable risk without additional mana ement Better than CRM
Lower Capability but a table Lower Capability but acce table
Capable of Reactive sustainment in Australia
Acceptable risk with additional mana ement SameasCRM
Requires Upgrade to bea ble Capability Edge can be maintained with acceptable extra
Limited Capability for Sustainment in Australia
Risk can be acceptable with upgrades
Within 20% CRM
Table 2: Definitions for Top Level Traffic Light Report
Ml8'fftt:CifAE8 iiH o mnm , I M 1 iOt mwwt 1 i£
Unable to maintain capability edge
Not capable of sustainment in Australia Unlikely to meet deliv schedules Risk is too high
More than 120% of CRM
RMI RIC I ED CGl!E!JbltdliE ht 26242 IBZECZ
4.1 Capability Vs CRM
None of the respondents can provide an off the shelf system that meets the CRM capability. Significant modifications and customisations will be required for any MOTS Combat System to meet RAN mission requirements.
While the RFI did elicit a large amount of information, it was not possible to fully assess performance in all functional areas. For most of these cases an optimistic assessment score was applied.
The key sub-systems required for the operation of an effective Combat System cover those of Sonar, Imaging, Tactical, Weapons and Communication sub-systems. Support sub-systems (such as training facilities, maintenance facilities and support organisations) form the infrastructure required for the on-going maintenance of the capability.
Combat System design trade-offs result from conflicting customer requirements. For example, sonar signal processing capabilities for the detection and tracking of quiet targets is a very processor intensive capability, and thus requires a far greater hotel load than is typically supplied as part of the MOTS Combat Systems. The lack of these specialised signal processing techniques indicate that the MOTS Combat Systems have been optimised for different missions than those mandated for the Future Submarine.
All the MOTS Combat Systems considered have shortfalls in areas of capability that seem to indicate design customisations that meet current customer requirements rather than the actual technical capacity of the Combat System respondent. Additionally, all respondents would require significant upgrades to their proposed systems to address missing and degraded capabilities that are beyond the scope of normal system tailoring i;tpplied during MOTS procurement.
4.1.1 Tactical Subsystem
The Tactical Subsystem collects all sensor data input, and where required processes this information to provide a full state solution (bearing, range, course and speed). It is the central component of a CS and is utilised to form an overall tactical picture for the command
KW I me I ED C8loHotl9R:@IAls ff J @@I JFte:f!t f@f!
li&IHICIZB COIOLOIERCMOHQ•eE>tU IDENCE
team. It includes capabilities such as mission planning, contact management and track management.
There are other cases where there is significantly lower capability provided by a vendor compared to that provided by the CRM, and desired for the FSM. Upgrades to this capability would require significant effort in algorithm development as well as associated hardware, rather than a change to the CONOPS.
4.1.2 Sonar Subsystem Capability Assessment
4.1.3 Weapons Subsystem Capability Assessment
RWIRkiED @elOHUIEll@f21t15i!fI f @eNfI1'1!I f@E
RID I me I Lb COIOllCIERLIAE-HQ-CONfilbEIQCE
4.1.4 Optical Subsystem Capability Assessment
A direct comparison was limited by the fact that all respondents offered at least one nonpenetrating Optronics Mast. Optronics Mast imagery is in many ways acquired and consumed in a different manor to the conventional periscopes which represent the CRM baSeline. Whilst direct view optical paths have both strengths and weakness compared to digitally mediated displays, and the comparative impacts on the control room and latform can be far reachin a ha s more im rtant discriminator is one of stealth
The ongoing development of optronics masts and capabilities is an area of intense interest and is currently being actively developed by a number of countries, but is not expected to be offered in MOTS systems at any time in the near future. As reflected in all respondent offerings, at least one Optronics Mast is a strong possibility for FSM. DSTO is monitoring and providing guidance to the operational community on the impacts of this evolving technology, yet such a decision involves a wide trade off space and is not be possible in isolation of other FSM requirements.
The existing Collins external communications subsystem is planned to be upgraded under SEA 1439 Phase 5B2 that will provide a communications capability that meets the CRM specification baseline. The respondents' proposals were assessed against the Phase 5B2 capability.
RESIRIC I ED lllilltlSitlb £8 21 t 661 ii £3£1 :CE
WMMERLIAE-fi<J-Wi<JFIOEl<JCE
Notable shortfalls in respondents' external communications subsystems were proprietary management interfaces; unsatisfactory satellite communications and the level of integration to enable networked operational capabilities such as Maritime Tactical Wide Area Network MTW A or Command and Control Local Area Network CCLAN .
4.1.6 Support Systems Capability Assessment
In order to support through life technical refresh and capability upgrades, not only do the shore facilities need to be able to be upgraded in line with the at sea system, the shore facilities need to support older baselines for submarines. None of the respondents' solutions have addressed this issue and would require significant re-architecting to achieve it.
Supporting the training load for a large number of submarine crews effectively requires a significant investment in training facilities other than a single command team trainer. The current Collins training facilities utilises part team trainers, maintainer trainers and computer based training in class rooms to manage the training loads. Computer Based Training (CBl) must include operator training on the operational HMI with realistic simulation of data to reduce the load on other facilities. No respondent has provided these capabUities.
For the Collins combat system support, the RAN is the integration authority and CAT 3 certification authority; hence a land based test site has been put in place which is maintained with all operational AN/BYG-1 baselines. The use of an LBTS for the FSM combat system will be required to undertake third party integration and final CAT 3 certification. The procurement strategy may place this responsibility on a commercial basis and the facility may be located in company premises.
For the Collins combat system support, the RAN is the Configuration Management (CM) authority; however, the FSM procurement strategy may place this responsibility on a commercial basis. No respondent proposed a CM support system but all provide configuration data on their delivered components.
4.1.7 Overall Respondent Assessments
RE'SFRICTISB C@hlftlf!RCh\&tN C@f WJ8Efi@IS
IU!!81't!U@T!!r8
4.2 Capability Vs FSM Additional Requirements
For the Future Submarine, the prime missions are considered to be Intelligence, Surveillance and Reconnaissance (ISR), Anti-Submarine Warfare (ASW) and Anti-Surface Warfare (ASuW). .
The use of the Future Submarine as a strategic asset within a network centric environment requires the installation of additional functionality and capability in the Combat System well beyond what is currently available in Collins. Primarily, this strategic capability is enabled through the installation of secure and covert real-time communications capability.
As the FSM capability requirements have not yet been formalised, a comprehensive assessment cannot be made against these particular requirements. However, the ability to integrate products and maintain confidentiality and protect third-party IP is considered vital for the ability to improve FSM capability shortfalls, and is also considered as part of this assessment.
The Combat System respondents provided responses for inherent capability that potentially meets FSM Additional Requirements, and are characterised through the responses to areas that are listed in Table 3 below which is an extract from the detailed traffic light report that is attached as Appendix A. The legend of the colours used in this table is listed below in Table4.
R:ES'fR::ICTBB SQ' 0 lFPSI 'I A I SQt Jfll2it JSi
RBIRICIED
Table 4: Legend for Capability vs FSM Additional Requirements
None of the Respondents offered Combat System elements that met all the additional capabilities sought for Future Submarine.
The major capability shortfall is the lack of any implemented Information Assurance frameworks. Based on the evidence to date, the installation of these frameworks will require significant re-architecture of the proposed Combat Systems, and impose additional protection mechanisms for the release and integration and protection of sensitive third-party products and intellectual property.
The other prime capability shortfall is the lack of full ISR Mission Support. None of the respondents have had experience with the ISR Mission Requirements as articulated by Australia, or is currently operated on the Collins Submarines. This is evident through the system architectural solution that has been proposed by each of the suppliers, which includes a single security enclave that has implications for the sourcing and storing of sensitive information.
RWIMCIEB COMICIEREIA&HG-LOIGFIBEIGCE
Although the responses indicate that the full ISR Mission Support has not been supplied, it is likely that some of the suppliers have the capability to develop this requirement, with the associated costs and risks to be fully home by the Commonwealth.
4.3 Cost Effective Submarine Capability Edge
Maintaining a cost-effective submarine capability edge over potential adversaries in the region is a significant driver for ongoing submarine capability. The ability to maintain and improve Combat System functionality, and to integrate and protect third-party Intellectual Property and Products are key characteristics necessary to maintain a cost-effective submarine capability edge.
The sections of the questionnaire focused on encapsulating these capabilities included Technical Refresh, Capability Upgrade and Third-Party Integration.
4.3.1 Technical Refresh
Technical Refresh was defined as 'System baseline change with scope limited to hardware and/ or software updates to overcome current and future obsolescence, such that no change to functionality results.' This question set focussed on an ability to accommodate obsolescence.
e e u es eac su marine or tee ·ca re on every o er Tee ·ca Insert cycle, which results in a four_yearly refresh cycle. The current Australian approach links the technical refresh to the scheduled docking program for each submarine. This approach results in an eight year technical refresh cycle as a result of the platform impacts required by the combat system foot print. Prom TI 10 baseline onwards the US design has allowed more flexibility in this approach where the hardware change has shifted from cabinet level to drawer level and thus the engineering change package can be independent of the docking cycle.
KWIRIEIED
It& I me I Lb CotMCIEICCMDliG-CblQf IOEIGCE
4.3.2 Capability Upgrade
Capability Upgrade was defined as 'System baseline change with unlimited scope to hardware and/ or software updates that may result in significant change to functionality, and may also include change to overcome current or future obsolescence issues.' This question set focussed on an ability to insert new capability.
4.3.3 Third-Party Integration
Third Party Integration as a goal has been defined as being the ability for third parties to integrate their products without the involvement of the CS vendor. Under these conditions the Commonwealth would need to accept responsibility for ensuring the CS design was not compromised.
As will be seen, some vendors have the notion of the 'Integration of Third Party products'. This was not the intention of this question set and has been subsequently rated as noncompliant with the CRM.
For reasons of National Sovereignty and the protection of sensitive third-party IP the Commonwealth wishes to retain the right and ability to insert capability without a need to involve the Combat System vendor.
4.6 Scheduled Delivery Prior to Collins Withdrawal
The majority of the MOTS Combat System suppliers could deliver their products within timeframes that would not be schedule drivers in the Future Submarine program.
Any changes required to be implemented before the lead delivery of the combat system would present schedule risk, however, this would still be manageable provided sufficient lead time for first system delivery was scheduled from the contract start date. The quantity of lead time deemed sufficient is beyond the scope of this study.
The schedule risk lies less in the contractor's abilities and is more concentrated in the ability of Defence and Government to baseline the FSM Combat System capability and respective requirements to enable a contract to be negotiated and established in time.
4.7 Technical Risk
2.
required to be implemented. The risks are represented as {SCENARI01/SCENARI02}.
As discussed previously, the risk assessments offered below should be considered to be relative risks assessed against each respondent rather than an absolute risk level.
4.7.1 Capability Shortfall
Rl!S'fRIETff18 cm u rnnSI , I u r iii IF••• Iii
RBI RIC I HD LClOilOiEICLIHL IIGSCOIQI IDEIGCE
4.7.2 Through Life Support
Through Life Support of current RAN weapons is not considered for each respondent as these systems are maintained outside of the Combat System supplier. Consequently, through life support for these systems is rated NA for each supplier.
PEFWPli•• - ·- - . - -
CONilOIERLIADllG-COIGFIDEIGCE
4.7.3 Through Life Evolution
The Through Life Evolution path for Scenario 2 for all Combat System suppliers is considered high risk for all supplier , due to potential data access ~ssues and the willingness to share c for continued integration and o ration of these s stems
4.7.4 Support Infrastructure
For scenario 2, the ability to incorporate USN technology into the Combat System supplier's support system is considered to be a high risk activity, due to the requirement to obtain approval for the release of data that would be r uired to u ade and maintain the support
PFfrmqypp SQ) u rnpcz a I RI QQ) ID1D£)1QE
RWllEICIEB LCiiilJIEillll iE ii 4 @Ct 41 WEI 4 Ct
4.7.5 Platform Impacts
The platform impacts of integrating USN technologies would need to be addressed outside of the Combat System supplier. The requirement to manage the integration of these technologies would need to be transferred to the Commonwealth and the risk is regarded as high for scenario 2 due to the inability of the su lier to address these concerns. This is true for all the Combat S stem su liers assessed.
PFGTWOTFi' @ll§f&flsff!!I.. I IJ
EOlihhtltCLiZ 114 €6141 IBEICCC
4.7.6 Certification Risks
Certification of USN technology is not a requirement that the Combat System supplier will be capable of meeting. Completing the certification of the use of these technologies will be required of the original supplier and the certification will be held within the Commonwealth. Thus all the Combat System suppliers have been rated as NA for scenario 2.
Some upgrade of the system may be required to meet Australian legislative requirements but this is seen as low risk.
Some upgrade of the system may be required to meet Australian legislative requirements but this is seen as low risk.
Some upgrade of the system may be required to meet Australian legislative requirements but this is seen as low risk.
Some upgrade of the system may be required to meet Australian legislative requirements but this is seen as low risk.
IU!91 IllC I Eb @@lillilER@t 215 ht @!Cf 41 MEI CCL
4.8 Commercial Risk
All respondents have provided responses as to the commercial organisation and prospective industrial capabilities to provide and support a MOTS Combat System. The responses received are limited to corporate structure and costing data associated with the acquisition of the MOTS Combat System and do not address other key commercial considerations such the provision of Intellectual Property and potential conflicts of interest between suppliers.
While data related to the costs associated with the on-going development and maintenance of the capability was requested, the feedback from the respondents was that this information is extremely commercially sensitive, and not suitable for provision as part of the response to the RFI. Several respondents indicated a desire to enter into more formal arrangements for the sharing of this particularly sensitive data.
No respondent provided all data required to fully model costs associated with the through life costs of the required Combat System Capability. Costing data associated with the costs of directly supplying the Combat System to the platform builder have been provided, however, the costs associated with ongoing sustainment and development of capability and the associated facilities for the Commonwealth was not made available, as these costs are typically closely held by the various Parent Navies. Accurate costs associated with sustainment and capability development is a prime area for ongoing work related to the development of the business case for the Combat System.
The RFI assessment has been conducted against a CRM combat system context defining the capability scope and respective subsystems as well as a MOTS combat definition that requires the full scope of the context to be delivered by the respondent. All respondents are by definition combat system integrators with various levels of components produced in house and different boundaries at which they believe their "integrator" responsibilities lie. None would be able to deliver the same baseline as delivered to their parent navy as these will contain some capabilities defined especially for that navy. Export variants delivered to
RESTRIC'fED eemnwtEftCIAI::!H:f!fBMt¥lH:i@f!
ililllllf@IEB C@foffoff!ftCl>'tL ff lfst'.!@f cffll8f!f If Cf!
other international customers could also contain capabilities specifically requested by that customer.
To be competitive in the international marketplace, all the respondents have architectural and system level features that enable them to integrate customer preferred subsystems or system attributes at low cost and risk. As such, not all responsibility for decisions on subsystem choice needs to be placed on the vendor who will certainly trade off risk for capability.
The main failing of the RFI was the failure to provide an indication of the total cost of ownership for each of the Combat Systems surveyed. The RFI provided an indication of the acquisition cost of a Combat System, which the respondents were able to address. However, the ongoing maintenance of the capability, which also comprises facilities, is the responsibility of the Parent Navy and will need to be addressed separately.
Further studies will be required to establish the true through-life cost of ownership of the Combat System that includes all Parent Navy support facilities and infrastructure and any associated commercial aspects.
The Combat System RFI was delivered to potential commercial Combat System suppliers to undertake a survey of capabilities of extant Combat Systems in service around the world. SEA1000 staff conducted UNCLASSIFIED briefings to clarify any potential misunderstandings about the intent and purpose of the RFI and to receive any briefings from the commercial entities.
None of the proposed combat systems would provide an off the shelf solution that would match the existing Collins Class submarine combat system capability or requirements. Significant customisation and modifications would be required for any MOTS Combat System to meet Australian requirements.
The RFI respondents all have significant capability shortfalls in the weapons subsystem, sonar subsystem, optical subsystem, external communications subsystems and shore facilities.
A key capability is the ability to support the insertion of 3rd party technology into the Combat System. There are two parts to fully support this capability, while also ensuring the protection of the 3rd party intellectual property. The first part is the implementation of an information assurance framework is required to prevent the unauthorised leakage of information across defined boundaries. The second is the ability to insert the capability without the assistance of the Combat System Provider to maintain National Sovereignty and to protect National Security information that may not be releasable to the Provider. None of the Combat System respondents were able to fully meet both of these requirements.
I
RESTRICTED 111 II 11111 I L Ill I F JP F 7 7
ltf!B't'ltl@'t'EB @@Jlifltl8R@li 115 11 t @@if tPIBDI IEE
Maintaining a cost effective capability edge will not be sustainable without considerable indi enous Parent Nav resources and fundin .
Export variants delivered to other international customers could also contain capabilities that have been degraded from those supplied to the Parent Navy, as well as specifically requested by that customer. To be competitive in the international marketplace, all the respondents have architectural and system level features that enable them to integrate customer preferred subsystems or system attributes at low cost and risk.
£0) , , ' FRGZ 0 I 1 ) 1 CQ\ WIQFb ICF
- - - - -6. References
[1] MOTS Combat System RFI Assessment and Analysis Plan, vl .O. [2] MOTS Combat System RFI Evaluation Methodology, vl.O. (3] Collins Class Combat System Function and Performance Specification (SMCSP00107),
vl.0, 11 December 2011. (4] Collins Combat System Interface Requirement Specification (RCS0100), v4.0, 02
November 2011. (5] Collins Engagement Sub-System System Specification (SMCSP00028), v3.0, 02
November 2011. (6] Collins Mission System - System Specification (RCS0099), v7.0, 02 November 2011. [7] Collins Navigation Sub-System System Specification (SMCSP00013), v6.0, 02 November
2011. [8] Collins Support Sub-System System Specification (SMCSP00030), v4.0, 02 November
2011. [9] Collins Tactical Sub-System System Specification (SMCSP00026), v4.0, 02 November
2011. (10] SEA 1000 Future Submarine RFI Statement of Work (SEAl000-2012-001) (11] SEA 1000 Mfuute - Combat System RFI Overseas Visit Report (12] DMOFSP-RFI0230-2012 MCSRSOW A-C Analysis (13] Defence White Paper 2009. [14] Defence Capability Plan 2009. [15] T.Mansell and B.Clark, et al., "SEA1000 Combat Systems Options Report (U)", DSTO
TR-2333 [16] Submissions from respondents for DMOFSP-RFI0230-2012 (ORMS Folder - fAB2052591
ll@M'RIE I f!1' @@hHofl!!ft@IAE H. @@I tPie@I I@@
It& I MC I Lb SGJ u fill ii I L It J ii') Jiilli'li) Ji'li
Appendix A Detailed Traffic Light Report
The detailed traffic light report is listed below in Table 6, Table 7, Table 8, Table 9 and Table 10 and is broken down according to function. Please see Table 5 below for a description of the traffic lights.
For noting, the assessments listed below are a direct comparison against the CRM. If there is no capability available for the CRM, the response from the Combat System Respondents is automatically marked as equivalent to the CRM.
Acceptable Lower Capability
Table 5: Legend for Detailed Traffic Light Report
Evaluation Section 1 Combat System
1.0 System Integration
1.1.1 Own ship Signature Monitoring
1.1.1.1 Below Water Signature
1.1.1.2 Above Water Signature
1.1.2 Environment sensing
1.1.2.1 Above Water Environment
1.1.2.2 Below Water Environment
1.1.3 Navigation
1.1.3.1 Sensors
1.1.3.2 ECDIS
1.1.3.3 Bridge Navigation
1.1.3.4 Navigation Plans
1.1.3.5 Fixing Methods
1.1.4 Detection & Tracking
1.1.4.1 Displays & Controls
1.1.4.2 Acoustic
1.1.4.2.1 Broadband Passive Sonar
1.1.4.2.2 Demon Processing
1.1.4.2.3 Narrowband Passive Sonar
1.1.4.2.4 Intercept Sonar
1.1.4.2.5 Active Sonar
tlJiQijiU iijJili' GS' a 'FPG1
I I p z sea mrnm zgp
Rl!ISllUCIES lll!U!L 2112£6 11 I 66! 12 23£! :CL
Evaluation Section 1.1.4.2.6 Sonar Audio
1.1.4.2.7 Mine Avoidance and Bottom Navigation Sonar
Description The ability of the target CS to be able to be brought to an acceptable level of capability. The risk is characterised by scope of change required, the ability of the CS architecture to support the change, the cost of the change and the ability to implement the upgraded baseline within the schedule.
The ability of the target CS to be able to be support through life. The risk is characterised by the ability of the CS design and construction to support maintenance, defect rectification, tolerance to failure, adequate supply lines and clear strategies to manage obsolescence. The ability of the target CS to be able to be evolved to higher levels of capability through life. The risk is characterised by the ability of the CS architecture to support the periodic change, the ability to support multi-vendor components and the ability to support sensitive RAN and other allies' information and components.
The ability of the target CS support systems to enable effective training in parallel with development activities associated with defect investigations and system testing of upgraded baselines. The risk is characterised by the fidelity and scope of simulation systems, scope of simulated systems, independence of each type of facility, and ability to effectively manage and support multiple system baselines. The impact of the footprint of the target CS on the platform (space, weight, power and cooling). The risk is characterised by margins available for future growth when compared to a nominal CS allocation derived from Collins and MOTS platform contenders.
The ability of the target CS to be able to be brought to an acceptable level of compliance with RAN certification requirements. The risk is characterised by scope of change required, the ability of the CS architecture to support the change, the cost of the change and the ability to implement the upgraded baseline within the schedule.
co> " mnrn · 1 n 1 cournem mr
.
COMMERCIAL RISKS Risk Area CS Prime Capability
Certification
Through Life Support
Through Life Evolution
-
@etelhtl!!l\@!ML II. @et fftel!JI f@!l!J
.,
Description The ability of the target CS supplier to undertake the prime and integrator roles. This risk is characterised by the supplier's experience in managing development and production of the full system context, supplier's experience in working with shipbuilders in integrating the full system context and the supplier's experience in managing the acceptance of systems into service. The ability of the target CS supplier to effectively manage the certification of delivered systems at the various development stages. This risk is characterised by the supplier's experience in managing certification of the full system context over multiple vendors, the supplier's engineering and quality systems capability. The ability of the target CS supplier to undertake through life support of full system context. This risk is characterised by the supplier's experience in managing through life support including repair and supply lines, the experience in providing and maintaining the full range of support facilities, and the experience in managing obsolescence.
The ability of the target CS supplier to undertake capability enhancement of full system context. This risk is characterised by the supplier's experience in managing staged evolution programs, the experience in managing IP (customer, commercial third party and foreign government) and the ability to establish effective "Australian Only" workforce to implement "sensitive" developments.
