m
c
m C
m I 1 W I
H
I I II
I
E 3 m
II E R
E
R
Measuring the reliability of MOD equipment
Presented at 16 ISMOR, 31 Angust - 3 September 1999
DERA/CDA/HLS/CP990133/1 .O
Cover + vi + 28 pages
September 1999
C T J Allard
This document is subject to the release conditions printed on the reverse of this page
D€RA
DERA is an Agency of the UK Ministry of Defence
0 Crown copyright 1999
Defence Evaluation and Research Agency UK
Approval for wider use of releases must be sought from:
Intellectual Property Department Defence Evaluation and Research Agency
D E W Farnborough Farnborough, Hampshire GU14 OM
Authorisation
Prepared by Dr C T J Allard Title Lead Analyst
Signature C' C,P4 Date 2o/d?Y Location CDA (High Level Studies)
Authorised by DrDJHarris Title B M C D A (High Level Studies)
Record of changes
Issue
Original
Paper for ISMOR
Date Detail of Changes
March 1999 Issued as DERA/CDNHLS/CP990133/1 .O
September Minor editing to allow UNLIMITED distribution 1999
11
Executive summary
Introduction
The UK MOD needs to know whether the reliability of its equipment is improving or deteriorating. There is currently no agreed method for doing this. Estimates were made in the 1980s which put the cost of unreliability to MOD at over €lbn. However, the basis for these estimates is unknown. The customer for this study (UK Defence Procurement Agency) has identified the definition of a metric to measure unreliability as the first step in determining whether reliability is improving or deteriorating.
A metric of unreliability
The UK’s Defence Mission requires the MOD to ‘generate modem battle-winning forces and other defence capabilities’. If we seek to measure equipment reliability at the highest level of aggregation within the MOD, then such a measurement should be expressed in terms of its effects on the peacetime provision of operational capability. The metric should represent the annual cost of equipment unreliability as a percentage of the defence budget. On this basis the following generic metric is proposed:
Cost of unreliability = Cost of maintenance + Equipment cost * (1 - Availability * Mission reliability)
where,
- Cost of maintenance is all maintenance costs (both scheduled and unscheduled) caused by equipment unreliability; Availability is defined as the proportion of MOD equipment which is able to cany out the role to which it is assigned; Mission reliability is the proportion of missions which are not aborted due to equipment unreliability during an operation; Equipment cost is the annualised cost of equipment;
-
-
-
The metric is a very broad representation of unreliability. Therefore it will have many potential uses and users, either directly from the measure itself or from what is learnt during the measurement process:
- political: answering the question ‘how much does equipment unreliability cost us?’, overall, for each service and for each category of cost; problem areas: highlighting which areas are increasing or decreasing in cost, and identifying where more detailed investigation is needed; policy development: suggesting areas where appropriate organisations (Systems area, Defence Procurement Agency, Chief of Defence Logistics, Service Chiefs, etc.) should consider policy changes; policy effect: possibly measuring the effects on reliability of changing procurement policy (e.g. smart procurement) or logistics procedures.
-
-
-
... 111
Executive summary 1 m a
Conclusions
Any meaningful comparison of previous estimates of the cost of equipment unreliability is likely to be impossible. The emphasis should be on capturing the main drivers consistently over a series of years, in order to identify trends overall and by service andor cost factor. This does not require a high level of accuracy, and where possible existing aggregated data should be used.
The metric meets the customer’s requirements:
- - - - -
-
it measures the cost of unreliability and includes a measure of availability; it will be consistent fiom one year to the next, allowing trends to be identified; it is transparent, and the individual components are identifiable; it is simple in principle; it is justified by reasoned analysis and has ‘engineering credibility’; it should be practical to measure.
Recommendations
A study should be carried out on the availability of data to support the measurement of the cost of unreliability using the metric proposed in this paper. This would include a comprehensive survey of potential data sources and the evaluation of the quality of that data in terms of coverage, accuracy and bias. The study would identify shortfalls in the data and recommend ways of coping with them, such as modifications to the metric or changes or additions to existing data collection processes.
A follow-on study should be carried out (possibly in parallel to the data availability study) to investigate further some of the theoretical issues identified in this report. These two studies would result in a detailed metric that could be applied in practice to calculate the cost of equipment unreliability, taking into account both theoretical and practical considerations.
Once a detailed metric has been defined, then a pilot study or feasibility study should be conducted. This would involve calculating the cost of equipment unreliability, using the metric, for a particular force element (possibly a single equipment if the costs can be identified).
All three studies should make use of existing models, such as the Force Planning Models and relevant results fiom other high level studies such as Military Capability Assessment. Since a cost metric is proposed, the CDA Cost Cell would be well placed to co-ordinate this work.
m
Further work should depend on the results of these studies.
iv
8 I E m m w
V
I
I
List of contents
Authorisation
Record of changes
Executive summary
1 1.1 1.2 1.3
2 2.1 2.2 2.3
3 3.1 3.2 3.3 3.4
4 4.1 4.2 4.3
5 5.1 5.2
6
7
8
A
B
Introduction Background Aim Scope
Towards a metric of unreliability Basic principles A high level cost model Definition of terms
Practical considerations Data requirements Data availability Implications Statistical issues
Discussion Interpretation of the metric Alternative approaches Potential uses and other issues
Conclusions Firm conclusions Areas where further work is required
Recommendations
References
List of abbreviations
Statement of requirement
Summary of suggested approaches
Report documentation page
1
ii
iii
1 1 1 2
3 3 4 5
7 7 8
10 11
12 12 13 14
16 16 17
18
19
20
21
22
27
vi
I
This page is intentionally blank
1.
1.1
1.1.1
1.1.2
1.1.3
1.1.4
1.2
1.2.1
1.2.2
Introduction
Background
The UK Ministry of Defence (MOD) needs to know whether the reliability of its equipment is improving or deteriorating. There is currently no agreed method for doing this.
In 1989 the National Audit Office (NAO) reported [ 11,
‘special studies by the Air Force Department and MoD(PE) suggest that unscheduled repair and maintenance stemming from unreliability currently costs the RAF some E500m a year. Based on this, and the known reliability of Army and Navy equipment, the services estimate that unreliability adds over Elbn a year to support costs. These assessments cover the costs of service maintenance swf , spares, repairs in industry and resulting modifications.’
This figure is based on a 1983 study which estimated the cost of unreliability in the RAF to be €500m. It is not known how this was derived. A 1990 House of Commons Defence Committee (HCDC) report [2] stated a figure of E650m as the cost of unreliability to the RAF, calculated by adjusting the 1983 cost to allow for inflation. No basis has been identified for the HCDC claim that this figure,
‘represents the estimated cost of all unscheduled maintenance less unscheduled work not attributable to reliability defects plus scheduled work beyond that expected as a result of the reliability criteria sought in the Requirements.’
At the individual equipment level, there is a wide variety of measures relating to individual systems and fleets, collected by many different organisations with varying levels of fidelity and record keeping.
Aim
The customer for this study (UK Defence Procurement Agency) has identified the definition of a metric to measure unreliability as the first step in determining whether reliability is improving or deteriorating. Their aim is to define an appropriate quantitative high level metric which can be used at a ‘political’ and not just a technical level. The formal statement of requirement is reproduced in appendix A.
The customer’s stated objectives for a metric include:
- it is justified by reasoned analysis, has ‘engineering credibility’ and is practical to measure; it should ideally be related to the cost of unreliability; it should preferably incorporate a measure of availability.
- -
Page 1
Introduction
1.2.3
1.2.4
1.3
1.3.1
1.3.2
1.3.3
Page 2
Furthermore the intended use of the metric, to determine whether the reliability of MOD equipment is improving or deteriorating, suggests that any metric proposed should possess the following qualities:
- it is consistent from one year to the next, providing a sound basis for comparison and identifying trends it is transparent, allowing the components to be identified (once it is known what the overall trend is, trends in the components of a metric may be used to support policy decisions or suggest areas for further investigation); it is simple, and therefore less likely to be attacked or undermined by having the assumptions questioned.
-
-
The aim of this study was to review the potential options for a metric of reliability and to propose one or more metrics which meet the objectives stated above. The aim of this report is to define and explain the proposed metric. The other options considered are listed and briefly discussed in appendix B.
Scope
This study is limited to deriving a metric which meets the above specification. The practical issues are dealt with at a very general level, for example concentrating on the type of data which might need to be collected rather than how they should be collected. This in turn means that the metric is defined at a fairly generic level; the emphasis is on the form of the metric rather than detailed calculations.
This study identifies the issues to be resolved and problems still to be solved, and outlines how these might addressed. Thus it should be seen as a form of ‘pre-feasibility study’ or ‘concept of analysis’.
The report is organised as follows.
- In section 2 we consider the principles of measuring equipment reliability and derive a generic metric. In section 3 we examine the practical issues involved in using such a metric to measure reliability. In section 4 we discuss the implications of the proposed metric and briefly review some alternative approaches and variations. Section 5 lists the conclusions and identifies the key issues still to be resolved. Section 6 gives the recommendations, proposing a way ahead.
-
-
- -
I
m m I I m
2.
2.1
2.1.1
2.1.2
2.1.3
2.1.4
Towards a metric of unreliability
In this section we consider how equipment reliability should be measured, with a view to identifying trends over time. Our starting point is the requirement for the MOD to generate a defence capability. We derive a metric which incorporates the key costs of unreliability in the context of providing that capability: maintenance, unavailability and mission unreliability. Unless otherwise stated, the concepts discussed in this section are generic and are intended to be applicable at all levels, from the individual sub-system up to the MOD as a whole.
Basic principles
The Defence Mission requires the MOD to ‘generate modem battle-winning forces and other defence capabilities’. In order to measure equipment reliability at the highest level of aggregation it should be expressed in terms of its effects on the generation of operational capability. It is important here to note the distinction between maintaining, in peacetime, the ability to generate operational capability and the use of that capability for a contingency operation. In order to allow year on year comparisons we restrict our attention to the former. This may however include indirect effects such as the need to generate extra capability to allow for mission unreliability .
Equipment reliability, or rather unreliability, affects the generation of operational capability for a number of reasons. The main categories are:
- maintenance: the need to carry out maintenance to correct or prevent faults caused by unreliability imposes a substantial additional burden on the MOD in providing a capability; unavailability: unreliability leads to reduced availability of equipment and thus a lower operational capability; mission unreliability: unreliability may lead to equipment failure during a mission, thus reducing operational capability; procurement: unreliability during procurement can delay the in service date (ISD); unreliability in service might necessitate additional improvement costs; indirect: for example, equipment unreliability may hinder training, thus reducing operational capability.
-
-
-
-
Since the purpose of the metric is to measure the change in reliability over time, the requirement is to define a consistent baseline to identify trends rather than to include every possible factor. We can simplify the problem, therefore, by excluding indirect effects. This is justified because there is typically a large overlap with one or more of the other factors. For example, the effect of equipment unreliability on training is often due to unavailability; a broken item of equipment should not be counted twice simply because it is unavailable for both operations and training.
We can also exclude procurement-related effects. This is justified on grounds of principle. The delay to ISD represents a different type of unreliability and it is not clear
Page 3
Towards a metric
2.1.5
2.1.6
2.1.7
2.2
2.2.1
2.2.2
2.2.3
Page 4
f unreliability
how to make its measurement independent of the other factors; if it leads to an extended life of the equipment it was due to replace, the effects will be seen in the remaining three factors. The exclusion is also justified on pragmatic grounds. It is hard to distinguish reliability issues from other causes of delay or needs for improvement, even without the artificial nature of the equipment programme.
Of the other three factors, two - maintenance and unavailability - are very closely related. More importantly, they are negatively correlated. For a given equipment, if nothing is spent on maintenance, the level of availability will reduce over time. As more is spent on maintenance, availability improves up to a point. Thus we must include both of these in our metric in order to provide a fair comparison.
The same principle applies to scheduled (preventative) and unscheduled (corrective) maintenance. In general, increasing scheduled maintenance reduces the level of unscheduled maintenance required to achieve a given level of availability (although too much scheduled maintenance can in some circumstances increase the requirement for unscheduled maintenance). This point is particularly important given the tendency in some areas towards minimal or no scheduled maintenance. For an equipment which is very reliable, this may be the most cost effective policy, and our metric should represent this fact. Therefore, both scheduled and unscheduled maintenance should be included.
That just leaves mission unreliability. The impact may be thought of as similar to that of unavailability, the difference being that it represents equipment becoming unavailable during an operation rather than before it. It is also undoubtedly an important factor, and on these grounds it should if possible be included in the metric. However, it is a lot harder to measure than maintenance and availability, and it may not be practical to include. In particular, it may be very difficult to estimate how it changes over time. Nevertheless, in the rest of this section we assume that it is included.
A high level cost model
Having determined what to measure, the next question is how to measure it. In order to meet the objectives set out in section 1, our metric should be quantitative and one- dimensional (i.e. can be represented as a single figure). It should be measured at regular intervals and, since it includes more than one factor, these should be easy to combine and individually meaningful.
The only dimension in which it is practical to measure all three factors is monetary cost. It is also the customer’s preferred approach. Thus our metric will represent the cost of unreliability. Maintenance is naturally measured in terms of cost. Therefore we need to determine the cost of unavailability and mission unreliability.
There are two methods of costing the effect of equipment unavailability on operational capability. The first is to cost the effects on the outcome of an operation, or the political impact of the UK’s inability to carry out an elective operation, due to the shortfall in capability caused by equipment unavailability. The second is to take the reduced
2.2.4
2.2.5
2.3
2.3.1
2.3.2
2.3.3
To wards a metric of unreIiabiXty
capability as given, and instead cost the equipments which do not contribute to that capability due to their being unavailable. We should use the second method as it is more practical and less controversial. The same approach can be taken for mission unreliability, for example by costing the equipments which fail during operations due to unreliability.
All that remains is to add these costs together, making sure that we do not double count; for example, the cost of equipment designated as unavailable should not include any element of maintenance. Noting that the same principles apply at any level of aggregation, this calculation can be represented as follows:
- -
Cost of unavailability = Equipment cost * (1 - Availability) Cost of mission unreliability = Equipment cost * Availability * ( 1 - Mission reliability) Cost of unreliability = Cost of maintenance -+ Cost of unavailability -I- Cost of mission unreliability;
-
Thus our generic metric is:
Cost of unreliability = Cost of maintenance f Equipment cost * (1 -Availability * Mission reliability)
Definition of terms
There are many possible definitions for the each of the wide range of terms used in connection with reliability, and it seems as if most of them have been used at some time or other. It is therefore necessary for us to clarify what each of the terms used above means in the context of this metric.
The cost of maintenance should in principle include all costs which are caused by equipment unreliability, including:
- - -
- logistic and organisational costs.
costs of both scheduled and unscheduled maintenance; costs of 1 st, 2nd, 3rd and 4th lines of repair, including contractors’ costs; costs of spares, manpower and facilities;
Ideally it should exclude costs of maintenance carried out for reasons unrelated to reliability, such as battle damage or bird strikes. However, it may not be possible to measure the maintenance cost to this specification. Such costs may not be identifiable, contractors’ costs may be hidden and the cost of large permanent facilities such as dockyards might be better excluded on pragmatic grounds.
Page 5
To wards a metric of unti Izability
2.3.4 Unavailability may be defined as the proportion of MOD equipment which is unable to carry out the role to which it is assigned in order to provide or support operational capability. This is most easily described at the individual platform or system level. Where an equipment has more than one role, availability should be assessed against its primary role. Since unavailability is multiplied by equipment cost in the metric, an aggregate measure of unavailability is defined as an average of the unavailability of the equipments included, weighted by cost. As with maintenance, any unavailability not caused by unreliability should be excluded if possible.
2.3.5 Mission unreliability may be defined as the proportion of missions (actions, activities, functions, etc.), weighted by the cost of equipment involved, which are not carried out due to equipment unreliability during an operation. The precise interpretation will vary
. according to type of equipment and role. A fractional value may be meaningful in some cases and not in others. One approach is to measure mission unreliability at unit level (a ship, battlegroup or air squadron) as the degradation in capability due to unreliability, an approach which might also be applied to unavailability.
2.3.6 It is clear that the equipment cost must include an appropriate element of procurement cost and exclude the costs of manpower involved in maintenance. However the treatment of a platform’s crew or a system’s operators is less obvious. Where crew are tied to a specific item of equipment, they no longer contribute to operational capability if that equipment fails, and should be included in the cost. If they can use spare equipment, their cost should not be included. This will vary between equipments, and different costs may apply to unavailability and mission unreliability. For example, if a Warrior armoured vehicle is unavailable, an armoured infantry section can probably use another; if the Warrior breaks down during a mission, that option may not exist, and the capability of the section is reduced. Other costs, such as strategic lift, should also be accounted for in the cost of mission unreliability.
2.3.7 Equipment cost can be measured either in terms of equivalent annual cost (EAC), a recognised method of accounting for the full cost implications of equipment, or by depreciation in value, a method which may be thought more consistent with CAPITAL.
Page 6
3. Practical considerations
In this section we look at the practicalities of applying the generic metric defined in paragraph 2.2.5. The ideal is to use existing aggregated data, and to avoid the problem of interpreting and combining data from large numbers of individual systems. However, it is recognised that many of the required aggregate data do not exist and that much of the system level data are inaccessible. Thus any measurement of unreliability will be incomplete and inaccurate. This may not be disastrous to the identification of trends in unreliability in fbture years, but it is likely to prevent any meaningful comparison with figures from previous years.
3.1 Data requirements
3.1.1 A measurement using the metric must represent the cost of unreliability over a period of time. The obvious period of time to use is a year, since almost all relevant data that are available will be available on an annual basis, and many will be available only on an annual basis. The requirement to identify trends suggests that we should measure the annual cost of unreliability over a series of years.
3.1.2 The metric will be used at a highly aggregated level: the total cost for all MOD equipment, and probably the cost broken down by factor (maintenance, unavailability, mission unreliability) and by service. Thus the requirement is for data at this level, and wherever possible, we should seek to use existing aggregated data.
3.1.3 Given this level of aggregation, and the requirement to identify trends rather than calculate a ‘true’ figure, a high level of accuracy in measurement is not essential. Instead the emphasis should be on capturing the main dnvers for the cost of unreliability and ensuring that the data are consistent from one year to the next, allowing valid comparisons to be made.
3.1.4 The cost of maintenance is the area for which aggregated data is most likely to be accessible. It can be measured by identifying the organisations or parts of organisations whose activities are generated by equipment unreliability and adding their costs. In the Army, for example, these would include much of what is now QMG, the Land Command elements of REME and RLC activities in support of maintenance.
3.1.5 Unavailability was defined as the proportion of MOD equipment unable to carry out its primary role. This suggests availability is assessed with respect to a level and a timescale defined by the required readiness profile of the unit to which an equipment belongs. For example, the Army Plan specifies availability levels for weapons, for combat vehicles and for combat support vehicles for each of the 10 readiness states. Perfect availability would be defined by all these targets being reached and by the ability of the unit to reach 100% availability within the timescale defined by readiness state.
3.1.6 Data on equipment availability may exist at service level for some categories of equipment, but it is likely that lower level data (by fleet and/or by unit) will need to be used. If possible, direct measurements of availability should be used. Since the definition
Page 7
Practical considerations
3.1.7
3.1.8
3.1.9
3.1.10
3.2
3.2.1
3.2.2
3.2.3
Page 8
(the proportion of equipment unable to carry out its role) relates to a particular point in time, a sample of availability returns over the year is to be preferred.
If direct measurement of availability is not possible, then an altemative is to estimate it from quantities such as the mean time between failures (MTBF), mean time to repair (MTTR) or maintenance free operating period (MFOP). However, such figures may not be reliable so should be used only as a last resort.
Recall that mission unreliability is measured in terms of its peacetime effects. However, the definition of mission unreliability (the proportion of missions not carried out due to equipment unreliability during an operation) is scenario specific. Therefore the measurement of mission unreliability should in principle relate to those scenarios which are drivers for the provision of operational capability in peacetime.
Accurate estimates of mission unreliability are likely to be elusive, and a high degree of accuracy is not in any case essential. So there is little point in trying to estimate and combine measures of mission unreliability for each equipment in every relevant scenario. Instead, estimates should be made at the highest levels of aggregation that are consistent with the differences in drivers between and within each service.
The cost of equipment is used in the metric as a multiplier of the measures of unavailability and mission unreliability. Therefore, equipment costs must be measured at the same levels as these factors. Where related costs such as crew or logistic load must be included, these will also need to be identified.
Data availability
This study has not included a comprehensive survey of what data is available; such a survey is an essential next step if a metric for measuring unreliability is to be put into practice. The purpose here is to check whether the proposed metric is likely to be feasible, and to identify some of the potential problems with data availability, considering a variety of alternative sources and types of data.
There is no shortage of data collection organisations. Indeed, the multiplicity of such organisations and the varied ways in which they collect data is likely to present a problem when it comes to calculating a high level metric. For the most part, the data collected is at a very low level.
There are considerable differences between the services, both in the way in which data is collected and in the quality of that data. It is generally agreed by reliability specialists that the RAF collects the most detailed reliability data and the Army the least. However, that is a generalisation which hides differences within the services, where different data is recorded by different organisations for each category of equipment. For example, Army Air Corps reliability data collection has more in common with that done by the RAF. The information system initiatives planned by the Chief of Defence Logistics (CDL) should improve both the availability of relevant data and its coherence between the services. However it will be some time before these initiatives bear h i t .
Practical considerations
I E 8
a
3.2.4 As suggested above, the cost of maintenance is likely to be the easiest element to measure. Potential problems include:
- identifying all of the relevant parts of each organisation, and excluding those not relevant to reliability; determining the actual spend of each relevant organisation (some figures .may not be available); an alternative is to use budget figures, but these may not correspond closely to actuals; where the risk of unreliability is borne by the contractor, such as under a service level agreement, the maintenance costs are likely to be hidden; public sector comparator data might be able to provide estimates of the cost; the unwanted inclusion of the effects of operational costs; to some extent this is unavoidable (unless the operation and knock-on effects are carehlly costed), and years in which there is a major conflict (e.g. 90/91) can be expected to skew the figures.
-
-
-
3.2.5 Where maintenance cost data are unavailable, a possible alternative is to estimate the required figures using the CDA Cost Cell's Force Planning Models.
3.2.6 Availability data is collected in some form or other for major equipments in all three services. This is typically done at unit level (e.g. regiment, ship, air squadron), although some aggregation is done. For example, unit battle-winning equipment reliability returns @ERR) are collated at formation HQs.
3.2.7 There is also the potential problem of bias in reported availability figures. For example, it has been suggested that:
- some units may be tempted to report good availability to justify the amount which they spend on maintenance; others may find it in their interests to report bad availability to justify claims that they do not have enough money; and, some may not measure availability properly in the first place.
-
-
3.2.8 The ideal situation would be for each service to generate aggregated availability data with respect to the provision of operational capability. One possible example is the CinCFleet Naval Operational Management Information System for Operational Capability, currently being developed. It incorporates equipment as one of the four pillars of capability. It provides an objective and repeatable measure of operational capability using absolute standards. Equipment impacts on the operational capability if it is not fitted, is unserviceable or degraded.
3.2.9 Data on mission reliability may come from operations, exercises or historical sources. However, all of these present problems. A recent study reviewed the availability of logistics data from operations and exercises. It found that reliability, maintainability and
Page 9
Practical considerations
3.2.10
3.2.1 1
3.2.12
3.2.13
3.3
3.3.1
3.3.2
Page 10
availability data for equipment support were not available at the right level of aggregation for high level models. The availability of historical data was also found to be mixed.
The use of trials data has been suggested as an alternative, but they are not easily obtainable. This is especially true for older equipment for which the data may not apply to current reliability. They may not be comparable with more recent trials, and the equipment may have been modified since. Also, the terrain for carrying out trials is usually unrepresentative of the terrain for deployment. More importantly, the results of trials which are not run specifically to test reliability are unlikely to be valid for our purposes.
Another suggestion is to use the reliability requirements specified in Operational Requirements. One problem with this is that actual reliability may fall short of or exceed the requirements. It might be that the level of reliability stated in a requirement document is a good reflection of what is achievable (and, by extension, what is expected or observed). Alternatively, the reverse may be true, with an increased emphasis on reliability reflecting greater concern, due to observed falls in reliability. If this approach is taken, it will be necessary to validate it against observed reliability figures.
Another use of estimates, which applies to both availability and mission reliability, is to look at the assumptions made by planners in building up the quantities of equipments to be procured or in the planning of deployments. As before, these would need to be audited by looking at different sources of real data where available.
Equipment cost data are likely to be more straightforward. Once CAPITAL is set up, depreciation costs can be derived by taking the difference in equipment capital asset valuation fiom one year to the next. If EACs are required, cost models exist which can be used to generate them.
Implications
As we saw in section 1, the basis of previous estimates of the cost of unreliability have not been ascertained. If a comparison with previous years is required, then the only suggested basis is to add the cost of unscheduled maintenance attributable to reliability to the cost of additional scheduled work, as reported in [2]. Such a measurement may not be consistent with the metric suggested in this paper, and even if it is, any comparison would have to be treated with great caution. The implication is that any meaningful comparison with previous estimates is likely to be impossible. Therefore we should focus on tracking reliability in the future, building up a picture of trends over time. While any estimate will be a fairly rough one, with substantial omissions, it should be able to identify trends in reliability.
The approach discussed in this section envisages that data are collected separately for each of the services and for each contributory factor. All of these measures will be expressed as or converted to cost figures, so in principle they can be summed to produce an overall figure. However, it may be that the individual component costs are not truly comparable, for example due to the'effects of different service traditions. If this were the
Practical considerations
I.
3.3.3
3.3.4
3.4
3.4.1
3.4.2
3.4.3
3.4.4
case, some care would need to be taken in using the total figure, as it might prove to be misleading.
Ths problem is most likely with the cost of mission unreliability. There will be greater uncertainty in the estimate of this than that of the other factors. It will be nearly impossible to observe real year on year changes to the cost of mission unreliability. Thus its inclusion in the total may either swamp more subtle changes in the other factors, or act as a constant, providing no information on the trends. Therefore, further investigation is needed to determine whether it is worth including mission reliability in the metric.
,
It may be possible to influence the current methods of collecting and aggregating data to make them more useful for the measurement of the cost unreliability, while still fulfilling the existing purposes of the different data collection organisations.
Statistical issues
Given that we are talking about sampling and estimation of some parts of the metric, attention will need to be paid to the statistical aspects of the problem. Here we flag up a few of the relevant issues.
We have already accepted that there is no need for a high degree of accuracy. Therefore an estimate of the level of accuracy or inaccuracy becomes more important. It is highly desirable to provide more than a single point answer. While a mean and variance will be difficult to specify, it should be possible to provide three-point estimates for at least some of the elements.
For practical reasons it may not be possible to include all equipments in the metric; the emphasis will then be on including the large fleets and other major drivers. This will not be a major problem provided some care is taken over how the results are presented. Such a sample of equipments is biased because it concentrates on the largest items. It will not, therefore, be representative of the MOD equipment inventory as a whole. Any measurement must be explicitly limited to those equipments for which measurements have been taken. This makes it clear what any such figure represents, while not invalidating it for the purpose of identifying trends.
For availability, a sampling regime is suggested. This will be determined by the services, and thus out of the control of whoever is carrying out the measurement of reliability. Some care is needed in interpreting of such data: for example, checking that the sample is likely to be representative of the levels of availability over the year. Similar caveats apply to any other data which are estimates rather than direct observations.
Page 11
4.
4.1
4.1.1
4.1.2
4.1.3
4.1.4
4.1.5
Discussion
In this section we discuss how the metric may be interpreted and what it might tell us about various related issues. Some alternative suggestions are considered.
Interpretation of the metric
The choice of metric has a number of important implications for the way in which it is interpreted. It is not a narrowly defined measurement of reliability. One consequence of including several of the major effects of reliability is that the metric is by definition influenced by many other factors such as:
- maintainability; - dependability; - durability; - equipment complexity;
usage; logistic on-costs (e.g. transport to and fiom maintenance depots); efficiency of the repair process; obsolescence and unavailability of spares; age of equipment inventory (and the ‘bathtub’ curve for unreliability);
- -
- - - - procurement policy;
and the list goes on.
To take one example, the complexity of equipment is generally increasing (driven by the need for improved performance to meet an increasing threat). The more components a system contains, the greater the potential for failure. This has a downward impact on reliability, counteracting the improved reliability of individual components.
It has been suggested that we should try to factor these effects out of the equation: that by deducting logistic costs or dividing by the level of usage or performance we could produce a pure measure of reliability. A better approach is to accept that the metric incorporates these effects and make use of this knowledge. After all, the factors listed above are of interest to the customer, and all may be used to reduce costs.
There are two ways of interpreting these relationships. One is to accept the other factors as fixed parameters so the metric represents the cost of unreliability given those other factors. Thus a 10% increase in unreliability might be acceptable if it were the indirect result of a 100% increase in performance. The other is to use the detailed information collected in the course of measuring the cost of unreliability to identify which factors are leading to increased costs, and how these might be reduced; there are many ways of reducing the cost of unreliability which do not involve improving reliability itself.
Another considerable advantage of this approach is that we avoid having to think about all the complicated relationships between these parameters, let alone determine their
Page 12
4.1.6
4.1.7
4.1.8
4.1.9
4.2
4.2.1
4.2.2
4.2.3
Discussion
functional forms, until measurements are taken which can shed light on these relationships at a high level.
However, we should explicitly take account of change in the amount of operational capability provided (as measured by its cost) and inflation. These may be represented by the size of the defence budget. Therefore, it is suggested that the cost of unreliability is expressed as a percentage of the defence budget.
Another consequence of the metric is the lack of a baseline level of reliability. A score of zero implies perfect reliability, meaning that all equipment is maintenance free and works perfectly for the whole of its in service life. The metric measures the cost consequences of the extent to which reliability is imperfect. Since perfect reliability in this sense is unattainable, the metric does not represent the potential for savings due to improved reliability. Moreover, there is no way of defining a baseline other than zero. The best alternative is to specify a target for the cost of unreliability, although it is not clear how this should be done.
The measured cost of unreliability is likely to be a large figure (the consensus among those canvassed on this issue during the study is that a figure of several Ebn would not be surprising). This stresses the importance of how the figure is explained at a political level. In particular, it should be interpreted as an implicit cost rather than an actual spend in a particular year, rather like an accounting figure.
This metric is designed only for measurement of current cost of unreliability, not prediction of fbture cost. In other words it is not a ‘what if?’ tool.
Alternative approaches
Here we consider a number of alternative approaches, either to the metric as a whole or to its constituent parts. Previous requests for ideas on how to measure the reliability of defence equipment have elicited a variety of responses. Adding the ideas generated during the study brings the total number of proposals to almost 100. These are summarised in appendix B.
In costing the impact of equipment unavailability on training, an alternative is to cost the additional time taken to train due to unreliability (assuming that existing collective performance levels are acceptable). However, this leaves us with the problem of separating out the unavailability cost implications for training from those for other activities, such as deployments. Another way of representing this effect is take account of the need for equipment with which to train as an indirect effect on unit readiness.
This issue of what to include in equipment costs was discussed in section 2, and it was noted that the decision on whether to include crew depends on what the crew does when the equipment is unavailable. For many units, the current peacetime policy is to allocate the same number of equipments to units as there are men to operate them. This means that the manpower time is wasted while equipment is unavailable, implying that we
Page 13
Discussion
should include manpower costs. If we do this, then we need to recognise that the costs incurred are a consequence of policy and not just unreliability.
4.2.4
4.2.5
4.3
4.3.1
4.3.2
4.3.3
Page 14
One traditional approach to the measurement of reliability is that based on failure rates, including numbers of failures per system or platform over a given period, MTBF and other similar measures. While these are simple to understand and relate directly to every day experience of unreliability, it would be hard to use them to define a consistent basis for comparison. Defining a measure of cost would be hard, and any metric based on failure rates would have to rely on the collection of huge amounts of low level data. Therefore this approach is unsuitable for a high level measurement of unreliability.
Subjective methods such as user surveys have also been suggested. These are fairly easy to cany out, but there is no way to determine their accuracy. There will be a problem of interpreting trends, given that judgements will vary between people, and over time for the same person. Since objective data is available, this should be seen as a last resort. It may, however, be a useful way of identifying reliability issues at a low level, and could be benchmarked against an objective metric.
Potential uses and other issues
The metric is a very broad representation of unreliability. There are many potential uses and users, either of the measure itself or of what is learnt during the measurement process:
- political: answering the question ‘how much does equipment unreliability cost us?’, overall, for each service and for each category of cost; problem areas: highlighting which areas are increasing or decreasing in cost, and identifying where more detailed investigation is needed; policy development: suggesting areas where appropriate organisations (Systems area, Defence Procurement Agency, CDL, Service Chiefs, etc.) should consider policy changes; policy effect: possibly measuring the effects on reliability of changing procurement policy (e.g. smart procurement) or logistics procedures.
-
-
-
The different parts of the metric are measured separately, and are of interest in themselves. If we are only interested in the cost of unreliability up to the point of deployment then we could leave out the Mission unreZiabiEity term. It may be desirable to identify separately the 3rd and 4th level maintenance facilities ‘deep maintenance’ (e.g. dockyards), as these represent substantial overheads which respond more slowly to changes in requirements for maintenance due to unreliability.
The measurement process may be a source of information on other questions. For example, we may wish to examine the optimal balance between preventive and corrective maintenance. One possibility is to compare the maintenance regimes at similar units with different readiness requirements. It may also be possible to extend the measurement
Discussion
process in order to assess other relationships such as those between reliability, maintainability and availability.
4.3.4 One issue to consider relates to the problem, identified in section 3, of hidden maintenance costs under a service level agreement or public finance initiative (PFI) agreement. It raises the question of whether they should count as ‘procurement’ costs or unreliability costs. This will become more important as PFI agreements become more common.
4.3.5 Another problem raised is the knock-on effect of unavailability of mission critical equipment. This may result in a lot of other (hctioning) equipment not being used (if a ship’s entire navigation system fails, the ship will not put to sea). This effect is implicitly included in the measurement of mission unreliability; should it not be included in the measurement of unavailability as well? This further strengthens the case for assessing availability at unit level.
4.3.6 Even a unit-based approach does not cover the case when a front line unit cannot be transported into battle due to unreliability of equipment in the logistic chain (usually defined as part of another unit). Such a case is very difficult to assess, so we should probably draw the line here and exclude this possibility.
4.3.7 We should be aware that current perceptions of unreliability are likely to affect any metric whether we want them to or not. For example, it is the perception of unreliability, rather than unreliability per se, which - along with the expectation of attrition - drives the deployment of large quantities of additional equipment and spares for the most important operations (e.g. Operation Granby). Is the resulting figure still a fair and/or reasonable measure of the cost of mission unreliability? Even if most of the spares are not required, the decision may still have been justified in terms of providing a safety margin.
4.3.8 This is related to the idea that defence spending can be considered a form of insurance. The requirement is then no longer expressed in terms of an average level of operational capability over time, but a guaranteed minimum (or in practice something like a 95% confidence lower bound). This might impact on the measurement of availability and mission reliability; instead of taking averages, we would need to take the 5th percentile of the relevant distribution (e.g. based on a sample of availability returns over time). One problem with this approach is that it makes the implicit assumption that no additional advantage is gained by the fact that this level of capability is often exceeded in practice. Such an assumption depends in turn on the role to which a unit is allocated.
Page 15
5.
5.1
5.1.1
5.1.2
5.1.3
5.1.4
Page 16
Conclusions
This section lists the conclusions identified in this report. Firm conclusions are given first, followed by an indication of the areas where further work is required.
Firm conclusions
Any meaningful comparison of previous estimates of the cost of equipment unreliability is likely to be impossible. The emphasis should be on capturing the main drivers consistently over a series of years, in order to identify trends overall and by service and/or cost factor. This does not require a high level of accuracy, and where possible existing aggregated data should be used.
An approach based on failure rates is unsuitable for a high level measurement of equipment reliability. Reliability should be measured in terms of its effects on the peacetime provision of operational capability. The metric should represent the annual cost of equipment unreliability as a percentage of the defence budget, and include the following factors:
-
- the cost of unavailability. the cost of maintenance (both scheduled and unscheduled); and,
If possible it should include the cost of mission unreliability. These factors are combined in the following generic metric:
Cost of unreliability = Cost of maintenance + Equipment cost * (1 - Availability * Mission reliability)
The cost of maintenance is defined to be all maintenance costs caused by equipment unreliability. Unavailability is defined as the proportion of MOD equipment which is unable to carry out the role to which it is assigned, according to the required readiness profile of the unit to which the equipment belongs. Mission unreliability is defined as the proportion of missions which are not carried out due to equipment unreliability during an operation, as represented by the scenario(s) which are drivers for the provision of operational capability. Equipment cost is defined on an annualised basis.
The metric meets the requirements listed in section 1.2.
- - - - - -
it measures the cost of unreliability and includes a measure of availability; it will be consistent from one year to the next, allowing trends to be identified; it is transparent, and the individual components are identifiable; it is simple in principle; it is justified by reasoned analysis and has ‘engineering credibility’; it should be practical to measure.
I I I I B I I I I I I I
I
I
I 8 8 8 8 8 m 8 1 8 8 I I U I 8 U
5.1.5
5.2
5.2.1
5.2.2
5.2.3
5.2.4
5.2.5
5.2.6
Conclusions
The metric is influenced by many factors other than pure reliability, such as maintainability, usage and procurement policies. This is an advantage as it provides additional information to the user. However, it also means that the metric includes much more than is usually thought of as the cost of unreliability, with the measurement likely to come to several Ebn. The potential for savings due to improved reliability will be much lower than this figure.
Areas where further work is required
A comprehensive survey of data availability is an essential next step before the measurement of unreliability can be put into practice. This would include the type of data collected by other organisations and its suitability for this purpose. Consideration could be given as to what changes in procedure might be beneficial.
The practicalities and likely benefits of including the cost of mission unreliability in the metric are uncertain. Further investigation is required to determine whether it should be included.
More work is needed to decide how equipment costs should be calculated, including:
- - -
whether they should be on an EAC or a depreciation basis; whether crew costs should be included, and in which cases; what other costs should be included (for example, the knock-on cost effects of unavailability of mission critical equipment).
Where the cost of maintenance is borne by a contractor, some thought needs to be given to the circumstances under which those costs should be included in the metric. For example, if the maintenance is part of a PFI contract (say, to provide a certain number of available platforms), do the costs count as maintenance costs, thus contributing to unreliability costs, or as procurement costs? The costs may be hidden, so we may not have a choice.
A number of statistical issues will need to be addressed, including:
- -
how to indicate the accuracy of an estimate of the cost of unreliability; the extent to which a sample of equipments is representative of MOD equipments as a whole; the extent to which snapshots of availability are representative of availability over time; the biases which may be present in existing data.
-
-
The insurance principle is a fundamental concept, deserving of further consideration. This is related to how we deal with effects which are due to the perceptions or expectations of unreliability, rather than to unreliability itself.
Page 17
8
Recommendations
This section takes the form of a list of proposals for further work. These cover:
- - -
the availability and applicability of data, and the practical aspects of using them; further consideration of some theoretical aspects of the problem; and, a pilot study to estimate the cost of unreliability.
A study should be carried out on the availability of data to support the measurement of the cost of unreliability using the metric proposed in this paper. This would include a comprehensive survey of potential data sources and the evaluation of the quality of that data in terms of coverage, accuracy and bias. Other issues such as the compatibility of data from different sources, the level of aggregation required for the metric and sampling and other statistical issues would be covered. The study would identify shortfalls in the data and recommend ways of coping with them, such as modifications to the metric or changes or additions to existing data collection processes.
A follow-on study should be carried out (possibly in parallel to the data availability study) to investigate further some of the theoretical issues identified in this report. Areas covered would include the question of whether mission unreliability should be incorporated, the method of calculation of equipment cost, the treatment of contractors’ maintenance costs and the insurance principle. These two studies would result in a detailed metric that could be applied in practice to calculate the cost of equipment unreliability, taking into account both theoretical and practical considerations.
Once a detailed metric has been defined, then a pilot study or feasibility study should be conducted. The study would attempt to calculate the cost of equipment unreliability, using the metric, for a particular force element (possibly a single equipment if the costs can be identified).This would involve the collection of real data for as many of the inputs to the metric as possible (if a feasibility study were chosen, then some of the data might be estimated using judgement).
All three studies should make use of existing models, such as the Force Planning Models and relevant results fiom other high level studies such as Military Capability Assessment. Since a cost metric is proposed, the CDA Cost Cell would be well placed to co-ordinate this work.
I n I I 8 8 8 8
8 I I 8
8 8 8 8 8 m 8
Further work should depend on the results of these studies.
Page 18
7. References
1.
2.
National Audit Office, ‘Ministry of Defence: reliability and maintainability of defence equipment’, 1988-89 HC 173, February 1989 House of Commons Defence Committee, ‘Fourth report $-om the Defence Committee Session 1989-90: The reliability and maintainability of defence equipment ’, HMSO HOC Paper (1 989-90) No 40, 1990
Page 19
I
8. List of abbreviations
BERR CDA CDL CinCFleet DERA DPA EAC HCDC HQ ISD MFOP MOD MTBF MTTR NAO
Battlefield Equipment Reliability Returns Centre for Defence Analysis Chief of Defence Logistics Commander in Chief Fleet Defence Evaluation and Research Agency Defence Procurement Agency Equivalent Annual Cost House of Commons Defence Committee Headquarters In-service Date Maintenance Free Operating Period Ministry of Defence Mean Time Between Failures Mean Time to Repair National Audit Office
NOMIS(0C) Naval Operations Management Information System (for Operational Capability) PE Procurement Executive PFI Private Finance Initiative QMG Quarter Master General RAF Royal Air Force REME RLC Royal Logistics Corps
Royal Electrical and Mechanical Engineers
Page 20
8 8 8 8 8 8
8 8 I m I 8 m
8 8 m
I 8 m I 8 m I I 8
A Statement of requirement
The requirement is to:
a.
b. c.
d.
recommend metric(s) which could best describe whether the reliability of MoD's equipment is improving or deteriorating; provide a report including supporting analysis of the various options considered; identify any special constraints or measures to be observed in practically applying the proposed metric(s); determine the viability of calculating the cost to the MOD of unreliability of service equipment.
The task should encompass a flee-thinking and wide-ranging review of potential options. The solution should be framed in terms of one or more metrics. While the metric(s) could be framed in qualitative or quantitative terms, the latter would be a preferred option. Furthermore, the proposed metric(s) could indicate directly a change in levels of reliability (for instance, an MTBF based parameter) or be an indirect, but nevertheless salient, measure (for instance a parameter based on changes to support costs). The metrics should take account of all relevant factors, in particular:
a. b.
the practical viability of applying the metric(s) including the availability of any data required; the context in which the metric(s) will be employed: specifically, while the basis of the metric(s) must be coherent and credible, the primary use of this information will be in the context of politicalhigh level decision making rather than in support of a detailed engineering study; the breadth of equipment types employed across the three services c.
m
I I I 8 Page 21
B Summary of suggested approaches
Previous requests for ideas on how to measure reliability have elicited a variety of responses. Together with suggestions made during the study, these total just under 100. This section summarises and discusses the merits of these ideas under a number of broad categories:
- failure rate; - maintenance or support cost; - availability; - capability; - delay to ISD; - subjective measures; - compound metrics; - other suggestions.
Some of the suggestions are summarised as lists of numerators, denominators and comparators. Where this is done, a metric could in general comprise almost any combination of these factors.
Failure rate
Numerators: number of faults, failures, incident reports, operational defects, incidents of scheduled or unscheduled maintenance.
Denominators: time, number of equipments/systems/platforms, usage (distance travelled, time in use, time deployed, environmental stress history, fuel consumption), number of users.
Comparators: target, benchmark or required reliability, contractor’s plans, equivalent previous system.
Advantages: simple to understand, relates directly to every day experience of unreliability, could be used to compare new equipments with their predecessors; suitable for a low level assessment of changes in unreliability.
Disadvantages: only a limited measure, not straightforward to cost and only representing part of the cost of unreliability - e.g. not preventative maintenance, may be hard to define a consistent basis for comparison (either over time or between services), data availability is likely to be a problem.
Verdict: unsuitable for a high level measurement of unreliability.
Maintenance or support cost
Numerators: (total) cost of: support, unscheduled and/or scheduled maintenance, contractor maintenance, spares and/or repairs, manpower effort, ownership, lifecycle, operating costs of maintenance/support organisations, cost of ownership of logistic fleet.
Page 22
I I a
8 1
8 m
Summary of suggested approaches
Denominators: equipment purchase cost, whole life cost, availability, time, defence budget, defence effort, usage, total service manpower cost/numbers, number of front line platforms, total value of inventory.
Comparators: equivalent previous system.
Advantages: simple to interpret, high level budget data likely to be available, direct measurement of cost, straightforward to compare over time (and to some extent across services), similar basis to that used for previous study.
Disadvantages: only forms part of the cost of unreliability, may be difficult to identify costs below high level budget headings so reducing accuracy.
Verdict: include maintenance as a measure of cost, probably as part of a broader metric.
Availability
MTBFMTTR; MTBF/(MTBF + MTTR); MFOP; planned vs. achieved tasking; ratio of operational to maintenance man hours; proportion of time spent training (rather than waiting for equipment to be fixed); operational time lost on corrective maintenance; proportion of equipment available (averaged over a year); cost of shortfall in availability.
Advantages: represents the impact of reliability (and maintainability) on availability of equipment, may be related to cost indirectly.
Disadvantages: harder to interpret, data may be patchy/inconsistent, varies according to maintenance costs, as for previous measures only captures part of the answer.
Verdict: include availability in the metric, convert to cost if possible.
Capability
Force generation metrics: number of equipments required to be purchased to get X working equipments in the hands of operators; cost of deployment per effective equipment deployed; time and cost to upgrade the readiness of a force in transition to war; compare proportion of planned readiness profile achieved against maintenance cost
Mission reliability metrics: mission abort rates due to technical causes; frequency andor extent of failure to achieve required objectives due to loss of equipment operational capability, its cost and implications; failure to complete mission; opportunity cost (the need for platfondelement X to cover for the expected degradation of platfodelement Y); logistics burden (taking into account the vulnerability of logistic support elements).
Overall capability metrics: compare cost of achieving capability for a specific operation against cost if reliability were perfect; cost of unreliability for 100 roles; total cost to perform a given task; cost to achieve constant effectiveness; effect on battle outcome of the loss of force capability fiom system degradation.
Page 23
Summary of suggested approaches
Advantages: represents the impact of reliability on the outputs of defence, includes all the effects of unreliability, can be converted to a cost basis.
Disadvantages: diflicult to assess, problems with other reasons for mission failure, may need to take into account increased performance and threat - several steps up in complexity, problem with aggregating across defence roles.
Verdict: include capability if a simple measure can be found; exclude considerations of changes in performance and threat.
Delay to ISD
Metrics: change in net present value and/or cost of extra maintenance of unreplaced in-service equipment (due to unreliability).
Comments: Delay to ISD represents a different type of unreliability, and it is not clear how the costs should be measured (in theory, a delayed ISD will affect all of the above measures). Furthermore, it is hard to distinguish reliability issues from other causes of delay, let alone the artificial nature of the equipment programme. Therefore, exclude this aspect.
Subjective measures
Ask the user/maintainer/operator who depends on it if new equipment is more reliable/do they trust it to work in times of conflict; number of complaints/other anecdotal evidence; educated guess.
Comments: This is fairly easy to carry out, but there is no way to determine the accuracy of this measure of reliability. It is also unlikely to be a consistent measure given that judgements will vary between people, and over time for the same person. Given that objective data is available, this should be seen as a last resort. It may, however, be a useful way of identifying reliability issues at a low level and could be benchmarked against an objective metric.
Verdict: not recommended as a subjective metric does not meet the requirements.
Compound metrics
A number of suggestions combine two or more of the above categories. These include:
- -
Combination of maintenance cost, unavailability of equipment and missions impaired; cost of maintenance + cost of extra equipment procured (a hybrid of manpower organisation costs and key equipment/platform drivers); might also include the effects on battle outcome; whole life cost of unreliability per equipment. -
Advantages: allows the inclusion of all relevant aspects of the problem, preferably using sub-metrics which are themselves meaningful and useful, thus if it can be done will answer the question in full.
Disadvantages: more effort involved in data collection than for a single measure, may be difficult to combine measures in a consistent way, interpretation becomes harder.
Page 24
8 i
m m m 8 m I m 8
8 I I 8 8 8 I 8 I
m W
I 8 8
8 m
w I
m 8
I m I I m 8 8 m m -m I m I 8 m
m B
m
m m
m m
I
111
Summary of suggested approaches
Verdict: since none of the individual measures provides anything like the complete answer, a compound metric is recommended.
Other suggestions
Other suggestions made include:
- - - - - - -
average annual number of replenishment spares per item of apital war fighting equipment; ratio of number of maintainers to number of operational platforms; ratio of number of 1st line engineers to flying hours; number of availability, reliability and maintainability specialists; increase in equipment lifetime (durability); proportion of contractors willing to offer long term warranties on their equipment; spares holdings related to mission fleets usage.
Comments: These measures are all likely to be correlated with reliability, however the strength of correlation is unknown. They may be more closely related to predicted or perceived reliability. They may be used to gain a quick indication of the trend, or as other contributor factors, but they do not meet the customer’s requirements.
Page 25
Summary of suggested approaches
This page is intentionally blank
I I
Page 26
m I m
m m 8 B
m 8 B m 8
m 8 8 8 B m 8 8 I B
m 8 H m
m 8
I
': I I I I
m
I m m
I m m
5. Report Classification and Caveats in use:
Unclassified
I I
6. Date written: Pagination: References:
Sept 1999 Cover + vi + 28 pages 2
Report documentation page
1 Ob. Abstract classification: Unclassified
1. Originator's report number: DERNCDNHLSICP990133/1 .O
FORM MEETS DRlC 1000 ISSUE 5
~ ~ _ _ _ _ _ _ ~
2. Originator's Name and Location: Dr C T J Allard
CDA (HLS)
DERA Farnborough
3. MOD Contract number and period covered: DPNBCP98102 FY98/99
4. MOD Sponsor's Name and Location: Defence Procurement Agency
MOD Abbey Wood
7a. Report Title: Measuring the reliability of MOD equipment
~ _ _ _ _ _ _ ~~~ ~
7b. Conference details: Presented at ISMOR 16, 31 August - 3 September 1999
Note: this report is adapted from the original customer report, DERA/CDA/HLS/CR990048/1 .O
7c. Title classification: Unclassified
8. Author:
9. Descriptors / Key words:
Dr C T J Allard
availability, equipment, maintenance, operational capability, reliability
1 Oa. Abstract.
The MOD needs to know whether the reliability of its equipment is improving or deteriorating. This report defines a metric for the cost of unreliability which would allow such trends to be identified. The proposed metric includes three factors: the cost of maintenance (both scheduled and unscheduled); the cost of unavailability; and the cost of mission unreliability. The practicalities of the metric and its implications are discussed, and a way ahead is proposed.
Page 27
I
Page 28
I
This page is intentionally blank
