WORK SAMPLE:
Safety Related Control System Audit
The following pages contain the safety related extracts from an audit report which examined a complete plant control system.
Client and Supplier have had a troubled contractual relationship, and Client is experiencing significant operational issues with the delivered plant and control systems.
All identifications have been removed from the extracts owing to the confidential nature of the report.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 11 of 90
2.1.2 Review of the Functional Safety Engineering Process and Deliverables
The review of the Functional Safety Engineering Process and Deliverables consisted of
reviewing information gathered on site and the available project engineering documentation
obtained from Client including design documents and QA/QC documentation. This
documentation was generally received in softcopy format. It is noted that other documentation
may exist however it was not practical or possible to obtain any additional documentation
during the timeframe of the Audit.
The review was performed with reference to AS4024.1503:2014 in accordance with email
agreements between Client and Supplier.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 19 of 90
3 Executive Summary
The safety PLC application program appears to be well structured and the coding appears to
be of a high standard. However the safety integrity achieved by software can only be verified
from a review of the software development process, and no documented evidence of the
software development process has been identified. In addition, there has been no
documentation presented to show the scope and the results of software functional testing
carried out.
There are some inadequacies of the hardware design of safety functions within the safety
PLC that warrant further consideration. There has been no review of maintenance
procedures at this time, but it is possible that maintenance procedures could be adjusted to
overcome the design limitations.
The functional safety validation reports are extensive but still have some limitations and
omissions that should be addressed. The hardware design inadequacies are glossed over,
whereas they should be considered in more detail, and their impact on maintenance test
requirements should be considered.
One of the most significant concerns identified is an apparent lack of Client involvement in the
risk assessments that the safety system design is based on. Client are legally responsible for
ensuring that adequate risk assessments have been carried out on machines they own and
operate, and for ensuring that the risks have been adequately mitigated.
Another major concern is that the safety PLC has been used for only a subset of the identified
safety functions, generally involving simple switch inputs. Other safety functions derived from
the operational state of the plant are implemented in the control PLC and are not adequately
protected against abuse or designated as safety functions.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 30 of 90
4.5 Safety System Engineering Process & Documentation
4.5.1 Limitation of Scope
The available Safety Instrumented System design documentation was reviewed. As stated
this review is based on the documentation available during the audit. It is possible that
additional and updated documentation may exist however it was not able to be located during
the timeframe of the audit.
4.5.2 Safety Management Plan
No documented evidence was found for the existence of a safety management plan, but it is
considered likely that Supplier would have had internal project documentation in place for this
purpose.
The contract included references to AS 61508 (functional safety) and AS 4024 (safety of
machinery). It is understood that Supplier requested acceptance of safety implementation in
accordance with AS 4024.1503:2014, which uses performance level (PL) to specify the
required safety integrity of safety functions. In the context of the ...... plant this was a
reasonable decision.
AS 4024 does not place the same up-front emphasis on functional safety management via a
documented plan and safety lifecycle as required by AS 61508. However it does specify
requirements for documentation of safety functions and does require a planned lifecycle
approach for development of the application program for a programmable electronic system.
AS 4024 emphasises risk assessment and design to mitigate the risks, but does not address
operation and maintenance of safety equipment. It is recommended that Client should
consider drafting a high level safety management plan to co-ordinate operational and
maintenance activities so that the ongoing function and safety integrity of the plant safety
systems can be monitored and audited. Guidance can be taken from AS 61508.
4.5.3 Risk Assessment Reports
There is a set of Supplier risk assessment reports, with each report identified as a part of
XXXXXX1 that covers the primary area, crushing area, scalping area and out loading area of
the ...... quarry. A separate report without document number covers the finishing area. The
reports are all marked with revision identifiers, but only the primary area report is dated within
the document.
The location of the risk assessment workshops is not identified in the reports, but from the
brief list of attendees without Client representation it appears that these may have been
conducted overseas.
The risk assessment reports are based on machine safety standards and consider mainly
personnel safety risks. They appear to be comprehensive and well presented, reflecting a
planned and managed risk assessment process. However the lack of Client representation
remains a concern since the reports appear to have formed the basis for the design of the
safety functions. These reports determine the required performance level (PL) for individual
safety related control functions.
Another risk assessment report has been prepared by Supplier using a third party Australian
risk assessment professional as the facilitator. This workshop session was carried out in
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 31 of 90
mmmm yyyy, and included Client personnel. It considered safety, financial and environmental
risks.
The yyyy risk assessment was conducted using the principles set out in risk management
standards, but does not mention or reference functional safety or machine safety standards.
It reflected a process functional safety approach where it considered the effects of equipment
malfunctions, but diverged from normal functional safety methodology in that it mixed the
failure of safety functions in with the failure of non-safety control functions. It did not address
any of the machine safety considerations for harm to personnel as set out in AS 4024, and
which are mandatory under workplace health and safety legislation.
The format of the yyyy report does not lead to determination of safety integrity requirements,
but it does note that various safety functions are SIL rated.
Of particular concern is the statement made in various places within the yyyy report that there
are SIL 4 protective functions in place for collision detection on pivoting conveyors. SIL 4
functions are extremely difficult to implement, and it could be expected that there would be
very onerous maintenance requirements. No reference to SIL 4 functions was found in other
safety documentation. It is hoped that this is a misunderstanding and should have been
Category 4 rather than SIL 4.
There are also notes that emergency stop functions are SIL 3 rated. Supplier make no claim
for SIL 3 in the validation documentation.
4.5.4 Scope and Extent of Safety Functions
One of the requirements of AS 4024:1503:2014 that is not addressed in the design
documentation available for audit is a clear delineation of the scope and extent of safety
functions.
The safety PLC handles mainly emergency stops and conveyer lanyards, hence it is clear that
everything within the safety PLC is safety-related.
However there are a number of other “safety functions” that are classified as requiring
Performance Level “c” (PLc) in the risk assessments that have been implemented in the
control PLC. These functions do not appear to be segregated and identified as safety
functions in either the PLCs or the SCADA displays.
This is an area where Client are exposed to a significant risk should an incident occur due to
a failure of the safety function implemented in the control PLC.
The following table lists the safety functions identified from the audit documentation and their
implementation in either a safety PLC or control PLC:
ID Safety Function
Highest Required Performance Level
PLC
SRCF 1a Plant E-Stop c Safety
SRCF 1b Emergency lanyards c Safety
SRCF 1c Local E-stop c Safety
SRCF 1d Global Plant E-stop c Safety
SRCF 2 Audible alarm for prestart warning. c Control
SRCF 3 Automatic stoppage upon downstream c Control
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 32 of 90
equipment fault.
SRCF 4a & 4b Material & level sensor stopping upstream feed.
c Control
SRCF 5 Hopper level sensor for automatic stoppage of equipment.
c Control
SRCF 6 Audible and visual warning before movement Stockpile conveyors
b Control
SRCF 7a Safety bump stops on both ends of trolley - Stockpile conveyors
c Safety
SRCF 7b Local e-stops on both ends of trolley - Stockpile conveyors
c Safety
SRCF 7c emergency lanyards on both ends of trolley - Stockpile conveyors
c Safety
SRCF 8
Restricted access to the stackers walkways via castell key on local control panel and access gate to isolate the slewing movement - Stockpile conveyors
a MCC / Control
SRCF 9 Slewing motors equipped with brakes to be released by energy.
c Control
SRCF 10 Wind speed measurement. Automatic movement and locking into parking position.
a Control
SRCF 11 Train location sensors enable conveyor motive drive contactor.
b Control
4.5.5 Safety System Hardware Design
Design drawings for emergency stop and lanyard inputs show conventional dual channel
circuits. However they do not reflect best practice for the following reasons:
In all cases the two inputs have been taken into adjacent channels on the same input
module. In order to mitigate against common cause failures, best practice is to wire
the two inputs to separate input modules.
It is common to use antivalent inputs for machine safety functions, with one contact
normally open and the other normally closed. Where both contacts are normally
closed, as is the case at ......, best practice requires that test pulses be applied to
each input at different times to improve the diagnostic performance of the function.
Test pulses have not been used on ...... safety inputs.
In view of the above inadequacies, it is not considered valid to claim Category 3 compliance
for these circuits as there are fault modes that will not be detected at or before each demand.
These design limitations also mean that maintenance testing procedures need to be
enhanced in order to ensure that all fault modes can be identified.
Design drawings for safety outputs show a hard-wired quadruple redundancy output
arrangement. Two outputs drive the main contactor for each drive via dual redundant
interposing relays. The other two outputs drive the under voltage trip on the main air circuit
breaker (ACB) feeding the MCC, again via dual redundant interposing relays. The states of
the interposing relays and the contactors are monitored for fault detection purposes.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 33 of 90
It should be noted that this arrangement can be relied on if and only if the contactor and circuit
breaker monitoring contacts are mechanically linked to the main contacts, and hence will
show that the main contacts remain closed in the event that the main contacts become
welded. Typically this is the case for contactors only if the contactors are designed as “safety
contactors”. For general purpose industrial contactors, the monitoring contact will follow the
coil energisation state regardless of whether the main contacts become welded. No check
has been carried out on the types of contactors in use at .......
4.5.6 Safety PLC Coding Standards
No documentation detailing safety PLC Coding Standards was provided for review as part of
the audit. In the absence of a documented safety PLC Coding Standard document the safety
PLC code has been reviewed based on best practice.
The safety PLC code is implemented as rungs of ladder logic using built-in Rockwell function
blocks. This reflects the restricted set of programming facilities available for GuardLogix.
While it is possible to develop “Safety Add-On Instructions” there is no evidence that this has
been done for the safety PLC application program.
The code appears to be well structured, and is very well commented.
While there is no documented evidence provided to demonstrate that the code has been
developed in accordance with an appropriate documented safety lifecycle plan, the
presentation of the safety PLC code appears to reflect best practice.
4.5.7 Performance Level Evaluation Reports – Individual Plant Areas
Each report evaluates the performance level achieved for the individual safety functions in the
respective plant area based on the hardware design.
A “calculated” result is shown for PFH and PL for each safety function, apparently derived
from the Sistema software tool. A Sistema report is also presented for each plant area, but it
contains only the calculated results. No reference is made to the basis of the calculations,
including such factors as diagnostic coverage assumptions and common cause assumptions,
or to the source failure rate data. As such, without a documented basis for the calculations,
the calculated results in the report cannot be verified.
In this context, the inadequacies identified in the hardware design (refer previous section)
need to be considered in the common cause failure mode analysis.
Sistema contains a database of component failure rates that can be used for evaluation of
high demand safety functions. These failure rates are based on wear-out mechanisms for
electro-mechanical devices that operate frequently.
In typical machine safety cases where devices operate frequently and there are adequate
diagnostics incorporated to detect faults at or before each operation of a safety function, the
concept of proof testing has limited relevance.
Proof testing is more relevant for low demand functions with limited diagnostics, where the
safety function components may remain in the same state for long periods until called on to
operate. The primary intent of proof testing is to flush out any failures that are not observable
in normal operation or detectable by diagnostics, but that may prevent operation of the safety
function when a demand occurs. Proof testing typically requires more detailed testing of the
individual safety function components than would be expected in a simple functional test of
the overall safety function.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 34 of 90
It is important to note that component failure mechanisms are very different for the high
demand and low demand cases, and wear-out failure rates cannot be sensibly applied to low
demand functionality.
The calculated proof test intervals within the reports are based on applying wear-out failure
rates to a simplified formula for calculating a time averaged probability of failure for a low
demand function, and the resulting statements of the proof test intervals required to maintain
claims for SIL ratings are considered to have very limited validity.
A more thorough validation of the design would consider the mode of operation of each safety
function, and would determine if high demand or low demand calculations were appropriate in
each case. The Sistema tool is appropriate for high demand functions only. If all functions
are high demand and the design incorporates the required level of diagnostics, proof testing
may not be relevant, and periodic functional tests may be adequate.
However the inadequacies identified in the hardware design (refer previous section) with
respect to inadequate diagnostics on dual channel input circuits do need to be addressed in
maintenance test procedures.
It should also be noted that the claim that safety functions implemented in the control PLC
have achieved PLc is not supported by the available audit documentation.
AS4024.1503:2014 clause 6.2 limits the claim for a PLC system to PLa or PLb. Achieving
PLc and above requires the use of “well tried” components, and for a PLC this requires
evidence that it has been “made and verified using principles which demonstrate its suitability
and reliability for safety-related applications”. While it may be possible to provide documented
evidence of this for a high quality PLC product, it is most simply achieved by using a safety
PLC that has been certified in accordance with IEC 61508.
In the absence of any supporting documentation from Supplier for either the control PLC
components or the engineering design and programming of the safety functions in the control
PLCs, the claim of PLc for safety functions implemented in the control PLC is not considered
credible or valid.
4.5.8 Electrical Functional Safety Report XXXXXXXX2
The report purports to present the following information for each safety function:
1. Specifications review.
2. Equipment under control (EUC) Risk Assessment.
3. Determine the required Performance Level (PLr) – From Supplier Risk Assessment.
4. Identify the safety-related parts which carry out the safety function and evaluate the
required performance level (PL) considering –
• Category.
• MTTFd (mean time to dangerous failure)
• DC (diagnostic coverage)
• CCF (common cause failure)
5. Verification of PL.
6. Validation – to ensure all requirements are meet.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 35 of 90
7. Proof Test Intervals.
8. Software Validation.
It is noted with reference to item 4 above that the “evaluation” is addressed in the individual
plant area reports discussed in the preceding section, and that the functional safety report
contains only a compilation of the results from the hardware evaluation reports.
The following errors and omissions have been observed from a review of the report:
The report misrepresents the significance of the software safety lifecycle V-model,
which is intended to reflect a planning and control mechanism for software
development. The report examines no evidence relating to software development,
but purports to show compliance by analysis of the finished product. The safety
integrity achieved by software is a reflection of the development and test processes,
and is not derived simply from an analysis of the finished product.
The report analyses the software code against a subset of the guidelines set out in AS
4024.1503:2014. It does not consider all of the guidelines, and does not consider the
requirements set out in the Rockwell GuardLogix Safety Reference Manual. Refer to
Appendix F for further details.
4.5.9 Protection of Safety Functions
Review of functional specifications and operating manuals has not provided any evidence of
an approach to the design of safety functions from an operational perspective. In this context,
best design practice would consider the following:
Plant operators need to have a clear understanding of how safety functions operate,
what impact they have on the process, and how they will be reported and alarmed.
The reset mechanism should be clearly designed and documented. AS 4024
contains a number of specific requirements pertaining to the reset function and these
should be captured in the safety function design documentation.
Facilities to override safety functions may be required to allow continued operation
under abnormal conditions.
It is important for the override functions to be well designed and documented so that they can
be managed and used with due consideration of the increased operational risks that will be
present. Ad hoc allowance of overrides in an uncontrolled manner is not permissible for
safety functions because it potentially leads to situations whereby personnel who would
normally be protected by a safety function are carrying out their work totally unaware that the
safety function is inoperative. This represents an unacceptable potential risk of injury or death
to personnel.
The design of safety function overrides should ensure that they can be effected only by
authorised personnel, either by a physical key lock or by appropriate login restrictions. Where
practical, placing a safety function in override should enforce complementary equipment
lockouts or restrictions in logic to minimise the operational risk associated with the override.
Wherever possible, there should be an indication at the relevant workplace to alert personnel
that the safety function has been disabled.
It is observed that the safety functions implemented in the safety PLC do not have any means
of overriding the safety functions via the HMI, and overrides would have to be set and
removed via the safety PLC engineering workstation. In general the use of the engineering
workstation for this purpose is not desirable because it allows unrestricted access to the
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 36 of 90
safety functions once it is in use. Alternatively operating procedures should make it
mandatory that all E-Stops and lanyards are operational before the plant is allowed to run.
The other issue of concern is the capability to override safety functions that are implemented
in the control PLC, but are not differentiated in any way as safety-rated functions.
A further consideration for these safety functions is that they are not unconditional safety
functions that will always be available to protect personnel. The following statement from a
performance level evaluation report is typical of a number of safety functions:
The function to override (inhibit) safety functions implemented in the control PLC via the
SCADA is detailed in the Operational Manual. This document details that this function is only
available to the following user accounts:
Manager - Operation user account
Supplier - Maintenance user account
The safety functions can be inhibited and the status of the inhibit viewed on the equipment
SCADA popup via the parameter tabs or on area inhibit SCADA pages. The safety functions
can be inhibited on a per sensor basis if the sensor is selected as present for the equipment
The Operation Manual contains the following warning regarding the use of the inhibit sensor
functionality:
“Note: The inhibitions have to be considered the risks for the people and
the consequences for the material. When the sensor is inhibited may cause
a disturbance in the regulation. Only the responsible must be authorized to
change these parameters.”
During the audit operations and maintenance staff advised that operators run the system
under a high level user account that allows the override of safety functions instead of the
Operator user account as intended by the design and detailed in the Operation Manual.
It was reported that on plant handover to Client operators could not get the plant to run with
Operator level access as multiple instruments had to be disabled and this required higher
level access. Due to this operators were given higher level access so that they can disable
devices but there is no approval or tracking process around this to identify why a sensor is
disabled. This has resulted in uncontrolled disabling of safety sensor.
The equipment SCADA popups also allow the sensors associated to be selected or
deselected as present for the equipment based on whether they are physically present that
that device. For example one conveyor may have one belt rip sensors whereas another
conveyor may have two built rip sensors. This selection/deselection is available on the
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 37 of 90
Parameters P&ID tab of the equipment SCADA popup and is required as the SCADA
interface for all equipment of a specific type is generic, refer to section 4.4.7 SCADA
Standards for more information. This functionality is not detailed in the Operation Manual and
the SCADA user access level required to select or deselect a sensor is also not detailed.
Note that when a sensor is deselected on the Parameter P&ID tab its text appears grey (not
black) on the equipment SCADA Popup Parameters tabs and on the area inhibits SCADA
page.
Client maintenance staff advised that they are unsure how to determine what devices require
what sensors on the Parameter P&ID tab of the Device SCADA Popup and therefore they are
unsure if the correct sensors are selected. Refer to section 4.4.2 P&ID Documentation for
audit review of the plant P&IDs.
As shown in the screenshots above it is possible that a sensor is deselected on the
Parameters P&ID tab of the equipment SCADA popup but not inhibited on the Parameters
tab. In this case Client maintenance personnel have advised that it is believed that the sensor
operation is disabled but the SCADA representation to operation and maintenance personnel
is confusing.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 69 of 90
9 Summary/Recommendations
The major findings of the audit are summarised in the following table and recommendations provided
for each items.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 75 of 90
Item #
Engineering Process / Deliverables
Documentation Available For Audit
Requirement / Reference
Observation Conclusion Operation/Maintenance Impact
Recommendation Priority
Safety Instrumented System (SIS)
21 Safety Management Plan, including Safety Software Lifecycle
No
Documented plan for
achieving required safety functionality and safety integrity / Best practice - also recommended in AS 4024.1503:2014
There is no evidence of a safety management plan in accordance with functional safety standards or a project execution plan in accordance with AS ISO 9001 within the documentation made available for audit.
Compliance with standards cannot be verified Safety system development may not have been managed in a controlled manner
Potential for reduced integrity of safety functions – claimed Performance Levels and Category may not be achieved in practice
Client should request sight of functional safety management or alternative quality management documentation from Supplier for further review. Client should also ensure that safety system operating and maintenance procedures are developed and audited based on an appropriate safety management plan for the site.
High
22 Risk Assessment including Client involvement
Yes
Plant owner / operator to conduct a risk assessment for every machine / WHS legislation
1. Risk assessment (yyyy) that Client participated in is not appropriate for this purpose. 2. Risk assessments prepared by Supplier are appropriate, but Client personnel do not appear to have been involved, either as participants or as reviewers.
Client is exposed to legal risk because of lack of attention to risk assessments
Noncompliance with WHS Legislation
Client should carry out a detailed review of the Supplier risk assessments based on their knowledge of plant operation and maintenance.
Critical
23 Safety System Design Yes
Clear differentiation between safety functions and between safety and non-safety functions / Selection of appropriate equipment for implementation of safety functions / AS 4024.1503:2014
E-Stops and lanyards are implemented in the safety PLC. Oher safety functions are implemented in the control PLCs, but are not clearly differentiated as safety functions in the PLCs or on SCADA displays.
The implementation of safety functions in the control PLCs is not satisfactory. The functions are not clearly identified as safety functions in the control PLC or on SCADA displays. The safety functions implemented in the control PLCs are disabled in some operating modes and are easily disabled in SCADA. The safety functions implemented in the control PLCs are unlikely to achieve the claimed safety integrity (performance levels) in operational practice.
Personnel are exposed to significant risks because the safety function operation has very low integrity.
Client should evaluate the specific risks that the control PLC safety functions are required to mitigate in further detail during the risk assessment review. If the required performance level of any safety functions remains at PLc or above as a result of the review, it will be necessary to consider implementing those safety functions in a safety PLC. Where safety functions may remain in the control PLC, modifications to PLC code and SCADA displays will be required in order to provide clear differentiation of safety functions from control functions. The review should also consider whether the SCADA override functions and the PLC mode disable functions are acceptable in terms of the consequent unmitigated risk levels.
Critical
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 76 of 90
Item #
Engineering Process / Deliverables
Documentation Available For Audit
Requirement / Reference
Observation Conclusion Operation/Maintenance Impact
Recommendation Priority
24 Safety System Hardware Design
Yes
Manage common cause faults in design / AS 4024.1503:2014
Dual channel inputs operating in valent mode (both normally closed) are wired to adjacent channels on the same input module.
Compliance with standards cannot be verified.
Potential for reduced integrity of safety function – claimed Performance Levels and Category may not be achieved in practice
In view of the significant effort that would be required to correct this, it is recommended that it is not changed, but that maintenance procedures be reviewed to determine whether common cause faults will be detected by routine testing, and whether any single common cause fault can disable the safety function.
Medium
25 Safety System Hardware Design
Yes
Provide adequate diagnostic functions in design – inputs / AS 4024.1503:2014
Dual channel inputs operating in valent mode (both normally closed) are wired to 24V supplies rather that to supplies with test pulses applied.
Compliance with standards cannot be verified
Potential for reduced integrity of safety function – claimed Performance Levels and Category may not be achieved in practice
It is recommended that test pulses be applied to dual contact inputs operating in valent mode.
Medium
26 Safety System Hardware Design
Yes
Provide adequate diagnostic functions in design - output monitoring / AS 4024.1503:2014
It is not confirmed whether output contactor / circuit breaker monitoring is via mechanically linked contacts or via a monitoring capability with equivalent effectiveness.
Compliance with standards cannot be verified.
Potential for reduced integrity of safety function – claimed Performance Levels and Category may not be achieved in practice
Review detailed circuit design and component specification to see whether CAT 3 requirements are achieved.
Medium
27 Safety System Hardware
Design Yes
Design to be validated by calculation of achieved performance levels. / AS 4024.1503:2014
Sistema safety software tool used for Performance Level calculations, but only the results are available for audit. There is no visibility of the input data and the assumptions made in the calculations. There are significant outstanding queries about the effectiveness of common cause design safeguards and diagnostic coverage that need to be factored into the calculations.
It is impossible to verify that the calculated Performance Level results are valid. Compliance with standards cannot be verified.
Hardware safety integrity performance of the safety
system hasn’t been adequately validated.
Client should request complete Sistema calculation files from Supplier to enable the validity of the calculations to be evaluated.
High
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 77 of 90
Item #
Engineering Process / Deliverables
Documentation Available For Audit
Requirement / Reference
Observation Conclusion Operation/Maintenance Impact
Recommendation Priority
28 Safety System Hardware Design
Yes
Maintenance requirements to be clarified / AS 4024.1503:2014
There appears to be confusion between routine testing and proof testing in the functional safety report. Machine safety circuits typically feature full diagnostics that will identify all fault events, but routine functional testing may be required for equipment that does not get operated frequently e.g. E-Stops. If diagnostics are inadequate, "proof testing" may be required to reveal undetected dangerous faults before they can cause loss of the safety function. Of necessity, "proof testing" is more complicated and extensive than routine functional testing as it needs to exercise each channel of redundant circuits separately and needs to exercise all credible fault scenarios. Because of the emphasis on diagnostics in machine safety standards, proof testing is not normally considered necessary for machine safety applications.
Compliance with standards cannot be verified
Maintenance procedures may not be adequate to ensure that the safety integrity of the system will be maintained at the specified Performance Levels. Need to distinguish between “routine” tests which are simple functional tests relying on diagnostics to ensure that all faults are detected, and
“proof” tests which exercise all potential operating modes of a safety function to ensure that all hidden faults have been revealed. “Proof” tests should only be required where diagnostics are inadequate to reveal all possible faults including common cause faults.
Client should request a detailed explanation from Supplier on the basis of their test requirement calculations. This explanation should also detail the failure rate data sources used for these calculations - wear-our B10 rates versus failure after prolonged lack of use.
High
29 Safety System Functional Design
No
Specification of safety functions / AS 4024.1503:2014
No documentation available for audit
Compliance with standards cannot be verified
System documentation inadequate for safe operation and maintenance of the plant
Client should request issue of functional specification documentation from Supplier to enable further review of safety function design against the requirements of the standard.
High
30 Safety System Functional Design
No Manual reset function / AS 4024.1503:2014
No documentation available for audit
Compliance with standards cannot be verified
Reset function design / operation / integrity may not comply with requirements of standard
Client should request issue of functional specification documentation from Supplier to enable further review of safety function design against the requirements of the standard.
High
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 78 of 90
Item #
Engineering Process / Deliverables
Documentation Available For Audit
Requirement / Reference
Observation Conclusion Operation/Maintenance Impact
Recommendation Priority
31 Safety System Functional Design
No
Muting or override functions / AS 4024.1503:2014
No documentation available for audit on the intended design of the override facilities. Override of safety functions implemented in the control PLCs is occurring in an uncontrolled manner and operators have SCADA log on access to disable safety functions.
Compliance with standards cannot be verified
Inadequate facilities provided to identify safety functions on SCADA displays and to highlight the potential consequences of disabled safety functions. Safety systems disabled in an uncontrolled manner and personnel exposed to risks that are not being managed by the disabled safety functions.
Client should request issue of functional specification documentation from Supplier to enable further review of safety function design against the requirements of the standard. Client to:
Review and document all safety sensors and update SCADA equipment sensor selections
Review and document all disabled safety sensors
Put in place a system to control disabling of safety sensors
Restrict operator SCADA access so that only approved personnel can disable safety devices and that disabled sensors are regularly reviewed and faulty sensors replaced in a timely manner
Critical
32 Safety System Functional Design
No Fault considerations / AS 4024.1503:2014
No documentation available for audit
Compliance with standards cannot be verified
Safety system may not respond as required when fault conditions occur.
Client should request issue of functional specification documentation from Supplier to enable further review of safety function design against the requirements of the standard.
High
33 Safety System Application Program Development
No
Planning and documentation of software safety lifecycle / AS 4024.1503:2014
No documentation available for audit
Compliance with standards cannot be verified
Potential for reduced integrity of safety functions – claimed Performance Levels and Category may not be achieved in practice
Client should request sight of functional safety management or alternative quality management documentation from Supplier for further review. If the management plan cannot be provided for application program development, Client should consider a full review of the safety system software by a third party.
High
34 Safety System Application Program Development
No
Documentation of specification and design / AS 4024.1503:2014
No documentation available for audit
Compliance with standards cannot be verified
Potential for reduced integrity of safety functions – claimed Performance Levels and Category may not be achieved in practice
Client should request issue of software design documentation from Supplier to enable further review of software design against the requirements of the standard.
High
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 79 of 90
Item #
Engineering Process / Deliverables
Documentation Available For Audit
Requirement / Reference
Observation Conclusion Operation/Maintenance Impact
Recommendation Priority
35 Safety System Application Program Development
Yes Functional testing / AS 4024.1503:2014
Inadequate evidence of the required full scope of functional testing
Compliance with standards cannot be verified
Safety Systems may not function as intended
Client should request Supplier to provide evidence of test planning and recording that complies with all requirements of the standard. Client should undertake full re-commissioning of the safety functions to verify operation.
Critical
36 Safety System Application Program Development
No
Configuration management / AS 4024.1503:2014
No records available for audit
Compliance with standards cannot be verified
System documentation inadequate for safe operation and maintenance of the plant
Client should request Supplier to provide evidence of configuration management planning and recording for further review.
Medium
37 Safety System Application Program Development
No
Compliance with manufacturer's safety manual / Best practice AS 61508
No evidence available for audit
Compliance with standards cannot be verified
Potential for reduced integrity of safety functions – claimed Performance Levels and Category may not be achieved in practice
Client should request Supplier to provide evidence of compliance with the GuardLogix Safety Reference Manual for further review.
High
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 85 of 90
15 Appendix F – Functional Safety Analysis
15.1 Items Addressed in Electrical Functional Safety Report XXXXXXXX2
15.1.1 Example Programming Rules
The “example rules” presented in AS 4024.1503:2014 Annex J.4 are intended to provide
guidance on the issues that should be addressed in the functional safety management
planning and the software safety requirements specification that the application program is
derived from. Supplier have chosen instead to use them as a checklist for reviewing the
application program. All of the relevant items in Annex J.4 have been considered in the
Supplier report:
15.1.2 Application Program Development
Application program development and test requirements are set out in AS 4024.1503:2014
clause 4.6.3. Within this clause there are 10 sets of requirements designated as a) to j).
Supplier have chosen to use the points of section e) only as a checklist for software
verification.
Within the Supplier Report, the following “Analysis Result” entries are not considered valid:
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 86 of 90
The requirement for “justified or accepted coding guidelines” involves a much greater scope
than the use of standard function blocks. This refers to the requirements for project functional
safety documentation.
The requirement for “data integrity and plausibility checks” relates to defensive programming
requirements within the code, and again the implementation should be documented. It has
nothing to do with the validation and testing process, except that the robustness of the data
validity checking should be examined by testing with invalid data.
The requirement that “code should be tested by simulation” is specifically not related to field
testing.
15.2 Items NOT Addressed in Electrical Functional Safety Report XXXXXXXX2
15.2.1 Application Program Development
Application program development and test requirements as set out in AS 4024.1503:2014
clause 4.6.3 that are not addressed in the Functional Safety Report include:
Documentation – 4.6.3 a) and 4.6.3 g)
Software design – 4.6.3 b) and 4.6.3 c)
Verification and testing – 4.6.3 f) and 4.6.3 h)
Configuration Management – 4.6.3 i)
15.2.2 GuardLogix Safety Reference Manual
It is a fundamental requirement of functional safety standards that the usage of certified
equipment complies fully with the safety manual prepared by the manufacturer. For complex
equipment such as safety PLCs the manufacturers typically supply checklists to examine
compliance with the requirements of the safety manual. Best practice dictates that the
checklists should be filled out comprehensively and accurately, and it is reasonable to expect
that they should form part of the functional safety report.
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 87 of 90
Rockwell provide the following checklists for the GuardLogix safety PLC:
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 88 of 90
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 89 of 90
Control System Audit Report
For Client
xxxxxxxxx Rev 0 Page 90 of 90