| Date post: | 01-Dec-2014 |
| Category: |
Business |
| Upload: | yoshio-sakai |
| View: | 450 times |
| Download: | 5 times |
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Feature Analysis of Estimated Causes of Failures in Medical Device Software and
Proposal of Effective Measures
Seiko Shirasaka The Graduate School of System Design and Management, KEIO University
Yoshio Sakai Engineering Promotion Center, NIHON KOHDEN CORPORATION
Yasuharu Nishi Department of Systems Engineering, The University of Electro-Communications
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Background/Objectives
Research Background • 383 medical device software failures from 1983 to 1997 are analyzed in
“FAILURE MODES IN MEDICAL DEVICE SOFTWARE:AN ANALYSIS OF 15 YEARS OF RECALL DATA” (DOLORES R. WALLACE1 and D. RICHARD KUHN, 2001)
• Medical devices are getting more and more relying on software. Objectives • Analyze 86 medical device software failures in FY 2010 and identify the
change of the trend of failure causes from the former analysis • Confirm whether the International standard issued in 2006, IEC 62304
(Medical device software - Software life cycle processes), is effective to new failure causes or not.
2
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Recall Data
• 383 medical device software failures from 1983 to 1997
• The paper written by DOLORES R. WALLACE1 and D. RICHARD KUHN, 2001
3
• Analyze 86 medical device software failures in 2010
From • Pharmaceuticals and Medical Devices
Agency (Japan) Recall Information • FDA（USA） Medical Device Safety • Health Canada（Canada） Medical
Device Recall Listings • BfArM（Germany） News -Field
Corrective Actions • MHRA(England) Field Safety Notices
for medical devices • Swiss medic(Swiss） Archive "Field
Actions“
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Fault class distribution • FAILURE MODES IN MEDICAL DEVICE SOFTWARE:AN ANALYSIS OF 15 YEARS OF
RECALL DATA (DOLORES R. WALLACE1 and D. RICHARD KUHN) Fig.3. Fault class distribution
4
Fault class distribution From 1983 to 1997 Fault class distribution 2010 383
86
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
MEDICAL DEVICE SOFTWAR Fault class Changes
5
Fault class distribution From 1983 to 1997 2010
calculation 24% 7% change impact 6% 0% CM 1% 3% data 5% 2% fault tolerance 1% 21% Initialization 2% 0% interfaces 2% 14% logic 43% 26% omission 3% 0% other 3% 20% quality assurance 3% 0% requirements 4% 0% timing 3% 7%
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Feature of Fault class Change
• “Calculation” and “logic” decreased, “fault tolerance” and “other” increased.
• Reason of “Calculation” and “logic” decrease – Diversity of medical device system components cause the diversity of fault
causes and the rate of “Calculation” and “Logic” decrease relatively. – Causes of faults are getting complicated. It is difficult to categorize to one fault
class. • Reason of fault tolerance increase
– Increase functions to detect hardware random failure by software or to control risks by software
• Reason of “other” increase – Increase recall caused by IT/Network, Usability and Database
• Increase the functions to realize user interface by software, to utilize data through IT network and to manage data using database.
6
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
FY 2010 Analysis Result :Medical Device Category
7
• There are many medical device categories.
• Medical device categories which has more than one recall are listed.
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Feature of Fault class Change
• Calculation and logic decreased, “fault tolerance” and “other” increased. • Reason of “Calculation” and “logic” decrease
– Diversity of medical device system components cause the diversity of fault causes and the rate of “Calculation” and “Logic” decrease relatively.
– Causes of faults are getting complicated. It is difficult to categorize to one fault class.
• Reason of fault tolerance increase – Increase functions to detect hardware random failure by software or to control
risks by software • Reason of “other” increase
– Increase recall caused by IT/Network, Usability and Database • Increase the functions to realize user interface by software, to utilize data
through IT network and to manage data using database.
8
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Detail Causes in “Other”
10
Central Monitoring Station stops monitoring functionality unintentionally when the following condition occurs simultaneously. 1) Multi terminal monitors are connected to central
monitoring station through network and one or more terminal monitors are set to display patient data which comes from other terminal monitor.
2) The terminal monitor which provides patient data stops
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Detail Causes in “Other”
11
If the buttons of a certain medical device is pushed for more than 5 second, its mode is automatically changed to monitor mode. And in this mode, a user can not perform its functions.
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Detail Causes in “Other”
12
If a picture data is stored as the same file name at “picture archiving and communication system”, the picture data of a patient overwrites the picture data of other patient.
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Analysis Results：Feature of Probable Fault causes (2010)
13
Systematic Software Failures / Faults rate
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Trend of fault and probable cause
• Most software faults are Systematic Software Failures. (About 84%)
• Most common probable cause is lack of Validation which is related with environment and usage situation of medical device. Its number is 53.
• On the other hand, there are some hardware random failures, hardware systemic failures and timing caused failures. Their numbers are 12, 8 and 3.
14
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Medical device software related international standards which are newly relieved or revised after 2003
15
Risk Management
(2003)
Software Processes
(2006)
Usability (2007)
IT/Network (2010)
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Medical device software related international standards which are newly relieved or revised after 2003
16
Risk Management
(2003)
Software Processes
(2006)
Usability (2007)
IT/Network (2010)
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effective counter measure for recurrence prevention
Probable Cause Feature of Fault and Probable Cause No. Effective Counter Measure
Internal Hardware Hardware Random Failure 12 <->
1. Statistical Bug Analysis, Risk Control by Software
Hardware Systemic Failure 3 <-> 2. Hardware Design, Design Process Management
Internal Software
Systemic Failure/Faults
Consequence Analysis required Failure 18 <-> 3. FMEA
Configuration Management Mistakes 3 <-> 4. Software Configuration Management, Change Management, Regression Test
Software Systemic Failure 72 <-> 5. Software Process Management
Interface/Timing Interface caused failure 16 <-> 6. Input Simulation Test
Timing Caused Failure 8 <-> 7. Timing Caused tests
Requirement Analysis Usability
Lack of Usability Analysis 6 <-> 8. Usability Analysis and countermeasure, FMEA
Environment and usage condition 53 <-> 9.Requreiment Analysis. Scenario Test under user environment, Validation
External Software IT Network , Outer-Communication 14 <-> 10. IT/Network related test, Integration Test
COTS Software 4 <-> 11. SOUP/OTS Risk Analysis and Evaluation
Severity Probability of serious harm 13 <-> 12. FTA,
17
Activities requested by IEC62304
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Medical device software related international standards which are newly relieved or revised after 2003
18
Risk Management
(2003)
Software Processes
(2006)
Usability (2007)
IT/Network (2010)
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effective counter measure for recurrence prevention
Probable Cause Feature of Fault and Probable Cause No. Effective Counter Measure
Internal Hardware Hardware Random Failure 12 <->
1. Statistical Bug Analysis, Risk Control by Software
Hardware Systemic Failure 3 <-> 2. Hardware Design, Design Process Management
Internal Software
Systemic Failure/Faults
Consequence Analysis required Failure 18 <-> 3. FMEA
Configuration Management Mistakes 3 <-> 4. Software Configuration Management, Change Management, Regression Test
Software Systemic Failure 72 <-> 5. Software Process Management
Interface/Timing Interface caused failure 16 <-> 6. Input Simulation Test
Timing Caused Failure 8 <-> 7. Timing Caused tests
Requirement Analysis Usability
Lack of Usability Analysis 6 <-> 8. Usability Analysis and countermeasure, FMEA
Environment and usage condition 53 <-> 9.Requreiment Analysis. Scenario Test under user environment, Validation
External Software IT Network , Outer-Communication 14 <-> 10. IT/Network related test, Integration Test
COTS Software 4 <-> 11. SOUP/OTS Risk Analysis and Evaluation
Severity Probability of serious harm 13 <-> 12. FTA,
19
Activities requested by ISO14971
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Medical device software related international standards which are newly relieved or revised after 2003
20
Risk Management
(2003)
Software Processes
(2006)
Usability (2007)
IT/Network (2010)
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effective counter measure for recurrence prevention
Probable Cause Feature of Fault and Probable Cause No. Effective Counter Measure
Internal Hardware Hardware Random Failure 12 <->
1. Statistical Bug Analysis, Risk Control by Software
Hardware Systemic Failure 3 <-> 2. Hardware Design, Design Process Management
Internal Software
Systemic Failure/Faults
Consequence Analysis required Failure 18 <-> 3. FMEA
Configuration Management Mistakes 3 <-> 4. Software Configuration Management, Change Management, Regression Test
Software Systemic Failure 72 <-> 5. Software Process Management
Interface/Timing Interface caused failure 16 <-> 6. Input Simulation Test
Timing Caused Failure 8 <-> 7. Timing Caused tests
Requirement Analysis Usability
Lack of Usability Analysis 6 <-> 8. Usability Analysis and countermeasure, FMEA
Environment and usage condition 53 <-> 9.Requreiment Analysis. Scenario Test under user environment, Validation
External Software IT Network , Outer-Communication 14 <-> 10. IT/Network related test, Integration Test
COTS Software 4 <-> 11. SOUP/OTS Risk Analysis and Evaluation
Severity Probability of serious harm 13 <-> 12. FTA,
21
Activities requested by IEC62366
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Medical device software related international standards which are newly relieved or revised after 2003
22
Risk Management
(2003)
Software Processes
(2006)
Usability (2007)
IT/Network (2010)
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effective counter measure for recurrence prevention
Probable Cause Feature of Fault and Probable Cause No. Effective Counter Measure
Internal Hardware Hardware Random Failure 12 <->
1. Statistical Bug Analysis, Risk Control by Software
Hardware Systemic Failure 3 <-> 2. Hardware Design, Design Process Management
Internal Software
Systemic Failure/Faults
Consequence Analysis required Failure 18 <-> 3. FMEA
Configuration Management Mistakes 3 <-> 4. Software Configuration Management, Change Management, Regression Test
Software Systemic Failure 72 <-> 5. Software Process Management
Interface/Timing Interface caused failure 16 <-> 6. Input Simulation Test
Timing Caused Failure 8 <-> 7. Timing Caused tests
Requirement Analysis Usability
Lack of Usability Analysis 6 <-> 8. Usability Analysis and countermeasure, FMEA
Environment and usage condition 53 <-> 9.Requreiment Analysis. Scenario Test under user environment, Validation
External Software IT Network , Outer-Communication 14 <-> 10. IT/Network related test, Integration Test
COTS Software 4 <-> 11. SOUP/OTS Risk Analysis and Evaluation
Severity Probability of serious harm 13 <-> 12. FTA,
23
Activities requested by IEC80001
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Medical device software related international standards which are newly relieved or revised after 2003
24
Risk Management
(2003)
Software Processes
(2006)
Usability (2007)
IT/Network (2010)
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effective counter measure for recurrence prevention
Probable Cause Feature of Fault and Probable Cause No. Effective Counter Measure
Internal Hardware Hardware Random Failure 12 <->
1. Statistical Bug Analysis, Risk Control by Software
Hardware Systemic Failure 3 <-> 2. Hardware Design, Design Process Management
Internal Software
Systemic Failure/Faults
Consequence Analysis required Failure 18 <-> 3. FMEA
Configuration Management Mistakes 3 <-> 4. Software Configuration Management, Change Management, Regression Test
Software Systemic Failure 72 <-> 5. Software Process Management
Interface/Timing Interface caused failure 16 <-> 6. Input Simulation Test
Timing Caused Failure 8 <-> 7. Timing Caused tests
Requirement Analysis Usability
Lack of Usability Analysis 6 <-> 8. Usability Analysis and countermeasure, FMEA
Environment and usage condition 53 <-> 9.Requreiment Analysis. Scenario Test under user environment, Validation
External Software IT Network , Outer-Communication 14 <-> 10. IT/Network related test, Integration Test
COTS Software 4 <-> 11. SOUP/OTS Risk Analysis and Evaluation
Severity Probability of serious harm 13 <-> 12. FTA,
25
Activities requested by ISO13485
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Medical device software related international standards which are newly relieved or revised after 2003
26
Risk Management
(2003)
Software Processes
(2006)
Usability (2007)
IT/Network (2010)
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effective counter measure for recurrence prevention
Probable Cause Feature of Fault and Probable Cause No. Effective Counter Measure
Internal Hardware Hardware Random Failure 12 <->
1. Statistical Bug Analysis, Risk Control by Software
Hardware Systemic Failure 3 <-> 2. Hardware Design, Design Process Management
Internal Software
Systemic Failure/Faults
Consequence Analysis required Failure 18 <-> 3. FMEA
Configuration Management Mistakes 3 <-> 4. Software Configuration Management, Change Management, Regression Test
Software Systemic Failure 72 <-> 5. Software Process Management
Interface/Timing Interface caused failure 16 <-> 6. Input Simulation Test
Timing Caused Failure 8 <-> 7. Timing Caused tests
Requirement Analysis Usability
Lack of Usability Analysis 6 <-> 8. Usability Analysis and countermeasure, FMEA
Environment and usage condition 53 <-> 9.Requreiment Analysis. Scenario Test under user environment, Validation
External Software IT Network , Outer-Communication 14 <-> 10. IT/Network related test, Integration Test
COTS Software 4 <-> 11. SOUP/OTS Risk Analysis and Evaluation
Severity Probability of serious harm 13 <-> 12. FTA,
27
Activities requested by IEC62304
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Overview of software development processes and activities (IEC62304)
28
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effectiveness Analysis of IEC 62304 Process and Activities-1
29
Software development process 1, (3), 4, 5, 6, 7, 8, 9, 10, 11,
12 REMARKS
5.1 Software development planning 1, 4, 5, 8, 11 Software Development Plan is useful to clarify how much Verification and Validation should be conducted. 5.1.1 Software development plan 5
5.1.2 Keep software development plan updated 5
5.1.3 Software development plan reference to system design and development ---
5.1.4 Software development standards, methods and tools planning ---
5.1.5 Software integration and integration testing planning 5
5.1.6 Software verification planning 5
5.1.7 Software risk management planning 5
5.1.8 Documentation planning 1, 5, 8, 11
5.1.9 Software configuration management planning ---
5.1.10 Supporting items to be controlled 4, 5
5.1.11 Software configuration item control before verification ---
5.2 Software requirements analysis 4, 5 Clarification of Intended Use is effective to clarify the function of software.
5.2.1 Define and document software requirements from system requirements 9
5.2.2 Software requirements content 9
5.2.3 Include risk control measures in software requirements 9
5.2.4 Re-evaluate medical device risk analysis (3), 8, 9, 10, 11, 12
5.2.5 Update system requirements ---
5.2.6 Verify software requirements ---
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effectiveness Analysis of IEC 62304 Process and Activities-2
5.3 Software architectural design 10, 11, 12 Complexity of software architecture causes software anomaly.
5.3.1 Transform software requirements into an architecture 12
5.3.2 Develop an architecture for the interfaces of software items 10, 11, 12
5.3.3 Specify functional and performance requirements of SOUP item 11
5.3.4 Specify system hardware and software required by SOUP item 11
5.3.5 Identify segregation necessary for risk control 12
5.3.6 Verify software architecture 12
5.4 Software detailed design ---
5.4.1 Refine software architecture into software units ---
5.4.2 Develop detailed design for each software unit ---
5.4.3 Develop detailed design for interfaces ---
5.4.4 Verify detailed design ---
5.5 Software unit implementation and verification ---
5.5.1 Implement each software unit ---
5.5.2 Establish software unit verification process ---
5.5.3 Software unit acceptance criteria ---
5.5.4 Additional software unit acceptance criteria ---
5.5.5 Software unit verification ---
30
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effectiveness Analysis of IEC 62304 Process and Activities-3
5.6 Software integration and integration testing 4, 5, (6), (7), 9, 10, 11 Software interface causes some failures. Software integration tests are effective to cover these causes..
5.6.1 Integrate software units ---
5.6.2 Verify software integration 9
5.6.3 Test integrated software 5, (6), (7), 10, 11
5.6.4 Integration testing content 5, (6), (7), 10, 11
5.6.5 Verify integration test procedures (6), (7), 10, 11
5.6.6 Conduct regression tests 4
5.6.7 Integration test record contents ---
5.6.8 Use software problem resolution process 5
5.7 Software system testing 4, 5, (6), (7), 9, 10 Software system testing is effective to cover software systematic failures.
5.7.1 Establish tests for software requirements (6), (7), 9, 10
5.7.2 Use software problem resolution process 5
5.7.3 Retest after changes 4
5.7.4 Verify software system testing 5
5.7.5 Software system test record contents ---
5.8 Software release 4, 5 Software release process is effective to cover software systematic failures.
5.8.1 Ensure software verification is complete 5
5.8.2 Document known residual anomalies 4
5.8.3 Evaluate known residual anomalies 4
5.8.4 Document released versions 5
5.8.5 Document how released software was created 5
5.8.6 Ensure activities and tasks are complete 5
5.8.7 Archive software 4
5.8.8 Assure repeatability of software release 5
31
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Effectiveness Analysis of IEC 62304 Process and Activities-4
Software maintenance process 3, 4, 5, 8, 9 Software maintenance process is effective to monitor anomaly occurred in the field. 6.1 Establish software maintenance plan 5
6.2 Problem and modification analysis 3, 4, 5, 8, 9
6.2.1 Document and evaluate feedback 3, 8, 9
6.2.1.1 Monitor feedback 3, 8, 9
6.2.1.2 Document and evaluate feedback 3, 8, 9
6.2.1.3 Evaluate problem report's affects on safety 3, 8, 9
6.2.2 Use software problem resolution process 5
6.2.3 Analyze change requests 4
6.2.4 Change request approval 4
6.2.5 Communicate to users and regulators 5
6.3 Modification implementation 4, 5
6.3.1 Use established process to implement modification 4, 5
6.3.2 Re-release modified software system 4, 5
32
[email protected] ISSRE 2011 Nov 29 - Dec 2, 2011
Summary
• As the results of the comparison between old and new fault class distribution, definite difference is identified. Three differences are identified. The percentage of “Logic” and “Calculation” are decreased and that of “Fault tolerance” and “other” are increased.
• The reasons are : – software in medical device is getting more complicated ( that causes systematic failures) – software technology is introduced into medical device ( that causes IT network failures ,
usability failures and database failures)
• Recent standards can be effective counter measures for recent failures.
33
