Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | porter-cobb |
View: | 42 times |
Download: | 6 times |
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
The problems of decision
• Sound the alarm when the signal is very small– Advantages
• Plenty of time to get the fighters off the ground
• Reduce the number of bombers reaching the target
– Disadvantages• Lots of false alarms
• Waste of gasoline, wear and tear on fighter planes
• Exhaust fighter pilots
• Sound the alarm when the signal is unmistakable– Advantages
• No waste, no wear and tear, no exhaustion
– Disadvantages• More bombers get through, more bombs, more destructions
Solution to decisions
• Code Yellow
– Signal suggests possible incoming bomber
– Pilots get dressed, fighter planes get loaded with gasoline and ammunition
• Code Orange
– Signal suggests incoming bomber likely
– Fighter planes towed to runway, pilots goes to the planes
• Code Red
– Signal is unmistakable
– Fighter planes take off
Receiver Operator Characteristics
False alarms
Se
nsiti
vity
Useless operatorSensitivity and false alarm rate changes together
Perfect operator100% Sensitive0% false alarms
Most operators
Receiver Operator Characteristics
False alarms
Se
nsiti
vity
Code YellowCode Orange
Code Red
Increasingsignal
strength
False alarms
Se
nsiti
vity
Code YellowCode Orange
Code Red
Increasingsignal
strength
Receiver Operator Characteristics
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
Statistical ROC
• A measurement is normally distributed in two groups, those outcome negative and those outcome positive
• Using a cut off level to make a decision will create a number of TP, FN, FP, and TN. From these Sensitivity and Specificity is calculated
• If the cut off value changes
– TP,FN,FP,TN changes
– Sensitivity and Specificity changes
• The relationship between Sensitivity and Specificity over the range of the measurement defines the ROC
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
Advantages of using ROC
• It defines the quality of a test or prediction using a measurement without specifying a cut off value for decision making
• Assuming Normal distribution
– The mean and Standard Error can be estimated
– The 95% CI can be estimated
– Statistical significance can be determined
– Whether one test is better than another can be determined
23
چهار ترکیب ممکن برای تصمیم و خطا
درص�د اف�راد غ�یر مبتال ک�ه م�دل :FPمثبت نادرس�ت•داده تش�خیص مبتال را آنه�ا اش�تباه ب�ه تص�میم
است.نادرس�ت• :FNمنفی م�دل ک�ه مبتال اف�راد درص�د
تص�میم ب�ه اش�تباه آنه�ا را غ�یرمبتال تش�خیص داده .است
:TNمنفی درس�ت• ک�ه مبتال غ�یر اف�رادی در ص�د مدل تصمیم به درستی آن را تشخیص داده است.
درص�د اف�رادی ک�ه بیم�اری دارن�د :TPمثبت درس�ت•داده تش�خیص را آن درس�تی ب�ه تص�میم م�دل و
است.
TP+FN=100%FP+TN=100%
هدف مدل تصمیم حداقل کردن
)FN وFP (خطاهاست
24
نکته
و FNدرص�د FP مس�تقل ص�ورت ب�ه شود حداقل نمی
اف�زایش می یاب�د و FN کم ش�ود FPاگ�ر برعکسمی ROCنم�ودار پی�دا را بهین�ه مق�دار
کند.
25
:مثال
هدف تش�خیص فش�ار خ�ون ب�اال در اف�راد اس�ت. ب�رای انج�ام این ک�ار فش�ار خ�ون اف�راد در س�ه
گیری می شود. گروه فرضی اندازه
- جمعیت س�الم ک�ه تحت م�راقبت ق�رار گرفت�ه1اند.
- جمعیت بیم�اران مش�مول م�راقبت اولی�ه ک�ه ب�ه 2خاطر شکایات جزئی تحت بررسی هستند.
- جمعیت بیم�ارانی ک�ه در بخش قل�بی بس�تری 3هستند.