Post on 18-Mar-2016
description
transcript
Performance measures
Morten Nielsen
Performance measuresMeas Pred0.4050 0.83440.9373 1.00000.8161 0.63880.6752 0.98410.0253 0.00000.3196 0.53880.6764 0.62470.1872 0.19210.4220 0.65460.6545 0.65460.7917 0.13420.4405 0.35510.1548 0.00000.2740 0.19930.4399 0.64610.1725 0.39160.0539 0.00000.3795 0.56230.2242 0.19680.3108 0.21140.2260 0.03360.2780 0.56470.0198 0.12240.5890 0.55380.5120 0.43490.7266 1.00000.1136 0.00000.0456 0.21280.0069 0.41000.4502 0.3848
Performance measuresMeas Pred0.4050 0.83440.9373 1.00000.8161 0.63880.6752 0.98410.0253 0.00000.3196 0.53880.6764 0.62470.1872 0.19210.4220 0.65460.6545 0.65460.7917 0.13420.4405 0.35510.1548 0.00000.2740 0.19930.4399 0.64610.1725 0.39160.0539 0.00000.3795 0.56230.2242 0.19680.3108 0.21140.2260 0.03360.2780 0.56470.0198 0.12240.5890 0.55380.5120 0.43490.7266 1.00000.1136 0.00000.0456 0.21280.0069 0.41000.4502 0.3848
Pearsons’ correlation coefficient
Performance measures
• Accuracy of prediction method
Sort
Matthews correlation - Threshold of 0.5
False negative
True positive
True negative
False positive
Matthews correlation - Threshold of 0.5
5 98 5
Evaluation of prediction accuracy
TPFN
TN FP
Evaluation of prediction accuracy
AP
AN
Performance measure – Roc curve
False negative
True positive
True negative
False positive
4 10 1 12 0.080.29
Performance measure – Roc curve
False negative
True positive
True negative
False positive
4 10 1 12 0.080.29
Threshold TP FN
TP/(TP+FN) FP TN FP/(FP+TN)
>0.8 4 10 0.29 1 12 0.08>0.6 8 6 0.57 3 10 0.23 >0.4 11 3 0.79 6 7 0.46 >0.2 13 1 0.93 9 4 0.69 >0 14 0 1 13 0 1
AUC = 0.5AUC = 1.0
ROC curves
AUC (area under the ROC curve)
Summary
• MSE• Small is good• Perfect = 0.0
• MCC and PCC• Random = 0.0• Perfect = 1.0 (or -1)
• ROC (AUC)• Random = 0.5• Perfect = 1 (or 0)