Performance measures

Morten Nielsen

Performance measuresMeas Pred0.4050 0.83440.9373 1.00000.8161 0.63880.6752 0.98410.0253 0.00000.3196 0.53880.6764 0.62470.1872 0.19210.4220 0.65460.6545 0.65460.7917 0.13420.4405 0.35510.1548 0.00000.2740 0.19930.4399 0.64610.1725 0.39160.0539 0.00000.3795 0.56230.2242 0.19680.3108 0.21140.2260 0.03360.2780 0.56470.0198 0.12240.5890 0.55380.5120 0.43490.7266 1.00000.1136 0.00000.0456 0.21280.0069 0.41000.4502 0.3848

Performance measuresMeas Pred0.4050 0.83440.9373 1.00000.8161 0.63880.6752 0.98410.0253 0.00000.3196 0.53880.6764 0.62470.1872 0.19210.4220 0.65460.6545 0.65460.7917 0.13420.4405 0.35510.1548 0.00000.2740 0.19930.4399 0.64610.1725 0.39160.0539 0.00000.3795 0.56230.2242 0.19680.3108 0.21140.2260 0.03360.2780 0.56470.0198 0.12240.5890 0.55380.5120 0.43490.7266 1.00000.1136 0.00000.0456 0.21280.0069 0.41000.4502 0.3848

Pearsons’ correlation coefficient

Performance measures

• Accuracy of prediction method

Sort

Matthews correlation - Threshold of 0.5

False negative

True positive

True negative

False positive

Matthews correlation - Threshold of 0.5

5 98 5

Evaluation of prediction accuracy 

TPFN

TN FP

Evaluation of prediction accuracy 

AP

AN

Performance measure – Roc curve

False negative

True positive

True negative

False positive

4 10 1 12 0.080.29

Performance measure – Roc curve

False negative

True positive

True negative

False positive

4 10 1 12 0.080.29

Threshold TP FN

TP/(TP+FN) FP TN FP/(FP+TN)

>0.8  4  10  0.29  1  12  0.08>0.6 8  6  0.57  3  10  0.23 >0.4 11  3  0.79  6  7  0.46 >0.2 13 1  0.93  9  4  0.69 >0 14  0  1  13  0  1 

AUC = 0.5AUC = 1.0

ROC curves

AUC (area under the ROC curve)

Summary

• MSE• Small is good• Perfect = 0.0

• MCC and PCC• Random = 0.0• Perfect = 1.0 (or -1)

• ROC (AUC)• Random = 0.5• Perfect = 1 (or 0)