Table 4 Results in the holdout set of all models after training on the full training set, ranked by AUCPR
Algorithm | AUCPR | AUROC | Accuracy | Balanced accuracy | Sensitivity | Specificity | Brier Score |
|---|---|---|---|---|---|---|---|
XGBoost (Bayesian Optimization) | 0.9974 (0.9961–0.9984) | 0.9997 (0.9996–0.9999) | 0.995 (0.993–0.9965) | 0.9787 (0.9719–0.9849) | 0.9586 (0.943–0.9742) | 0.9987 (0.9978–0.9996) | 0.0041 (0.0031–0.0051) |
Random Forest (Bayesian Optimization) | 0.9973 (0.996–0.9983) | 0.9997 (0.9996–0.9999) | 0.9947 (0.9927–0.9963) | 0.9749 (0.9676–0.9817) | 0.9506 (0.9337–0.9676) | 0.9992 (0.9985–0.9999) | 0.0044 (0.0035–0.0052) |
XGBoost | 0.9971 (0.9958–0.9982) | 0.9997 (0.9995–0.9999) | 0.9949 (0.9929–0.9964) | 0.9764 (0.9693–0.983) | 0.9538 (0.9374–0.9702) | 0.999 (0.9983–0.9998) | 0.0042 (0.0031–0.0054) |
Random Forest | 0.9966 (0.9951–0.9979) | 0.9996 (0.9994–0.9998) | 0.994 (0.9918–0.9957) | 0.9745 (0.9671–0.9815) | 0.9506 (0.9337–0.9676) | 0.9984 (0.9974–0.9994) | 0.0048 (0.004–0.0056) |
Support Vector Machine | 0.9935 (0.9904–0.9962) | 0.999 (0.9983–0.9997) | 0.9941 (0.992–0.9958) | 0.9746 (0.9673–0.9815) | 0.9506 (0.9337–0.9676) | 0.9985 (0.9976–0.9995) | 0.0047 (0.0036–0.0058) |
Neural Network | 0.993 (0.9885–0.9966) | 0.9993 (0.9989–0.9997) | 0.9934 (0.9912–0.9952) | 0.972 (0.9644–0.9793) | 0.9459 (0.9282–0.9636) | 0.9982 (0.9972–0.9993) | 0.0049 (0.0038–0.006) |
kNN | 0.9846 (0.979–0.9894) | 0.9955 (0.9923–0.9988) | 0.9864 (0.9833–0.989) | 0.9324 (0.921–0.9433) | 0.8662 (0.8396–0.8929) | 0.9985 (0.9976–0.9995) | 0.0095 (0.0081–0.011) |
Logistic Regression | 0.9791 (0.9648–0.9905) | 0.9973 (0.9952–0.9994) | 0.9915 (0.989–0.9935) | 0.9681 (0.9602–0.9757) | 0.9395 (0.9208–0.9581) | 0.9968 (0.9954–0.9982) | 0.0063 (0.0051–0.0077) |
Regularized Linear Discriminal Analysis | 0.9682 (0.9498–0.9856) | 0.9977 (0.9968–0.9986) | 0.9831 (0.9798–0.986) | 0.9134 (0.9009–0.9259) | 0.828 (0.7985–0.8575) | 0.9989 (0.998–0.9997) | 0.0138 (0.0118–0.016) |
SpO2 - SO2 difference (matched) | 0.9311 (0.911–0.9498) | 0.9825 (0.9761–0.9888) | 0.9871 (0.9841–0.9896) | 0.9342 (0.9228–0.9451) | 0.8694 (0.8431–0.8958) | 0.999 (0.9983–0.9998) | 0.0124 (0.0103–0.0145) |