Make sense of your data.
A guidon rating should be the highest level where the guidon meets all of the criteria for that level.
Example: If a guidon had a 1.2X-3.5X predictive power, AUC-ROC of 0.65, all of the ROC is above the line, and PR shape has 30% of PR averaging 0.9 precision, then the overall rating should be “Acceptable.”
|
ELEMENT |
DO NOT USE |
CAUTION |
ACCEPTABLE |
GOOD |
EXCELLENT |
|
Low End Predictive Power |
<0.75 |
<1 |
1 |
1 |
1 |
|
High End Predictive Power |
1 |
1.01-1.50 |
1.51-2.00 |
2.01-3.00 |
3.01+ |
|
AUC-ROC |
<0.5 |
.0.5-0.6 |
0.6-0.7 |
0.7-8 |
0.8+ |
|
ROC Shape |
50% or more of ROC is under the line |
Any part of ROC goes under the line |
All of ROC is above the line |
All of ROC is above the line |
All of ROC is above the line |
|
PR Shape |
No part of PR averages above .7 precision |
1-20% of PR averages 0.7 precision |
20%+ of PR average 0.7 precision |
20%+ of PR average 0.8 precision |
20%+ of PR average 0.9 precision |
General Guidon Notes for Low Ratings:
- Low End Predictive Power: “Guidon’s lowest predictive power needs to be improved.”
- High End Predictive Power: “Guidon’s highest predictive power can be improved.”
- AUC-ROC: “Guidon’s predictive accuracy can be improved.”
- ROC Shape: “Guidon’s predictive accuracy can be more consistent.”
- PR Shape: “Guidon’s precision can be more consistent.”
Ready to Create Predictions Using a Custom Guidon?