1. Help Center
  2. Using boodleAI's Software

How to Evaluate a Custom Guidon

Make sense of your data.

giphy

A guidon rating should be the highest level where the guidon meets all of the criteria for that level. 

Example: If a guidon had a 1.2X-3.5X predictive power, AUC-ROC of 0.65, all of the ROC is above the line, and PR shape has 30% of PR averaging 0.9 precision, then the overall rating should be “Acceptable.”

ELEMENT

DO NOT USE

CAUTION

ACCEPTABLE

GOOD

EXCELLENT

Low End Predictive Power

<0.75

<1

1

1

1

High End Predictive Power

1

1.01-1.50

1.51-2.00

2.01-3.00

3.01+

AUC-ROC

<0.5

.0.5-0.6

0.6-0.7

0.7-8

0.8+

ROC Shape

50% or more of ROC is under the line

Any part of ROC goes under the line

All of ROC is above the line

All of ROC is above the line

All of ROC is above the line

PR Shape

No part of PR averages above .7 precision

1-20% of PR averages 0.7 precision

20%+ of PR average 0.7 precision

20%+ of PR average 0.8 precision

20%+ of PR average 0.9 precision

General Guidon Notes for Low Ratings:

  • Low End Predictive Power: “Guidon’s lowest predictive power needs to be improved.”
  • High End Predictive Power: “Guidon’s highest predictive power can be improved.”
  • AUC-ROC: “Guidon’s predictive accuracy can be improved.”
  • ROC Shape: “Guidon’s predictive accuracy can be more consistent.”
  • PR Shape: “Guidon’s precision can be more consistent.”

Ready to Create Predictions Using a Custom Guidon?