Calculations for KPI Scores and Overall Grade
The Scorecard defaults to all providers over a 365-day period. Use the filters to view each Provider individually or to adjust the date range to compare recent performance with past performance.
Scorecard scores and grades answer many questions about Provider performance. Overall Grades tell you "How are my providers doing?" while the Key Performance Indicator Scores answer "Where are my Providers' strong, and where are their areas of improvement?". Let's break down these concepts to learn how scores are calculated on the Scorecard.
Key Performance Indicator (KPI) Scores
On the Contractor Scorecard, the actions Providers do in Service Automation determine their grades. These actions are called Key Performance Indicators, or KPIs. When Providers perform these KPIs in a timely fashion they can theoretically receive higher scores.
There are 6 Repair KPIs:
- Dispatch Confirmation: the percentage of work orders (with an ETA of up to 48 hours or less) that service providers respond to in a timely fashion.
- Check-in Compliance: the percentage of completed work orders that have both a check-in and a check-out entry, with a minimum check-in time of 10 minutes.
- On-Time Arrival - Schedule Date: for which the service provider was on-site no later than the date and time the work order is scheduled for. It can be either the updated date if an update was made or the date/time generated by the system based on the work order priority.
- On-Time Arrival - Updated ETA: the percentage of on-time arrival for work orders with an updated scheduled date only.
- First Time Completion: the percentage of work orders that were fixed on the first visit with only 1 check-in and 1 check-out recorded on the same day (or within 24 hours for overnight work orders).
Invoice Submission (within 30 days): the percentage of work orders with invoices submitted within 30 days of the work order being marked as Completed. Note that this also includes Completed/Pending Confirmation.
See Scorecard Repair KPIs - Repair KPIs Tab for a deeper understanding of these KPIs.
There are 3 Maintenance KPIs:
- Check-in Compliance: the percentage of completed work orders that have both a check-in and a check-out entry, with a minimum check-in time of 10 minutes.
- Schedule Date Entry Confirmation: the percentage of work orders where the scheduled date was updated by the service provider to indicate when the work will be performed (applicable only to Planned Maintenance work orders generated in bulk and with an expiration date).
Scheduled Work Orders Completed: the percentage of work orders that were completed by the service provider before the expiration date of the work order.
See Scorecard Maintenance KPIs - Maintenance KPIs Tab for a deeper understanding of these KPIs.
Calculating KPI Scores and Overall Scores
Each KPI has a calculated Score, which is the percentage of work orders that meet the criteria for that KPI:
Individual KPI Score (%) = Number of Compliant Work Orders ÷ Total Work Orders
(rounded to the nearest whole number)
In other words, of all the work orders assigned to the Provider, how many work orders did the Provider perform "positive" behavior (for example: confirming the dispatch, arriving onsite by the scheduled date or updated scheduled date, or finishing the job the first time).
The sum of all KPI Scores gives an Overall Score:
Overall Score = Sum of all KPI scores ÷ Total number of eligible KPIs
The Overall Score is then mapped to a Grade, which represents overall performance:
Providers with less than 10 completed work orders in the past 12 months will show “Needs more work orders” instead of a grade.
Score/Grade Calculation Examples
The formulas are the same for both Repair and Maintenance KPIs:
- For KPI Scores: Individual KPI Score (%) = # of Compliant Work Orders ÷ Total Work Orders (rounded to the nearest whole number)
- For the Overall Score: Overall Score = Sum of all KPI scores ÷ Total number of eligible* KPIs
- Repair work orders have 6 KPIs.
- Maintenance work orders have 3 KPIs.
- *When there are no work orders for a specific KPI, that KPI is not included in the calculations. See "Ineligible" KPIs, below, for more information.
★ In This Example: All Eligible KPIs
REPAIR KPIs
KPIs | # of Compliant Work Orders | Total Work Orders | KPI Score (%) |
|---|---|---|---|
Dispatch Confirmation | 893 | 900 | 99% |
Check-in Compliance | 875 | 900 | 97% |
On Time Arrival - Schedule Date | 303 | 900 | 34% |
On Time Arrival - Updated ETA | 220 | 900 | 24% |
First Time Completion | 720 | 900 | 80% |
Invoice Submission (within 30 days) | 878 | 900 | 98% |
Repair Overall Grade = C | Overall Score is 72% On-time arrival on both the original ETA and the updated ETA are the behaviors bringing the score down. | Sum of Repair KPI Scores | 432 | |
| Eligible KPIs | 6 | ||
| Overall Repair Score and Grade | 72% | ||
- If this was the overall score for one Provider, this client may want to work directly with their Providers to improve compliance in these specific areas. Also, adjusting the filter dates may highlight whether the Provider is showing recent signs of improvement.
- If this was the overall score across all Providers, checking the Repair KPI Comparison table will answer the question on which Providers are not in compliance, or the Repair KPI Heatmap to view whether certain Locations are not in compliance.
- In both scenarios, this client could also opt to work on the other metrics — the First-Time Completion Rate could potentially dip into the C overall score if not addressed.
MAINTENANCE KPIs
KPIs | # of Compliant Work Orders | Total Work Orders | KPI Score (%) |
Check-In Compliance | 401 | 437 | 92% |
Schedule Date Entry Confirmation | 325 | 437 | 74% |
Scheduled Work Orders Completed | 350 | 437 | 90% |
Maintenance Overall Grade = B | Overall Score is 85% Schedule Date Entry Confirmation is the behavior bringing the score down a little. | Sum of Repair KPI Scores | 256 | |
| Eligible KPIs | 3 | ||
| Overall Repair Score and Grade | 85% | ||
- Working with Providers to ensure they are updating the scheduled date for PMs is one way a client may work with this data, whether it is one Provider or all across the board.
"Ineligible" KPIs
KPIs with no eligible work orders are not included in the Overall Score and Grade calculations. Given the formula
Overall Score = Sum of all KPI scores ÷ Total number of eligible KPIs
if a Provider never confirms a dispatch and instead just checks in onsite, the Dispatch Confirmation KPI will have no data to calculate. Therefore,
- the Dispatch Confirmation KPI will not have a Score, and
- the Total number of eligible KPIs will reduce to 5.