Check the denominator before trusting an improving KPI

Before treating an improving KPI as evidence of better performance, check whether its denominator still represents the same population. A higher percentage can result from missing records, changed exclusions or a different reporting period. Reconcile the population and calculation before asking a team to explain the trend.

This guide is for Canadian operations and service managers using internal performance indicators, including cross-border reporting teams. NIST's Baldrige data guidance encourages organisations to choose useful measures and use analysis to support decisions. The fields and checks described here form our practical control method, not a mandatory Canadian reporting rule. NIST Baldrige data and analysis

Give each KPI a plain definition and named owner. Preserve the numerator, denominator and exclusions behind every reported rate. Explain changes to the measurement method separately from operating results. A dashboard colour should not conceal missing information.

State the decision supported by the KPI, the reporting period, eligible population and event that qualifies as success. For an on-time completion rate, specify whether the denominator includes all cases due during the period or only cases closed during it. These definitions can produce different results even with identical underlying work.

Record the source system, extraction time, treatment of reopened cases, duplicates and cancelled work. Define which time zone sets a deadline when work moves between Canada and another country. Have a second person reproduce one period from the written definition; disagreement reveals a control gap worth resolving.

Compare the eligible population with the source register. Explain additions and removals rather than accepting the dashboard total as its own evidence. Check whether filters, blank dates or a join between two tables dropped records. Separate a genuine exclusion from a case that could not be classified.

Keep missing cases visible in a data-quality note. If they could materially alter the conclusion, hold the performance interpretation or present a bounded result with an explicit limitation. A precise percentage does not compensate for an incomplete population.

Assume an invented Toronto team has 100 cases due in a month and completes 80 on time. Under its stated definition, the rate is 80 percent. Ten late cases have blank closure dates and are accidentally removed by a dashboard filter. The displayed result becomes 80 divided by 90, or approximately 88.9 percent.

The apparent improvement is a calculation-population error. No additional case was completed on time. These numbers are invented to demonstrate the effect and are not a client result or sector benchmark. The correction restores the 100 eligible cases, documents the filter issue and reviews earlier periods affected by the same logic.

If management had already acted on the inflated figure, tell the decision owner which conclusion changed. Preserve the original report and issue a clearly versioned correction. Silently replacing the chart prevents readers from understanding why previously reported performance has changed.

For the fictional 100-case example above, keep a compact reconciliation beside the corrected chart.

The difference is approximately 8.9 percentage points. It is not an 8.9 percent increase in work completed. In this invented case, the omitted cases are already established as late; do not apply that classification automatically to unknown records in a real dataset.

Test the repaired logic with three deliberately prepared records: an eligible late case with a blank closure date, a legitimately excluded cancellation, and a duplicate of an existing case. Under the documented example definition, the first remains in the denominator, the cancellation follows the approved exclusion rule, and the duplicate must not create another case. Record expected and observed treatment separately.

Do not report “test passed” until the actual check is performed. If historical data cannot be reproduced, mark the affected trend as not comparable rather than extending the corrected percentage backwards by assumption.

Where a business legitimately changes its definition, document the reason, effective date and expected effect. Decide whether a comparable historical series can be produced. If it cannot, mark the break and avoid presenting the old and new percentages as an uninterrupted trend.

Investigate both the level and the underlying counts. A stable success rate with a sharply reduced population may still be important. Conversely, a larger denominator can expose previously unreported work without indicating that the team suddenly became worse at its job.

Keep the definition version, extract reference, reconciliation, calculation, exception note and reviewer decision together. Automate checks where useful, but retain a person responsible for interpreting exceptions. For cross-border dashboards, ensure local teams understand the common rule and can challenge inconsistent classifications.

Use management reviews that produce accountable decisions to connect the corrected KPI to a management decision. The meeting should establish what the evidence supports, what remains uncertain and which action follows. It should not turn a data limitation into an unsupported performance judgement about an employee.

That depends on the indicator definition and evidence. Do not automatically classify it as success or failure; investigate and disclose the unresolved population gap.

Only in relation to the intended decision. A speed measure can improve while quality declines, so consider complementary measures and the risk of distorted incentives.

Only after checking definitions, periods, populations and operating context. A shared dashboard label does not establish comparability.

NIST Baldrige data and analysis — Live public improvement guidance checked 2 October 2026; voluntary business guidance.

QSE Advisory is part of the QSE Advisory brand family. Preparation included AI-assisted research and drafting. Examples are hypothetical.

This article is general information. Verify current requirements and seek qualified advice for your specific circumstances.

Spotted an issue? Contact our editorial desk.