A Metric Is Not Defined Until the Exceptions Are

The request sounds easy.

"Can we show planned work versus completed work?"

Sure.

There is planned work in the system. There is completed work in the system. Put the two numbers next to each other, calculate the difference, and give leadership a trend line.

On a whiteboard, this takes about thirty seconds.

Then somebody asks what should happen when work is scheduled for Friday but completed on Monday.

Now we are building a data product.

Does the work belong to Friday because that is when the business expected it to happen? Does it belong to Monday because that is when it actually happened? Should the plan remain in Friday while the actual appears in Monday? If someone looks at Friday two weeks later, should they see what was known on Friday or what eventually happened?

None of those answers are inherently wrong.

They answer different questions.

This is where supposedly simple metrics become expensive. The arithmetic is rarely difficult. The difficulty is deciding what the arithmetic means when reality stops cooperating with the clean example.

A metric usually arrives as a noun.

Revenue. Utilization. Completion. Headcount. Coverage. Variance.

Engineering eventually discovers that every noun contains verbs, timing, ownership, and exceptions.

When does something become complete?

Who is included?

Which date controls the reporting period?

What happens after a correction?

What happens when the source contains two records for something the business thinks of as one thing?

What happens when one customer configures the workflow differently from another?

These questions are often treated as implementation details.

They are not.

They are the metric.

The dangerous version of this problem is not when everyone disagrees. Disagreement is visible. You can put people in a room and make a decision.

The dangerous version is when nobody realizes a decision needs to be made.

The requirement says "completed work." The engineer finds a completion date. The model filters on it. The dashboard gets built. The numbers pass a few spot checks.

Done.

Except perhaps the operational system also has a status that can be reopened. Or completion can be entered late. Or historical records were migrated differently. Or some customers use a workflow where the field means something slightly different.

The implementation still has to choose what counts.

If the business has not made that choice explicitly, the data model makes it implicitly.

That is how business policy ends up buried in SQL.

Months later somebody finds a discrepancy. Another report produces a different number. Both teams inspect their logic and discover that each implementation is internally consistent.

They simply answered different questions.

The usual reaction is to ask which number is correct.

That is often the wrong question.

First ask what each number is measuring.

One may represent work scheduled during the period. Another may represent work completed during the period. A third may reconstruct the eventual outcome of work originally scheduled during the period.

Those can all be legitimate metrics.

Giving them similar labels does not make them interchangeable.

This is why I am skeptical when metric definitions are reduced to formulas in a requirements document.

A formula like completed divided by planned looks precise. It is not precise until "completed" and "planned" have operational definitions.

The useful specification is usually a short list of decisions.

What is the grain?

Which event determines time placement?

What qualifies a record for inclusion?

How are cancellations and corrections handled?

Do we report what was known at the time or the latest known state?

Can configuration change the definition?

What should happen when required data is missing?

And who owns those rules when the business changes?

That may sound like more process for a simple metric.

It is usually less process than spending months reconciling two dashboards after they reach production.

There is also a product benefit to doing this work early.

Once the definition is explicit, you can decide whether the business actually needs one metric.

Sometimes the argument over the "correct" number reveals that people are trying to answer two different questions with one KPI. Operations needs to understand execution against the original plan. Finance needs the final outcome. Leadership wants a trend that does not rewrite itself every time late data arrives.

Forcing those needs into one universal number does not create consistency. It hides disagreement.

Good analytics makes those distinctions visible before the chart gets polished.

The clean dashboard should be the last step, not the place where unresolved business semantics go to disappear.

So the next time someone proposes a new metric, do not start with the formula.

Start with the ugly records.

Ask what the metric should say when the dates disagree, the workflow changes, the record gets corrected, and the obvious rule stops working.

If nobody knows yet, the metric is not defined.