Metrics as judgments

A police officer arrests the same person dozens of times in order to make an arrest quota. A bank employee opens accounts for customers without their permission because of a cross selling incentive. A teacher helps students cheat in order to improve tests scores so that the school will receive more funding. 

What’s happening in all of these cases is that people are responding to a metric being used as judgment. The metric separates the bad from the good. It assumes that the numbers are a reflection of the skill, effort, or attitude of the people involved; rewards and punishments are doled out accordingly. People react to those judgments and – given sufficient incentives – they work to make their metrics look good rather than achieve the goal the metric was ostensibly put in place to meet.

Now consider a different type of measurement: a person’s temperature. This, too, is a metric. But a doctor never feels the urge to fake a patient’s temperature to make them seem healthy when they are not. Nurses don’t “fudge” the temperature numbers a little to make them look better. Patients aren’t blamed for having the wrong temperature. 

We all know that someone’s temperature is not their “fault” – it’s a measure of a system that tells us something about the workings of that system. We determine the desired state (98.6 degrees Fahrenheit, or “afebrile”) and use the measure to tell us where the system is relative to that state. 

What would happen if we treated temperature as a judgment? If we saw a fever as a failure of effort on the part of the patient or medical team? What if we assumed that a patient with a fever must have a bad attitude, and told them they should look at other patients without fevers, to see how it’s done? Imagine that we summoned doctors with febrile patients into the office to explain themselves, while rewarding doctors with afebrile patients with cash bonuses.

This, clearly, would not be a sane world. This would be a world in which we work against our own progress, where sick patients are incentivized to hide their illness to gain a reward, and doctors and nurses admit healthy people into the hospital just to make their numbers look better.

Now imagine that we treat our organizational metrics – productivity, budget, profit, injury rate, hospital readmission rate – like temperature. What if instead of “holding people accountable” for their suboptimal numbers, we used those numbers to understand the system and adjust appropriately? 

Don’t use metrics as judgments. Use them to understand the system.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s