Here Be Cobras

How to make metrics worth measuring

Justin Andersun

Jun 23, 2022

Cobra Effect

During the British rule of India, venomous cobras plagued Delhi. The government offered a bounty per dead cobra to reduce the snake population, but the plan backfired: Enterprising people began to breed cobras to kill them and earn the bounty money. When the government caught wind of this, they dropped the bounty program, and the breeders released their cobras into the street, exacerbating the problem.

Similar tragedies occurred with rats in Vietnam, poppies in Afghanistan, and feral pigs in Georgia. It’s almost like government-sponsored massacres are a bad idea…

Sadly, ill-conceived population control isn’t the only scheme afflicted by the Cobra Effect. Less tragic occurrences are ubiquitous in modern society. Whenever a metric drives the wrong behavior, perverse incentives are at play.

Measuring the lines of code written per day is a ridiculous metric for assessing the productivity of a software team, as it incentivizes developers to write inefficient functions. Or funding fire departments based on the number of calls they receive could incentivize arson.

Here be cobras!

Metrics drive behavior, so ensure you measure the right thing. To avoid the Cobra Effect:

Use simple incentives. Don’t create complex systems because complexity leads to loopholes, and loopholes lead to perverse incentives.
Get sneaky. When creating a metric, imagine all the twisted behaviors it could drive. Try to “Black Mirror” yourself by picturing the unplanned consequences of this new incentive.

What gets measured gets done—for better or for worse.

Input vs. Output

Imagine a big, strange Dr. Seussian machine. You feed it groceries, and it spits out gold bars or useless goop. You don’t know how the machine works, if it will produce, or when the gold or goop will arrive. All you control is what you feed the machine. Any goal is like this: You own the input to influence the output.

We can measure both:

Input Metrics: Levers that you control, like the number of days spent in the gym, how many words you write, or the volume of sales calls you make.
Output Metrics: Desired results, which you don’t control. Outputs are things like weight lost, stories published, or revenue earned.

Let’s say you own this temperamental Gold-or-Goo machine, and your mustache-twirling boss tells you he needs 100 gold bars by quarter’s end.

“Number of gold bars produced” is your output metric, but you can’t control if, when, or how the machine will produce gold bars. Since it’s not actionable, “100 gold bars by quarter’s end” isn’t a useful standalone metric.

You reflect on the machine’s past behavior and notice trends. For instance, the machine produced more goo after you fed it dark chocolate and more gold after feeding it carrots. There’s not a direct correlation (i.e., one carrot ≠ one gold bar), but there’s a general pattern of carrots yielding gold. An input metric could be the “number of carrots fed.”

To achieve your boss’s goal, focus on the inputs (number of carrots fed) to influence the outputs (number of gold bars produced).

When setting fiction writing goals this year, I focused on two inputs—the number of stories written and the number of submissions sent to relevant publications—to influence my output: the number of stories published.

Focus on the inputs to influence the output.

Baseline vs. Target

How many words do you write each day? If you don’t have a clear and precise number, the answer is zero.

To make progress on any metric, you need a baseline.

To create a baseline, you need a measurement system. For instance, if you’re trying to measure the words you write daily, you could have a spreadsheet to log your daily word count.

Once you collect metrics, define your baseline. You could sum your daily word count for the last month and divide it by 30 days to find your average output. Let’s say it’s 250 words per day.

Using this baseline, set the desired target. You could aim for 500 words per day when planning for the next month. At the month’s end, compare your performance against the target. If you didn’t hit 500 words but were consistently writing 450, maybe you could spend a few extra minutes each day to achieve it? Or, if you missed your target by a lot, reconsider next month’s target.

Most metrics are iterative: Create a baseline, set a target, and get results. The results inform your new baseline and inspire a new target.

Junk Drawer

I’m excited to announce that my inputs (writing stories and submitting to relevant publications) yielded an output! Blue Mesa Review kindly published one of my stories: The Time Donor.

You might like this story if you enjoy speculative/light science fiction.

After you read it, I’d love to hear your thoughts! You can email me directly (justin@andersun.com) or leave a comment below 😊