A Non-Quantified Unsolicited Opinion on Metrics for Business

Since I've started on this metrics/too much information kick, I figure I might as well write a couple other blog posts that have been rattling around my head and can be loosely grouped together.

In this post, I'll go into a bit more depth about crafting the right metrics for a team or organization.  I can't promise that I've got a magic formula or won't digress on some tripped-out tangent.  In fact why even bothering to promise?  This blog would be nothing if it weren't tangential.

Let's start with the one metric, from a business sense, that matters most - and one that's surprisingly overlooked more often than you expect is - money.  More specifically cash flow.

The important thing to focus on is "how much money am I bringing in?"  Notice I didn't say revenue or cost-cutting specifically.  You can obsess over revenue, but that won't matter if you decide all of your office furniture needs to be crafted in solid gold.  Likewise, you can be very stringent on costs, and it won't matter if you have no revenue (this is my current business model - I spent all of $1038 in 2024 and cleared all of $0).

Businesses often behave in a Jekyll and Hyde fashion when addressing money from the viewpoint of revenue of cost-savings.  When they're in revenue-generating mode, they'll disregard projects that could save significant cost simply because they don't bring in revenue.  This is akin to arguing that because I received $3 but had to give back $2, I'm richer than you are because you received $2 and had to give back $1.  We're making it up in volume! [whispers: no we're not]

When they're in cost-cutting mode, they'll ignore the effort and expense needed to make small incremental cuts simply to meet an arbitrary cost target.  The cynic in me would say this is tied to some VP's need to hit a very specific target to achieve their own personal bottom line, but it's a new year, so why start off a cynic (plus, soon, it'll be illegal to make fun of billionaires)?

This need to focus on either top line growth or bottom line maximization is further diluted in the bowels of various departments or teams that are decoupled from the immediate impact on the company's cash flow (say, an infrastructure team within the Tech org that creates systems responsible for allowing other systems to run smoothly in order to generate cash).  It's not uncommon for teams to set goals that run counter to the operation of the business, or at the very least, tread water in moving it forward, because they're so far removed from the actual business product.

I've encountered this most frequently in discussions with engineers about reclaiming technical debt or rewriting Service X using some new technology for some vague reason.

As I've mentioned before, technical debt should be prioritized when it's affecting the business directly.  Trying to determine other definitions of technical debt simply pits individual ideologies against one another and results in a standstill while tech debt continues to bloat.  

Code should be refactored when it's obvious that making changes to it is a common cause of bug injection.  It doesn't need to be quantified as such in order to build one's case, but if everyone grimaces when asked to make changes to the code in question, that's a sure sign it needs attention and feature creation should slow in favor of re-upping the infrastructure.

Determining when to use new technologies is a trickier path to navigate.  New technologies often provide tangible benefits to the business and those benefits aren't always immediately obvious, so managers need to provide a wide berth when permitting exploration.  On the other hand, re-writing a service simply because it's using the latest technology or trend (everything needs to be asynchronous because I read a TokTok!) can cause knowledge of the system and business to backslide, since there's no stability.

I don't advocate tying everything every team does directly to the bottom line, but the metrics that teams track should have a direct line to revenue generation or cost savings.  I've mentioned DORA metrics in the past, and I think they're a great indicator for measuring the health of a Technology organization within a company.

As a refresher, the 4 (or 5, now) metrics measure the speed with which the org can deliver features while maintaining system stability.  Those metrics track well with the overall business health but doesn't require an SRE team to justify its existence by determining exactly how they're contributing to revenue generation or upping their A/B test count.

In fact, paying close attention to just those metrics can help guide broader areas of consideration within an org.  If site stability is suffering, the first question, of course, should be - why? Once that's answered sufficiently (which can help by pointing to the drop off in DORA metrics), then the next question is - can we handle the issues using existing resources and personnel?

Assuming the answer is 'no,' then the next question is - where should we direct hiring efforts?  At this point, if a team like SRE is understaffed, there's a clear indication that they should hire more to assist with site stability.  The discussion around hiring is now a tangible one rather than the usual vague hemming and hawing about contributing more to cost centers while still paying mild lip service about how important the team's mission is.  And if, after answering those questions, you decide not to hire into the SRE team, you've now made an explicit decision about the site stability, even if it's an unpopular one. Or a poor one.  Or both! Congratulations!

At this point, this post dovetails well with my previous post in which I warn of the dangers of swallowing the alligator whole, or rather, relying on too many metrics before you need them.  There's the main metric that everyone should at least be aware of when it comes to business - cash flow - and there are the derived metrics that a team needs to focus on that allow them to make gains toward the overall goal that make more sense in their day-to-day world.  

Ideally, that would be 3-5 metrics per team.  Realistically, there will be more to track, so there will be more metrics.  However, they should be grouped into relatable themes (like the DORA metrics, for instance), and you shouldn't really have more than 3-5 of those groupings.  Get above that number and you're spending more time managing the metrics that are supposed to track your work than you are doing the actual work that you want insight into.  Talk about the statistical long tail wagging the proverbial bell curve (is there any scenario that old outage can't be applied to?).

And, wherever possible, use metrics that are commonly known throughout your industry or discipline (like the DORA metrics - I believe you may be catching onto a particular theme here).  There's often a lot of literatUre that can help you craft the metricS for your particular domain.  You'll also havE a reaDy-made lexicOn if you want to discuss ways to impRove your systems.  It's Also less intellectual overhead to deterMinE whaT exactly needs to be measuRed If it's already available.  Definitely something to ConSider. 

Until next time, my human and robot friends.