The Value of Value Metrics

You can’t improve what you can’t measure, or so the saying goes. Whether this is universally true or not, we in IT spend a lot of time working with and thinking about metrics. How is this project tracking against the budget? What is the velocity of that scrum team? How frequently are we releasing to production?

Most software delivery metrics have inherent limitations. They can only convey a sliver of the total information available; they may be misinterpreted or misapplied as incentives or penalties, which can lead to undesired side effects and/or gaming.

There are three main types of metrics for software delivery: compliance, process, and productivity. The first two categories are incredibly valuable within IT. They help IT leaders understand processes, people, and risks better, so that these may be improved. However, they are not particularly attractive to business stakeholders.

Productivity metrics are what the business cares about. These metrics are an attempt to capture the value that Agile teams are delivering. Unfortunately, the ones in everyday use are not very effective. The problem is that they incorporate imperfect analogs for business value. Earned value, story points, feature points, function points, lines of code, frequency of releases—these are measures of productivity of software delivery, and it can be a big leap to translate them to business value.

One of the arguments for Agile over Waterfall illustrates the problem. Empirically, Waterfall projects deliver large amounts of unused functionality, either because business needs change by the time the code was ready or because the feature was unnecessary in the first place. As such, productivity metrics for the project team may have been off the charts, but there were wasted efforts, and the project delivered limited value to the business.

This example captures the crux of it. Delivery of business value is essential; it is what justifies the existence of the development team and pays the bills.

Most productivity measures are too far removed from the front line of business value. Business value from IT is inherently hard to measure; it requires real alignment and a collaborative and cooperative relationship with the business.

How do we generate business value? Our Three Levers of Value framework captures it well: business value comes from selling more, costing less to operate, and managing risk better. Typically, this means somehow translating the delivered software into dollars—dollars earned, or dollars saved.

“Dollars saved” is usually straightforward to calculate. If a new feature reduces time-spend on a transaction by 50%, then it is simple math to work out the business value of that feature. Calculating “dollars saved” by risk management is a little more complicated, but it is usually achievable with some basic assumptions.

“Dollars earned” is a more complex calculation. It is hard to attribute revenue growth to a single factor, e.g., a newly released software feature. Even with a net new insurance product, it can be challenging to determine how much of the new revenue is attributable to the software vs. advertising vs. activities of the sales team, etc.

What can insurers do today to begin incorporating value-based metrics? One simple change is to incorporate value estimates into Agile stories as part of backlog grooming. Insurers can use these value estimates to calculate a “value-velocity,” a measure of how much business value the team is delivering.

The business should provide these value estimates. Product owners/managers who reside in the business should estimate the value of the story or feature as part of backlog grooming. Those who don’t can facilitate the value estimation process. Insurers can estimate value in dollars or a surrogate unit, e.g., value points. It is crucial either way that these estimates be consistent; using a small, fixed group of estimators is best.

Value velocity has some issues, but it is straightforward to implement and forms the basis of more insightful conversations with the business.

Measuring IT productivity is important, but it is the productivity of the entire organization and all its value streams that really matters. Finding alignment around what constitutes value is a foundational step toward that end.

Add new comment

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
8 + 2 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.

How can we help?

If you have a question specific to your industry, speak with an expert.  Call us today to learn about the benefits of becoming a client.

Talk to an Expert

Receive email updates relevant to you.  Subscribe to entire practices or to selected topics within
practices.

Get Email Updates