What to measure

My colleague, Kirui, wrote a very intriguing piece on using data to manage his team. I started using a data-driven model to manage my team — here are the results

As you already guessed, I am a big believer of data myself. At the end of the article, he makes a call to you to also join the bandwagon, but with the caveat that your model will likely be different.

In this entry, I will talk a bit more about the adoption of such a model and how it would look like for you.

To start with, the obligatory quote.

    What gets measured gets done

This is the mantra that I live by. You see, in most software teams, metrics usually take the back burner with the flawed assumption that what they are doing can’t be measured or computing metrics takes time from critical coding activities

The first assumption can be easily refuted by observing that anything that matters must have observable consequences to a degree.

On the second, the truth you rarely need more than two or three metrics. In fact, I propose that a metric is only useful to the extent that it informs a decision.

Then to get your metric, you must answer the following three questions:

  1. What concept are you trying to measure?
  2. What defines the Goal?
  3. What can show you if you are making progress towards the goal? (The metric)

This method is called the GQM model, advanced by Victor Balasi GQM

Let us start with Kirui’s model:

What concept are you trying to measure

How can I keep my business sustainable while having full visibility into its operations

What defines the goal

Are we profitable? Can I know in advance if this is about to change?

What can show you if you are making progress towards the goal

Billable hours/non-billable hours Future opportunities expected value

For a software team, the questions are likely to be answered a bit differently

What concept are you trying to measure

Are we still on time and budget?

What defines the goal

How much work is still remaining? How much money is still remaining? How much value has been delivered?

What can show you if you are making progress towards the goal

Features developed / week Customer ratings

As you can see the metrics to be used are quite specific to your goal, so then what are the common characteristics that would make a good metric?

Simple and Easy to compute

Software development should be fun, whatever metric you chose to use should be simple to use and compute.

An example of a good metric would be:

Velocity = Story points delivered/no of sprints

An example of a poor metric would be:

Velocity = Lines of code/ no of hours

This metric is hard to compute as it requires investing in some kind of time tracker and a line counting program.

Persuasive

Data is just a tool. For it to be of any value, it must be of use to the person utilizing it. In this sense then, you must consider who will be using the metric to make a decision.

Eg: Utilization rate, is a good metric. Anyone can clearly understand that a 10% utilization rate is bad. Return on resources is not a good metric. Although it conveys similar information, it is not intuitively clear, in fact, it comes off as a bit judgemental.

Consistent

Metrics measure a concept. The only acceptable reason for a metric to change is if the underlying concept has changed.

As such you want to avoid subjective measures as much as possible.

Show progress clearly

If you have made steps towards your goal or otherwise, the metric should reflect this. Even better, the metric should lend itself to extrapolation so that you can “see the future”.

Metrics such as Profit/Quarter or Features/Sprint are always positive if the number is higher. A metric such as LOC/Hour may be good if higher or may indicate the developer is bloating the product.

What metrics do you use in your own organization? Talk to me on my twitter @jchex or in the comment section below.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Published by

jchencha

API Engineer