What to measure

My colleague, Kirui, wrote a very intriguing piece on using data to manage his team. I started using a data-driven model to manage my team — here are the results

As you already guessed, I am a big believer of data myself. At the end of the article, he makes a call to you to also join the bandwagon, but with the caveat that your model will likely be different.

In this entry, I will talk a bit more about the adoption of such a model and how it would look like for you.

To start with, the obligatory quote.

    What gets measured gets done

This is the mantra that I live by. You see, in most software teams, metrics usually take the back burner with the flawed assumption that what they are doing can’t be measured or computing metrics takes time from critical coding activities

The first assumption can be easily refuted by observing that anything that matters must have observable consequences to a degree.

On the second, the truth you rarely need more than two or three metrics. In fact, I propose that a metric is only useful to the extent that it informs a decision.

Then to get your metric, you must answer the following three questions:

  1. What concept are you trying to measure?
  2. What defines the Goal?
  3. What can show you if you are making progress towards the goal? (The metric)

This method is called the GQM model, advanced by Victor Balasi GQM

Let us start with Kirui’s model:

What concept are you trying to measure

How can I keep my business sustainable while having full visibility into its operations

What defines the goal

Are we profitable? Can I know in advance if this is about to change?

What can show you if you are making progress towards the goal

Billable hours/non-billable hours Future opportunities expected value

For a software team, the questions are likely to be answered a bit differently

What concept are you trying to measure

Are we still on time and budget?

What defines the goal

How much work is still remaining? How much money is still remaining? How much value has been delivered?

What can show you if you are making progress towards the goal

Features developed / week Customer ratings

As you can see the metrics to be used are quite specific to your goal, so then what are the common characteristics that would make a good metric?

Simple and Easy to compute

Software development should be fun, whatever metric you chose to use should be simple to use and compute.

An example of a good metric would be:

Velocity = Story points delivered/no of sprints

An example of a poor metric would be:

Velocity = Lines of code/ no of hours

This metric is hard to compute as it requires investing in some kind of time tracker and a line counting program.


Data is just a tool. For it to be of any value, it must be of use to the person utilizing it. In this sense then, you must consider who will be using the metric to make a decision.

Eg: Utilization rate, is a good metric. Anyone can clearly understand that a 10% utilization rate is bad. Return on resources is not a good metric. Although it conveys similar information, it is not intuitively clear, in fact, it comes off as a bit judgemental.


Metrics measure a concept. The only acceptable reason for a metric to change is if the underlying concept has changed.

As such you want to avoid subjective measures as much as possible.

Show progress clearly

If you have made steps towards your goal or otherwise, the metric should reflect this. Even better, the metric should lend itself to extrapolation so that you can “see the future”.

Metrics such as Profit/Quarter or Features/Sprint are always positive if the number is higher. A metric such as LOC/Hour may be good if higher or may indicate the developer is bloating the product.

What metrics do you use in your own organization? Talk to me on my twitter @jchex or in the comment section below.


When to ignore historical velocity

George Santayana once said:

    Those who cannot learn from history are doomed to repeat it.

This theme holds even when we are doing our estimations. One of the best ways to predict your velocity for the coming iterations is to simply look at the velocity that was attained in the past sprints.

More often than not, the method will prove itself as the most effective and efficient ways of making reasonable plans. But what about the less often times?

Nassem Taleb puts it beautifully:

    If the past, by bringing surprises, did not resemble the past previous to it (what I call the past's past), then why should our future resemble our current past?

So then it today’s entry we are going to explore some of those times that you may want to be a bit more awry of your historical records.

Change in team members

Scrum advocates for tightly neat team of specializing generalists. More generally, you want a jelled team working on your project no matter your chosen process. A jelled team is a group of people so strongly knit that the whole is greater than the sum of the parts (Tom Demarco).

Yet this process takes time. So when you have to add or change team members, be sure that it will take time for jelling to happen. As such you can expect a drop in velocity.

In addition you also consider the impact of ramp up time for the developer being onboarded.

Change of product owner

In a scrum team, the product owner is the individual or team that dictates the conditions of satisfaction and prioritizes the work to be done. You can think of them as the client’s representative in the team.

Although the product being developed is still the same, it would be naive to assume that different product owners will provide feedback with equal efficacy. As such when the client changes the product owner you should brace yourself for a change in velocity.

Change of estimation team

Each team member, including the product owner, have a role to play in coming up with the estimates. It is not always possible to have the team work on estimates, say if you have been asked to estimate a story to be developed in the future and the team is just not around.

While relative estimates tend to be consistent, they also carry a systematic bias. A systematic bias An inherent tendency of a measurement process to favor a particular outcome. Thankfully this bias is easy to correct but some observations are needed to determine the new velocity. Thus if a different team did the estimates the velocity is also likely to change.

Change in technology

The technology landscape changes faster and faster everyday. When a change in the outside technology factors inevitably hits you, then you can say goodbye to your previous estimates.

This change can come in multiple forms including:

  1. New developer tools
  2. Upgrade of service that your product depends on
  3. Change in industry protocols
  4. Upgrade of language version


Organizational change

The development team is part of the ship that is the business. As the scrum master, you must keep your radar out for organizational politics. Anything that affects the livelihood of the team is likely to affect their productivity as well.

Even changes that may seem trivial such as being moved to a different office space can have adverse effects on productivity levels.

New project

It might seem obvious, but really a change in the project that the team is working on is likely to yield a different velocity from the historical one.

A similar project can give a feel of the size of a project, but that is it. Every project comes with its own people, requirements and other nuances that can not be ignored. So by all means use the estimates from previous projects for sanity checks but not as absolute guidelines.

Has your team ever experienced a change in velocity? What was the cause? Talk to me on my twitter @ jchex or on the comment section below.


Running an engaging standup

My distaste for meetings is widely known. Anyone who has been around me long enough knows that I consider most of them to be nothing more than an expression of power. So as you can imagine, I faced a good amount of internal struggle when I learnt about the concept of a daily meeting!

In scrum there is a daily meeting called a standup. Scrum in 4 easy steps. Unlike the other meetings, this one held promise. For starters technically people had to stand during the meeting, and even better, everyone gets a chance to talk!

My experience with the practice has been amazing. At this point my advice to anyone testing out scrum or any other agile methodology is to first implement daily scrum, see the benefits then roll out the other practices. What led to such a dramatic change in mindset? Well quite a number of things actually, in summary

  • Clarity: Nothing brings as much clarity to a project as much as knowing what everyone else is doing. As I always say, The speed of a project is largely determined by how well everyone understands what is going on..
  • Internal commitment: In a standup, you promise to do something in the coming 24 hours. This promise is not made to a “boss” or a stranger. Rather, it is made to peers who you feel some camaraderie with and would not want to disappoint.
  • Progress: There is no bigger motivator than a sense of progress. Standups provide the perfect avenue to celebrate milestones hit on the way to the goal. Also, no one wants to be the guy who has nothing to speak of at the meeting. Thus providing, even more, impetus to push things ahead.
  • Early detection of problems: Blockers get exposed almost as soon as they are discovered. This then ensures that they can be corrected when the cost of doing so is at its least.
  • Reduced documentation. What is not said in a standup would probably have been said in some lengthy document somewhere. Now documentation is important, but lengthy documentation is counter productive, it hides potentially useful information in a flood of obvious material.

Hopefully, by now, I have convinced you to at least give it a try. If so then here is the process by which to run one.

  1. Begin with all developers and the scrum master seated around a table
  2. Go around the table with each of the developers answering the questions: – What did you do yesterday to move the sprint forward? – What will you do today? – What are the obstacles in your way?
  3. Ensure that the meeting lasts a maximum of 15 minutes. If anything has not been said in that time, then have that discussion in a separate meeting. A long meeting is the worst way to start a day.
  4. As the questions are being asked, have the team looking at the sprint board or trello board for that sprint. It goes without saying that if you are using a physical board then the meeting needs to be held where the board is.
  5. If a person strays, say they start providing a solution to the blocker, then set time for a sidebar. Feel free to add that as a task item to the board
  6. Ideally, the meeting should happen the same time every day and preferably in the morning hours.
  7. Ensure no task is left on the Today list. If it has not been completed, then take it back to the backlog. Remember half complete features are worthless.
  8. For what is not met, you are to query gently the developer to know what went wrong and how the other developers can help out.
  9. The last person to arrive at the standup speaks first followed by the others in order of punctuality.
  10. Have developers speak of only what is on the board, if it is not on the board, then add it!
  11. Ensure that the developer actively answering the questions is addressing the team and not the facilitator.

By carrying out the practice as outlined above, you are likely to benefit in the way that I did.

Do you do standups in your organization? Tell me in the comment section below or on my twitter @jchex