At the iHub, we work on multiple projects with multiple teams sometimes all at once. The process of software estimation is perilous at best even when there is only one team working on it.
Our situation is even more complex with individual developers serving in multiple teams and sometimes in a team of one!
Getting to consistent estimates across this various scenarios is not easy. In this entry, I will share some of the tips we have found to work in the past.
Find a common denominator
Estimates usually come in two flavours, days or story points. Ideally, estimates should always be in story points from which you derive days of work.
Unfortunately for most people, story points are just not intuitive. This means they make a great tool for establishing the work to be done but not a particularly great one for communicating the same.
Thus to establish consistency in communication, all our estimates are communicated in ideal days.
To be sure, this has come back to bite us once or twice where the client confused ideal days with calendar days but all in all the gains in shared understanding made for a worthwhile trade-off.
Review stories from similar past projects
Hofstadter’s law states:
It always takes longer than you expect, even when you take into account Hofstadter's Law.
Obviously, this is not a physical law, but you would do well to be mindful of it.
Thankfully, well measured past projects cut through our biases and reflect on us the truth.
For example, one of the more successful projects we worked on, our initial estimate was 5 months, as an e-commerce platform we were sure we understood everything related to it. In the end, it took 8 months from the first commit to a fundable version.
In current projects, we keep this hard lesson in mind as we think of the estimates we provide to clients. Particularly if a client wants a similar application, we now know how long it takes to get it done.
By having such a repository of past projects and the time it took to complete it, disparate teams can base their estimates on them and come up with consistent results.
Estimate some stories together
I expect developers to be very good at writing clean code that has a solid architecture with minimal coupling. I am however quite forgiving if they are not exactly masters of the black art of software estimation.
Don’t expect it to be all smooth sailing, especially for well-established teams. They will question it. Unfortunately, it’s very easy to convince yourself that by virtue of your title you can push the practice and have it adopted whole heartedly
What you want to think of is the distinction between compliance and commitment.
Here are some of my suggestions for working with your sceptics.
Let them see the benefits
It’s bad enough you are introducing a new system for them to use it better work!
In leadership, it’s easy to assume what you know is correct and others are wrong. This dangerous assumption means you push practices that have no other value besides inflating your own ego.
If the practice is beneficial, then let the people see it. By adopting the practice, you should be able to show them via anecdotal evidence of its success and how it benefits them.
The knowledge that is painfully obvious to you is likely foreign and unintuitive to your team.
Hanlon’s razor captures it beautifully:
Don't assume bad intentions over neglect and misunderstanding.
Training can be very useful in this respect, by running even an hour session with the team showing them how the process is done, you can make a lot of headway in getting them to adopt it.
Celebrate vocal sceptics
There is always that thorn to your flesh. The individual who is very vocal about your changes and how stupid they are.
The natural human reaction to such hostility is to close them out or attack them.
I suggest you instead keep an open mind and assume the individual is questioning the methods not you as an individual.
As such, invite them to all relevant process improvement sessions, let them air their views. Genuinely take them into consideration and address what can be addressed.
Hopefully, they can become your champions, if not, then maybe they can disagree but commit.
Have Backbone; Disagree and Commit. “Leaders are obligated to respectfully challenge decisions when they disagree, even when doing so is uncomfortable or exhausting. Leaders have conviction and are tenacious. They do not compromise for the sake of social cohesion. Once a decision is determined, they commit wholly.”
~ Jeff Bezos
Challenge them to do their best so that they can prove you wrong
All processes introduce a level of drag into the system, in effect what was getting done, gets done slower. The benefit usually lies in improved quality and less rework.
That’s of course if your process is a good one, there is always the chance you are wrong and unnecessarily burdening your team.
So teach the team the easiest way to shove it in your face is to do their best at this new practice and if no benefit accrues, you will be forced to reconsider.
While obviously a dangerous move, they can choose sabotage, if it works it will provide compelling evidence, enough to move them from compliance to commitment.
Show don’t tell
This is a particularly useful heuristic in many areas of life including the change in the process.
One of the great commandments of science is:
Mistrust arguments from authority
You want your team constantly engaged in a constructive disagreement with you, that’s how you get the most creative ideas out of them. If they become “yes men” your innovation capacity effectively shrinks to just one person, you.
By allowing for disagreement you then have the chance to show your new method will be of benefit to the organization of which you are all part of.
How do you introduce new practices in your own organization? Talk to me in the comment section below or on my twitter @jchex
A few weeks ago, we had the chance to do Myers-brigs at our office. As it turns out I am an INTJ. You can find your own type on this site human-metrics.
As a base rule, mixing individuals from different personality types leads to better creativity albeit at the expense of efficiency.
Developers, in particular, have an even more nuanced environment. Fred Brooks famously said:
The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures....
This means as you grow your team, there are certain things to look out for. In this entry, we will be exploring the factors I think are the most important.
Does the team structure compliment individual’s strengths?
We perform best from a position of strength, yet somehow team leaders assign individuals to tasks optimised for the organisation rather than for the individual’s strength.
A simple test such as the MBTI can help you easily see who would work best with who.
As Wegner proposed in 1985 the greater intelligence born of the group mind which comes about from great collaboration is guaranteed to benefit your organisation.
Is anyone multitasking?
Human’s can not multitask. The illusion feels so powerful that the team leads don’t even need to push this agenda, the developers will take it upon themselves to do it!
Two projects are optimal for an individual developer, in this way, they can switch to a different task to give their mind a break and some room to chew on the project. Anything more is likely to lead to reduced productivity as the mind becomes too clouded.
Have you limited the team size?
We have limited cognitive ability in terms of the number of individuals we can maintain in our circle. Known as Dunbar’s number, it is shown we can actively keep at most 150 active social contacts.
I believe working groups are even smaller. Once the team size gets to more than 5, communication issues start sipping in. It becomes that much harder for the team to build and maintain a shared mental model.
Mike Cohn has an even simpler rule, no team should be so big it can’t be adequately fed by two pizzas!
What is the team’s purpose?
Shortly after joining Moringa, I was having problems getting the team to work together let alone towards any goal. I explained my dilemma to the CEO. She asked me, “Have you carried out a values exercise?”.
This question changed how I think about teams, yes we can impose what we want to the team, but the team just like any other complex system will work towards its own goals.
The only way to get your team to succeed is to imbue them with a sense of purpose which aligns with the organisation.
Can the team deliver the product from end to end?
A basic tenet of the scrum and agile methodologies, in general, is the concept of the cross functional team. The simplest definition I could find of such a team was:
A cross-functional team is a group of people with different functional expertise working toward a common goal.
Cross functional teams are the killer feature of agile. They bring about diverse views to the teams enabling good ideas to be quickly tested, built and shipped.
Scrum is a fast moving development style. There is obviously a lot of advantage to this, but just like a sports car needs a better technical design to function at top notch, so does your code.
In this entry, we will be looking at the top three technical practices that you must adopt to ensure you are moving as fast as you could be.
Test driven development
A few days ago, our team was faced with the challenge of stress testing one of our applications. One particular use case proved particularly hard to do, the only way to run it was to literally have an actual human run through the steps!
This might be fair if we were doing UX testing, but in this case, it proved to be a big red warning sign on our architecture.
We have since worked to correct our process, but the point here is TDD helps you unravel problems before they become Problems.
By writing the test first then the code, you are forced to think through what it is you actually want to do. This makes your code clearer as well.
Not to mention, by having a “second opinion” on the validity of your code, you can deploy far faster knowing another system is checking the integrity of the entire product.
If you have ever struggled with your weight, I am certain this thought has ever come up in your mind at one point or another
Let me just eat this burger, I will work it all out later in the gym
A fantastically bad idea! Not because you will procrastinate later and not visit the gym, thou this is likely, but because you essentially threw a good chance of working towards your goal and worse compromised your future self.
In the same way, almost every day you jump into your code, you will come across an opportunity for improvement perhaps a better design pattern to adopt or even just to maximise on an opportunity to reuse code. Here you have the choice of either saying you will just do a patch now and fix it in the future or just fixing it now.
The formal term for it is refactoring and there is a wealth of knowledge out there on how to do it.
By continuously improving your code, not only will your code not rot but you will have code that gets better with age.
We all care more about what we know someone else will look at than what we are sure only our eyes will see.
The long and short of it is you want as many eyeballs on your code as possible. Obviously open sourcing your code may not be the ideal solution, but what about having your team members work on the code together?
There shouldn’t be a part fully designated to only one developer, this gives you two key advantages:
Not everything goes to hell if the developer goes on holiday
Everyone cleans up their code just a bit better
In 2001, Paul Graham wrote one of the most prescient essays I have ever come across, it is titled The Other Road Ahead.
One of his key arguments revolves around the fact that faster code-test-deploy cycles lead to higher quality code.
I fully agree with this argument. Continuous delivery is guaranteed to significantly reduce if not eliminate the stress which comes naturally with software development.
No longer will demo day be a day of panic. For all intents and purposes, it will just be another day in the life of the boss developer.
I hope these pointers will help you towards a better scrum experience.
Which practices here do you use in your own practice?
The “+” in FURPS+ also helps us to remember concerns such as:
The idea is in addition to doing what it does, software is also required to do it well.
How effective do you think Google would be if it took 5 minutes to give you a response to your query?
I believe by categorising your backlogs in this way, you will able to get more ideas on what is to be included in the final product as well as appropriately prioritise the various backlogs resulting from it.
How do you categorise backlogs in your own organisation? How did you handle it? Talk to me in the comment section below or on my twitter @jchex
This situation is so common, it is by now something of an industry standard. There is almost no project that I have stepped into that had an appropriate amount of time scheduled for it.
You have to wonder why after all these years we still suck at allocating enough resources to get the project done in time.
In this entry, we will be looking at some of the common factors that lead to unrealistic schedules.
Development happens in a context. Unless it is a hobby project, you can be sure that whatever you are working on, someone else is depending on your work output for them to do their job.
A special case would be where the bid specifies the timeline in which the product must be built. In this case, submitting a longer timeline, even if justifiable may mean losing the project.
Since the project is still of interest to the team, the team takes it on knowing full well how tight the deadline is in the process displaying a mix of childish optimism and wishful thinking.
Best case estimates
Murphy warned us:
Whatever can go wrong, will go wrong
Scrum and other agile techniques give us robust estimation tools. But even with those, we more often than not end up with a range, say 3 – 5 months.
Unless you don’t like promotions, who wants to go to the boss with the higher estimate?
Of course, the problem with the lower estimate is that it holds the implicit assumption that everything will go right.
This is a dangerous assumption, you take it for granted that you have the best tools, language, physical environment, skills etc. When something does eventually go wrong, you find yourself with an unrealistic schedule.
Feeling up to a challenge
For reasons that are beyond me. Some developers prefer working under insane conditions. Something about an all-nighter makes them feel like heroes.
Whether inspired by cut throat work environment or personal bias, teams in these kinds of environments will consistently underestimate how long the project will take.
Even if the pressure is acceptable in the short run, the effect on the team morale and product quality are bound to suffer. Better to have a reasonable schedule and spend any extra time beefing up the quality.
Times of need can be artificially manufactured by creating deadlines. Our brain doesn’t know the difference. When you create deadlines, your brain stops procrastinating and gives you what you need.
It is easy to extrapolate this kind of thinking to the entire project.
While you do need to pace yourself, you need to be very careful of the kinds of commitments that you make. In a previous entry Estimates, Targets and Commitments. We looked at the dangers of confusing the terms.
You don’t want to make a commitment to your client based on the belief that somehow the deadline will boost productivity if you must make such a commitment, let it be internal.
Inevitably, new ideas will come in. Change is a core part of the software development process.
The problem arises when new changes are introduced to the project but nothing is removed from the backlog. As new features are added, eventually the entirety of the work pushed the project to an unrealistic one.
We have looked at this problem in greater detail here Managing scope creep. The TL;DR is any new change is also a chance to renegotiate commitments.
Have you faced unrealistic schedules in your own practice? How did you handle it? Talk to me in the comment section below or on my twitter @jchex
You have just completed your development sprint, done the demo and the review, what next?
The easy answer is jumping into the next sprint. That is of course until the end of the year when reviews are up and you are asked to specify what you have been doing the whole year.
Showing working products is great, in fact, the agile manifesto clearly states:
Working software over comprehensive documentation
Note that they didn’t say No documentation!.
Even if they did, what seems very clear in your mind right now about how the days were used may not be so clear months from now.
Thus my recommendation is that you write a brief report for yourself. It does not need to be official. You can even just call it skipper’s log.
Below are some suggestions on what to include in the report.
Not everything that happens in a sprint will be seen in a demo. Perhaps Rico, your new developer, mastered SQL or your team was acknowledged during all hands and other search achievements.
If you do this consistently enough, you will have your team’s story, one that perhaps you can celebrate at your end of the year party, use to review salaries or make decisions on how and what to invest in the team.
Of course, the start and end dates of the sprint are obvious, how could they not be? Still, the past has a way of blending into what seems like one long day. In this cases, it helps to know that the Sprint commenced on 5th June and ended 15th June and that within this period there happened to be one public holiday or a team mate’s birthday.
This information may give you insights in future planning sessions, specifically how blocked of days for development play out in practice.
Thankfully, most project management tools aimed at software teams will automatically compute this for you. If yours doesn’t. It may be worth it to do the computation yourself.
Velocity = Total story points done in this sprint
Average Velocity = Total story points complete/Total sprints done
You can then chose to plot this information in a burndown chart.
You may want to consider printing out the burndown chart and hanging it on your team wall if you have one.
Your retrospective will likely come up with good practices for the team to implement in the future. While this information is already captured in the retrospective document, I consider this information so important that a little bit of redundancy is allowed. In short, have it in your skipper’s log as well.
This will allow you to quickly reflect on what you thought at the time was important to improve on and thus see how things have evolved since then.
This would be a subjective account of how the sprint was. With all the talk about objectivity in our industry, telling you to record your subjective experience sounds rather counter productive.
Still, I have found that AHA moments can come from reflecting on your state of mind in action.
What do you, as an individual do at the end of the sprint? Talk to me in the comment section below or on my twitter @jchex
One of my colleagues at iHub has an interesting work philosophy. He usually says:
I will not work on any project that is obviously a waste of time no matter how much I am paid.
This obviously reflects a great craftsmanship ethic. Such a philosophy does remain useful throughout the entire organisation. No one wants to waste resources on projects that are not going to bear any fruit.
In this entry, we will be looking at how you can test if your application is fit for its purpose.
That is, does your application generate value for your users?
Are your users able to achieve the stated goals?
User stories are a potent way of expressing requirements. I have previously talked about them in more details here Invest in user stories.
They specify what a user is trying to achieve and why.
As you are developing your application, you need to continuously reflect on your user stories and the emerging application asking yourself the following questions:
Can the user meet their goals?
Are there any barriers?.
An example of an issue that may be noted is particularly slow load times. If a user has to wait more than a few seconds to say input a new item on their shopping list, they may quit the application having not met their goal.
How are users using the application?
A software application is a type of system and thus displays systems behaviour. In particular, no matter the goals of the designer, once the system is live it starts manifesting its own goals.
This will be evidenced as users using the application in ways that were not envisioned by the designers.
For you, this is a great learning opportunity. The new uses can be detrimental to your own interests or they can be beneficial and thus help shape the product roadmap. Either way, you gain value from observing how people use the application in the wild.
How are the users already solving the problem?
Evolutionary biologists have a saying:
Evolution is cleverer than you are.
Humans are incredibly creative. Your target user has probably found a way of solving their problems in some other way. It may not be as effective or efficient as yours but be assured that it is there.
Before word processors, there were text editors and before that there were notepads. More generally, ideas are conventional, they have a history that you can track to gain the deeper understanding of the problem and how the solutions have evolved over time.
By looking at how users are currently solving the problem, you can minimise the learning gap and have your users being productive in your application ASAP.
Can you test competing ideas?
In the course of development, you are likely to come across two or more viable ways of solving the problem. If they both seem to have equal weights, why not let the users decide?
For web applications, this is particularly simple, you can use A/B testing.
In this method, you go through the following steps:
Define what you are trying to test eg Home page with carousel of your products or one with your latest offers
Define what metrics matter to you eg click through rates
Develop the MVP for both pages
Roll it out randomly assigning incoming users to the pages
Monitor your metrics on both pages
Make a judgement call based on the data
This simple method gives you unambiguous feedback on how to proceed with development.
How do you gauge the effectiveness of the applications that you build? Talk to me in the comment section below or on my twitter @jchex
Development in Scrum happens with iterations or sprints. An iteration is a time-boxed period within which development is done. During an iteration, the team transforms one or more imprecise requirements statements into coded, tested, and potentially shippable software
Iterations are a core part of Scrum. They allow for the development team to quickly converge towards building what is of most value to the customers.
The iteration length is typically 2 – 4 weeks. This is not a hard figure. Some teams have iterations that are as short as 1 week and others have 8 weeks sprints!
In this entry, we will be discussing some of the factors that lead to variability in iteration length and what you should be considering as you are deciding on your own iteration length.
Length of the project
Projects tend to vary in length. I have worked on projects that lasted only a month and some multi-year.
Longer projects offer the luxury of a window of correction. That is if after the end of an iteration the client discovers what was built is not what they expected, there are a number of other sprints that are available to the team to correct the issue. Shorter term projects lack this kind of luxury.
Thus the length of a project acts as an important factor in choosing sprint duration. The longer the project length, the longer the sprint length you can afford have. In fact, if you already know how many reviews you want to have, you can derive the length as follows:
Duration of sprint in weeks = Total project length in weeks / number of reviews
For example, if the project is 2 months and you want at least 4 reviews then the duration of the sprint will be
Duration of sprint in weeks = 8 weeks / 4 = 2 weeks
In addition to allowing for more review periods, longer projects can also more easily afford the “costs of iteration”. The costs of iteration refer to the extra time used to facilitate iterative development. This would include such things as:
This means that for longer term projects, you can afford longer iterations and associated time costs. For very short projects, the costs can also influence the length since you don’t want to spend all your precious development time on this supporting activities.
Clarity on product and team
Some projects are simple even if they are not easy. This means that it is easy to communicate what is required and changes are not likely to come during the development phase. These kinds of projects are very rare to come across, most projects are messy especially at the beginning.
Thankfully Scrum thrives in the kind of complex projects that exist in the real world. It does this by ruthless application of Palchinsky principles.
First, seek out new ideas and try new things; second, when trying something new, do it on a scale where failure is survivable; third, seek out feedback and learn from your mistakes as you go along.
You see, iterative and incremental development is about generating feedback, learning from it, and then adapting what we are building and how we are building it. Sprints provide teams with the mechanisms for doing this.
Thus the more uncertainty you have in your project, the shorter the sprint should be.
Uncertainty tends to come in two distinct flavours. First one is, what are we building?. This is the product question and relates to meeting the client’s conditions of satisfaction. The second one is, how are we going to build it?. This relates to the team’s development practises. You may, for example, want to run short sprints at first both to quickly show the client what the product will look like and establish the team velocity.
Since for highly uncertain projects the priorities change quickly, shorter iterations ensure that what you are currently working on is what is most important.
At the end of the sprint, the development team needs to do a demo to the stakeholders. Your stakeholders maybe extremely busy or important individuals whose time is hard to secure. In this cases, it may be wise to plan your sprints according to their schedule.
For example, if the executive team meets once every month, it may make sense to schedule your sprints so that a demo day falls on that day. This way the hard work of convening all of them is already done for you.
You, of course, don’t want to vary sprint length from sprint to sprint. Thus you don’t want to unduly sacrifice your process in service of the demo day unless a good synchronisation opportunity exists.
What iteration do you think induces more stress, 2-week or 4-week iteration?
I have asked this question to a good number of my friends, the answer is almost always 2 weeks. They are wrong.
It may not seem like it, but longer iterations tend to induce more stress than shorter ones. This is because longer iterations have well defined, beginning, middle and ending points. Naturally, the team is a bit more laid-back at the beginning and gets more stressed as the deadline approaches.
Shorter iterations smooth out this effect, the team can then settle into a manageable tempo.
See this masterful piece by Tim Urban
How do you select iteration length in your own organisation? Talk to me in the comment section below or on my twitter @ jchex
Change is an integral part of the software development process. It’s what makes software soft. In fact, change is so important in software that it get’s it’s very own line in the agile manifesto
Responding to change over following a plan
It’s been a long time since software teams believed in the unchanging quality of requirements documents, but we are yet to fully understand where change comes from and how it affects us.
It costs far much more to make core changes during development than it does at requirements. Boehm and Papaccio estimate it at 50 – 100 times more. Even with the advent of agile techniques, it is still worth it to control change as much as possible.
In this entry we shall be exploring the various sources of changes, this should hopefully help you anticipate and identify impending changes as soon as possible.
Change from the customers
Of all the sources of change, this is the one worth most consideration. The reason is that as a consultancy or even engineering department, you want to harness change as a competitive tool for your client.
No matter how well you run your requirements gathering sessions, there are requirements you are bound to miss. This is because knowledge builds on knowledge ideas on ideas.
To illustrate this point, think of a car. The first car was a very basic box with wheels that happened to move. But from this idea, people began asking questions such as:
Why can’t the seats be more comfortable?
Why do I have to be hot in the car?
Why do I have to manually change gears?
Today, these previously extraneous features are deal breakers.
In the same way, once development has started, your customers are guaranteed to have new ideas as they see your progress.
In this kind of situation, you want to carefully weigh your options. You don’t want to do exactly what the customer wants you may end up always chasing after the new shiny thing. You also don’t want to completely ignore their wishes under the guise that you know best.
Change from sales and marketing
This part particularly affects product companies.
The product owner declares this Big Hairy Audacious Goal
We will build a one stop best in class application within the next 6 months
This is a great goal and everyone is excited about it. The engineers get to work “banging” it out. Somewhere within the development window, the competitor does a release that you had not even thought about. The sales team also wants to be able to sell the new feature so they insist that is also put in the product after all our product is best in class right?
It is tempting to think of your application as a big checklist of features, this is dangerous because then you will be locked in competition with other firms to produce more and more.
Perhaps then the product owner should have come up with a better goal say
We will build a product in the next 6 months that will improve our customers margins by atleast 10%
Change from developers
Developers, my favourite group!
More than any other stakeholder, developers have the highest intellectual and emotional investment in the project. They are the ones who had to deal with the design flaws, unexplainable bugs and late night coding.
Yet to them, requirements are usually provided as a checklist. In this case am also looking at a list of user stories as a checklist. Without an organising theme, developers will do what they do best, write the best-designed code that they can conceive off.
The problem here is that the devil is in the details. Your DBA may end up writing code that shortens query time from 10ms to 1ms. This is great, except for the fact that no one really specified that they have that requirement. Another example would be where your front end engineer goes out of their way to guarantee that the settings page will work for all screen sizes.
I believe by actively watching out for this sources of change, you will be in a better position to manage them when the changes do come.
What are the sources of change in your own projects? Talk to me in the comment section below or on my twitter @ jchex