Why have a deployment pipeline


When I was starting out on my software development path, I landed on a handy little trick.

On the root of the project, I would put in a new file deployment.md.

On this file, I would write out all the steps I took to deploy the code. This includes even code to ssh

ssh app@

Use password: secret

This system worked quite well. After all, I was a lone wolf and everything I deployed needed to work on only one server.

Of course, nowadays the systems we write are not as trivial:

  • They require coordination across teams
  • Security is a key concern
  • Downtime is unacceptable
  • Different services run on different servers

With this new conditions, I have learnt to embrace a new concept, automated deployment pipelines.

In this entry, we shall be looking at the benefits to be experienced in deploying the new system.

Feedback to all teams involved

In deployment just as in development, you will come across errors, bugs as it were. You may think this is a problem, I don’t think it is, such is the nature of the world. The problem would be not knowing what is going wrong.

Automation helps in the sense the bug happens every single time, not just when the sloppy developer is pushing their code. This makes it far easier to diagnose and thus permanently fix the issue.

Furthermore, each team formally or otherwise has the DevOps expert. But what happens if the person gets sick and the system goes down?

A proper pipeline is a crystallization of this professionals knowhow, it can be used even in their absence. Even better, other devs can explore what he did to add to their own knowledge.

Less documentation

Documentation is a whole lot of fun, isn’t it?

Without a deployment pipeline, there must be documentation guiding the rest of the team on:

  • The steps to deploy
  • How they will know they have been successful
  • The various error states and how to recover

Even if you are able to successfully do this, the work quickly decays and there is no obvious way of seeing this happen.

A CD pipeline is in tune with your code. The moment it breaks, the code can’t go into production. This means:

  • The deployment pipeline will always be up to date
  • The steps are self-documenting

Basically unit tests on steroids.

Free up time

To carry out a successful deployment. The developer needs to have a working knowledge of :

  • Unix commands
  • Servers
  • Proxies and load balancing
  • Networking
  • Etc

Beyond trivial websites, the work is not child’s play.

Yet after the first time, it is very repetitive. Which creates the dilemma for you, hiring expensive staff to work on rote stuff and bad for the professional who gets to basically bang their head on the keyboard every day.

A deployment pipeline frees up the devs to work on high value creative work. You will notice this in the form of higher engagement from them.

Easy to verify

Suppose the work was outsourced, the consultants then give you a working system together with the source code.

What happens if for some reason it went down and you need to restart it?

Sure, you can always call them back in and trust somehow the developer who worked on it is still employed with them and has retained a working knowledge of your system.

Quite the gamble my friend.

Alternatively, they could show you during the demo which button to push to bring the whole system right back up!

In conclusion, automated deployment will make your life much easier. With the rising popularity of containerization and development of great orchestration tools like Kubernetes, you really have no excuse to still be doing manual deployments.

How do you deploy software in your own organization? Talk to me in the comment section below, my Linked in chenchajacob  or my twitter @jchex


What NOT to do when someone pisses you off in a meeting

Kiss up and kick down is one of the more common phenomena in corporate environments. It makes sense, after all, bosses are able to offer rewards subordinates can’t.

So when you become a manager, you may be surprised to find there exists an entirely different paradigm, kiss down and kick up!

I have had bosses who I have taken to be some kind of demigod. They handled almost everything with grace and poise. To this leaders, I may have at times been aggressive to the point of being disrespectful.

Too commonly lesser managers have reacted in ways which made the situation worse. In this entry, will be discussing some of the things to avoid.

Ridicule the individual who has pissed you off

The need to revenge is deeply embedded within us. The story of Cain and Abel, a story of betrayal and revenge is literally one of the first stories in the bible.

Thus when we feel attacked, we will immediately feel a rush of emotion pushing us to give ’em all we got.

It doesn’t help our homegrown culture of “Mchongwano” has trained us on the art of snark.

Why it doesn’t work

Your reports know more about you than you do about them. If you think about it, there is nothing strange here, as their manager, you hold a dispropriate amount of control on events in their life. Companies depend on managers for reviews which then inform bonuses or even state of employment. To understand your boss is a survival strategy.

This means what you say may be taken personally affecting the person’s view of themselves. Even worse, you close yourself off to future feedback from the person and anyone else who may have been watching.

What to do instead

No one owns your emotional state. If you are angry, no one made you angry, you made yourself angry. Events in the world just are.

As Ryan holiday put it:

The perceiving eye is weak, he wrote; the observing eye is strong. Musashi understood that the observing eye sees simply what is there. The perceiving eye sees more than what is there. The observing eye sees events, clear of distractions, exaggerations, and misperceptions. The perceiving eye sees “insurmountable obstacles” or “major setbacks” or even just “issues.” It brings its own issues to the fight. The former is helpful, the latter is not.

Instead of reacting to whatever the person has said, be curious, try to understand why they said it.

If you are overwhelmed by emotion, take a break, go get a glass of water. Follow up with the offending individual later. This break will help you formulate a more useful response.

Chastise the whole team

Back when I was in primary school, one of our teachers called the entire class into the staff room. This was not a common occurrence. Neither was the big cane she was holding or the space created between the desks.

As it turns out, the class had performed very poorly, it seemed the best performing student had about 10/30. So she decided the best way to remedy it would be to administer the balance of marks to 30 as cane strokes.

This was not a good experience, it did not help later it was discovered she was using the wrong marking scheme.

While as a manager you hopefully don’t resort to physical abuse, verbally abusing the entire team is just as bad.

Why it doesn’t work

As humans, we naturally lean towards our groups for direction. Especially when feeling confused or afraid. When you attack the entire group, you characterize yourself as the enemy.

If you have ever participated in some kind of civil action, you must have experienced how fear of authority vanished under group fever.

Even if the team does not dramatically rise up against you, you will take hits in productivity. For knowledge workers, which all developers are, you need their commitment to your goal.

What to do instead

Take time off to reflect. Was the vision not clear? Was the team not empowered to act on it? Was there a major blocker which I was not aware of?

Call in a retrospective. This is not the time to blame anyone, you need to see what is going on as objectively as possible. Solicit feedback on how you can better define the objectives.

Finally, learn from your mistakes and move on.

Take over and make all the decisions

Winslow Taylor defined scientific management as knowing exactly what you want men to do and then see that they do it in the best and cheapest way.

Unfortunately, software developers and really all knowledge workers don’t take on too well to this paradigm.

If you feel you are dealing with a particularly idiotic individual or even worse team, the temptation may be to simply make all the hard decisions and give direct orders.

Why this doesn’t work

Modern products and companies that make them are complex. So much so that no individual human could ever fully understand the workings of a modern product, say a vehicle.

As Walter Powell put it:

There are a number of jobs that are based, in large measure, on either intellectual capital or craft-based skills, both of which have been honed through years of education, training and experience. Many of these kinds of knowledge-intensive activities, such as cultural production, scientific research, design work, mathematical analysis, computer programming or software development, and some professional services, require little in the way of costly peripheral resources. They are based on knowhow and detailed knowledge of the abilities of others who possess similar or complementary skills. Knowhow typically involves a kind of tacit knowledge that is difficult to codify

No matter how smart you are, the people working on the product will always have more details than you. This is not even a bad thing. It means your team can scale.

What to do instead

Embrace the power of the team, you see if everyone is working towards the same goal, they will make the continuous improvement on various aspects of the product making it better in ways you did not anticipate.

What you need then is come up with some kind of process with incentives which nudge the team towards good decisions.

How do you handle being kicked? Talk to me in the comment section below, my Linked in chenchajacob  or my twitter @jchex


Are you still agile?


Some of the most basic principles in productivity are ignored on the basis of it being too stiff. For example, the advise you should put everything you want to do on the calendar is usually met with some form of trepidation, after all, who wants to live their life in a straight jacket!

Yet, there is a certain freeing dimension to it, you now can be sure what you want to do actually has some scheduled time to it. This effect however only kicks in if you also realize you own your calendar, you can switch things around if you wish, subject of course to other work constraints.

Agile practices offer you a similar kind of freedom. In fact it does you one better, you can even change the structure of some of its most prominent variant, scrum, to whatever best fits your organization!

But then, when does agile stop becoming agile?

In this entry, I will be exploring some of the core characteristics of agile which I believe if you are not doing, you need to call your practice something other than agile.

Is there a timebox?

Scrum depends on timeboxes.

I believe a timebox is so important, it’s not even just a scrum or even agile principle, it is a life principle!

A timebox does to your life what a budget does to your finances. Without a budget, your money will likely disappear to the first expenses which happen to occur. A budget helps you be mindful of other important expenses which may not be as immediate but are as important.

In a previous entry, I discuss why and how to timebox.

If you ever find your team working endlessly with nothing to show for it, you are probably violating these principles.

Are features working?

Life happens, sometimes you are just not able to deliver on everything you said you would.

I used to work with a developer whose goto statement was:

This is just about done

It seemed all features were 90% done and nothing was ever 100% done. Thankfully, he has since moved past this stage and now provides fully functioning features useful to our end users.

If you find yourself in a similar state and still continue as if nothing has happened, then you are living in some form of denial.

As humans, we are good at recognizing when something is done or not done. The various shades of incompleteness bring nothing but confusion to the team.

It is tempting to mark a task/feature as complete if it does look nearly complete, but as our product manager likes to remind me:

If you tick of this feature, you are being dishonest with yourself, it needs to first reviewed, integrated and documented

Is the backlog prioritized by business value?

After being with Twiga for a couple of months, I was getting quite confident in my knowledge of the product.

In fact, I was quite sure I knew more about the product than say the agents its meant to serve who only use it as opposed to us who build it.

Such hubris never takes you to a good place.

I was relieved of this disillusion from a field trip where I observed one of the agents use a trick to quickly add a new delivery, a functionality not officially provided.

At this moment, I came to realize the end users of a product have access to a certain view of the product not available to myself or anyone else in the development team. Just because you wrote it does not mean you have hands-on experience using it.

This incident taught me not only an important lesson in humility but also, your backlog should not reflect your priority, it should be an expression of what your end users care the most about.

Are there estimates?

A common pushback when asking for an estimate from the developer team is we have no idea how long this will take.

I don’t think this is ever true.

Let’s take an example of my trip to work this morning. Will it take exactly one hour from my gate to the office?

Probably not.

Does saying it will not take one hour of any use? Does that mean it will take 2? Maybe 12 hours? Of course, I know from experience this is also not true.

In fact, I can say with confidence, over the last 30 days, it has taken me between 30 mins to 1.5 hours to get to the office. With this information, I can guesstimate with 90% probability if I leave the house at 7 am, I will arrive at between 7:30 and 8:30 am.

The numbers above may not be pinpoint accurate but they sure as hell do communicate a lot more than I will not reach at 8 am.

Furthermore, if I now actively start tracking my estimate vs actual values and reflecting on possible causes of variance. I generally get better at my estimation.

The questions above are not meant to be comprehensive, but I have come to find by constantly asking them. We are able to evaluate if we are still on the right path in our process.

How do you know you are still agile? Talk to me in the comment section below or on my twitter @jchex