Notes on a Kanban software development experience

I’ve mentioned the Kanban software development method in this blog before. For those who don’t know its “the new kid on the block” in Agile circles – although the originator (David Anderson) would be quick to point out it is designed to be a Lean development method.

Last year I did some consulting with a large online travel agency. I was involved with helping five teams “get Agile.” This presented an opportunity to try out Kanban in the wild. What I found was: it works, and I feel it is a better models of my own approach to software development than other methods.

What made this engagement more interesting was that I was able to start one team using the Blue-White-Red Agile (BWR) development method I’ve described before and two teams using Kanban so making for an interesting comparison.

The BWR team had been tasked with a project for which a large functional specification had been written. The spec wasn’t perfect but the organization expected them to complete it. BWR allowed the team to break the spec down and attempt it piecemeal. Some upfront estimates were made and a burn-down chart created.

(It almost looked Scrum like, that is, if you ignored the Project Manager, Development Manager, Team Leader, Architect and second Project Manager who joined the project later – and lets not forget me as Agile Coach. Its hard for a team of five developers (and a Product Mmanager) to be self-organizing when there is an equal number of self-organizers.)

This worked well. In the first few weeks I took a lot of heat because I refused to let a completion date for the work be given. Simply we did not have enough date to go on. Once the data started to come through I could do that.

The two Kanban teams were in a different position. Although they had “projects” to do most of their work was sustaining work. There were systems in place and issues needed fixing. Many of the things the company called “projects” were really to small to be worthy of the name – but that’s another story.

As a result the teams had more diversity in their work and more variability. Work would just appear – and disappear. They would have been hard pressed to keep to an unchanged task list (backlog) for a one week iteration, let alone a four week sprint.

This is the first good point about Kanban: its easier to handle changing work loads.

The teams still did “iterations” but the planning meetings quickly changed. Some work would be planned out but this could be usurped at any time. The iteration became the basic unit of comparison, allowing two periods to be compared. This meant that the start/end of iterations meetings were rather more of a review and retrospective session than planning session.

In future I think I would nominally keep iterations for this reason. They are useful time periods to review work and provide a regular review/retrospective point.

Second good point about Kanban: its easier to pick up.

On the whole I found the two Kanban teams needed less training, instruction and coaching than the BWR team (some of whom had been on Scrum training courses.) However, there was some work needed to map out the correct process flow for the Kanban teams to have on their boards.

For anyone creating a Kanban board some advice: big boards a better, involve the team and use coloured tape – insulation tape works well. In the past I’ve just drawn lines on the white board, with Kanban there is more movement and the lines are lost so I used insulation tape to create hard barriers.

A Kanban board

I expected the work in progress (WIP) limits in Kanban to pose a problem. I was expecting managers and others to want to break the WIP limit. This didn’t happen but I found developers breaking the WIP limits.

Developers broke the limits not because they disagreed but because they were accustomed to starting new work when they hit a problem. For example, a developer is working on item X, they come to a block on X so start on Y. In the extreme they block on Y and start on Z. You now have three partially done pieces of work. Trying to instil a “resolve the block” approach was difficult – largely because this organization was overloaded with managers and developers had been “taught” not to resolve these type of issues themselves.

In this organization lots of things blocked. Blocked on other teams. Blocked on IT support. Blocked on administration, etc. etc. I didn’t like it but I added a “blocked” column to the Kanban board. I also added a tally sheet to identify the biggest reoccurring blocks. I then tried to have the managers feel responsible for resolving the blocks. This was only partially successful because managers were themselves blocked because there were so many other “managers” who wanted their time. (The organization used talk as a substitute for action.)

All the teams held daily stand-up (Scrum) meetings and I encouraged the team manager, product manager and BAs to have their own pre-meeting to ensure the prioritisation queue was full and correct. This worked well.

I had the teams tracking cards by making a note on the back of the date as it progressed across the board. The date when it entered the priority queue, the date when work started and stopped, the date when it went for sign-off or test and the date when it was “done.” This data (when complete) was fascinating and started to reveal issues and opportunities, and give an idea of flow. In fact there was so much date I was overwhelmed.

In conclusion I find myself remembering Karl Scotland’s comment from a few months back. Karl had a quote which was something like: “The emphasis in Scrum is on being Agile and improvement follows; the emphasis in Kanban is on improving and being Agile follows.”

In the past I’ve shied away from labelling a team as doing “Scrum” or “XP” because I was aware they were not going it completely and we weren’t seeing all the benefits. I don’t feel that about Kanban, I’m happy to call these teams Kanban. Yes they had problems but we were resolving them. The teams were on a journey.

As anyone who has read Changing Software Development will know, I’m an advocate for incremental improvement. While Scrum (and XP, etc.) hold out the promise of overnight Agile – one big bang and your there – I see Agile as a journey not a destination.

Project plans (part 2 of 3): Turn the question around

As my last post was a moan I promised to give some advice on how to go about project planning. There is a lot to this and I can’t possibly give all the details in a short blog so I’ve split this into three blog posts. Still there is a lot more I could say.

Turn the question around
Instead of a confrontational question and defensive answer have a rich discussion.

When someone asks “when will it be ready?” turn the question around – (1) When is “it” needed by?

When does your business need a product to use? What are the important dates for the company – end of quarter? end of year? trade show? competitors? contract signing?

I always prefer running a project were there are hard business driven milestones to make. Dates which mean something.

Next (2) What is needed on these dates?

Unfortunately too many conversations in the software business seem to involve someone saying “I need everything and I need it yesterday” – not a very useful position, and if its true you should start looking for a new job because there is no way anyone can make that happen.

Look at each of those date and define what is needed. What are the qualities of the thing that is needed? What is the minimum marketable product or feature for that date?

At this point you know: what is needed, how long you have, and who you have to do it.

Yes, I slipped that last one in because its true: you have the people you have and in the short run your resources are constrained because of Brook’s Law and because hiring people takes time.

So within these parameters the engineering team can now come up with options. How can this need be met within this time with these people? This activity separates the real engineers from the rest. Engineers create solutions within these parameters.

Wikipedia (today) says: “Engineers are concerned with developing economical and safe solutions to practical problems, by applying mathematics and scientific knowledge while considering technical constraints. As such, the work of engineers is the link between perceived needs of society and commercial applications.”

And if you cant? Then go back to the “customer” and discuss it. Find a smaller feature, extend the time (shoot for a different trade show), find an alternative way of delivering.

(Tom Gilb has some good examples of engineering solutions into tight constraints and his impact estimation tables can come in useful here.)

Vision
All work needs to be guided by a bigger vision. You can call it a vision or a goal if you wish I don’t mind. Personally I like the term BHAG – Big Hairy Audacious Goal. You’ve got to know where you are going to give context to everything else.

There are two ways to run a project. The first – exemplified by Gantt charts – is to get all the pieces of string you think need to do the project, lie them end-to-end and see where you get to. If you read my previous post you’ll see why I don’t like that approach.

The second way, I’ll call it the BHAG way, is what I discuss above. Decide where you want to be, when you want to be there, what is will look like and then find a way of getting there. How you get there is called engineering. It is what engineers do.

My favourite example of the BHAG way? The Digital OpenVMS team committed to a delivery schedule and said “We don’t know how to achieve this, but we commit to finding a way.

Project plans (part 1 of 3): Why I don’t like Gantt charts
Project plans (part 2 of 3): Turn the question around
Project plans (part 3 of 3) – Planning cycles
Run Scope Creep backwards

Christmas reading: Classic essays on software development

 

There are many, many books about software development. The technical ones alone (e.g. Java, C++, Apache, .NET, etc. etc.) take up meters and meters of shelf space. The ones about project management take up a bit less space but they are still plentiful, and the ones about people in the process take up even less space but are longer lasting.

It is the later category that interests me most theses days. Some books seem perennially relevant and may never go out of print, books like:

Mythical Man Month by Fred Brooks
Psychology of Computer Programming by Gerry Weinberg
Peopleware by Tom DeMarco and Timothy Lister

But often you don’t need a full book to make a point. On these occasions an essay or journal article will do. Often they make the point better simply because they are shorter. Unfortunately these tend to be overlooked too often – books get all the publicity. I would like to highlight half a dozen essays which I think are perennially relevant and often overlooked. In some cases this is because the articles can be hard to get hold of or, more likely, people don’t know they exist.

1. How do committee invent? by Melvin Conway – the origin Conway’s Law. Although written in 1968 this piece is truer than ever. Arguments made by Conway explain much of what actually happens in software development. I’ve examined and looked at this subject myself elsewhere.

Conway suggests that organization that create systems (not specifically software systems) will create copies of their organizations. So, if you have one developer writing the entire system it will be all entangled, few interfaces and difficult for anyone else to work with. And if you have half a dozen developers who don’t communicate you will get six different coding styles, and six versions of most common functions.

What Conway didn’t foresee is that many of system now create the reverse force. Organizations are limited by the systems they use, thus their structures become copies of the systems in use.

2. Worse is Better by Richard ‘Dick’ Gabriel – this started as a keynote talk, became an essay and then a series of letters in which Dick argued anonymously with himself.

If you are one of these developers who think the best code will always win the day you need to read this. Dick suggests that sometimes the solution which is technically worse is economically superior. In a similar vein see Brian Foote’s Big Ball of Mud pattern.

3. No Silver Bullets from the author of Mythical Man Month, Fred Brooks. This piece came out in the mid 1980’s. The Wikipedia summary is here, otherwise you will need to by the Anniversary edition of Mythical Man Month to read it.

Brooks argues that software development is hard, and there is no ‘silver bullet’ which will make it easier. No language, no operating system, no methodology or any other tool will fix it. I have been told that Brooks has since said the if there is a Silver Bullet it is Agile Software Development but I can’t find this quoted anywhere so I don’t completely believe it.

4. A Rational Development Process and How to Fake It by Dave Parnas. Originally published by the IEEE this also appear in Software Fundamentals: The Collected Papers of David L. Parnas. You might also find some copies stashed on the Internet. Dave Parnas deserves to be more widely read by software engineers today so it you haven’t read much of his work buy the collected papers, well worth reading.

In this paper Parnas argues that while we may want to use a rational development process, and while it makes sense to do so it is impossible. This is because software development requires intuition and inspiration – or as I would put it learning. And since you can’t schedule these things or guarantee they will happen the chances of following a rational process are negligible.

However, in order to make our work accessible to those who come after us we need to fake the development process so it looks like we followed a rational process.

I like the logic, I agree with much of it but I have one disagreement. Each generation of software engineers comes along and believes the previous generation were rational and get confused by what they find. They slowly learn that they cannot be rational – causing a lot of angst. They also wonder how the previous generation made such a mess. So frustration is built up.

5. Cathedral and the Bazaar – by Eric Raymond: an explanation of Open Source software. And an explanation of why software created in corporations is frequently a mess. I don’t think Open Source is the answer to all our problems, but I think it is an answer to some and is here to stay.

6. Enrollment Management by Peter Conklin: until recently this was one you really have to hunt down, but it is worth it. Originally published in the Digital Technical Journal (Digital as in DEC or Digital Equipment) in 1992 it was republished by the IEEE in July 1996. HP now have back issues of the Digital Journal online so you can read it here.

This piece tells the story of the Alpha AXP programme in the late 1980’s and early 1999’s. In theory, on paper, on GANTT charts the project couldn’t be done. The author tells how enrollment management was used to engage those working on the project. Although written in 1992 this paper is about Agile development. And it is about Agile development in the large – over 2,000 engineers worked on the project.

I wrote about this article before, a few years go. One of the small stories in this paper always brings a smile to my face. The OpenVMS team committed to a delivery schedule and said ‘We don’t know how to achieve this, but we commit to finding a way.’

So there you have it. Six essays I recommend you read. And if you only have time for one? Well, chances are you know about Open Source and the Silver Bullet so I recommend you read Enrolment Management or A Rational Process, and if you push me more I’ll say: Enrolment Management.