Splitting & slicing user stories

I’m please to announce the fourth part of my User Stories tutorial is now available.

User Stories by example, part 4: Splitting stories

In this tutorial I look at 10 days to split a story and illustrate each with examples and exercises.

I have one more part of this tutorial series to deliver, Workflow and Lifecycle, hopefully I’ll have that out in the next month.

Until then please try the tutorial and let me know what you think.

The Sprint Goal?

Hi Allan, what do you think of this as a Spring Goal?
  • Prototype store locator
  • Deploy product selector to live
  • Fix accessibility defects identified by client
  • Complete visual design of search feature
  • Security fixes & updates
  • Team improvement: refactor VX tables, page template processor”

I answer:

“This looks more like a sprint backlog than a sprint goal.”

This e-mail exchange sums up the problem with the sprint goal, or rather, the sprint goal as it so often ends up being used.

The sprint goal has always been part of Scrum even if it has often been forgotten. The idea behind it was to say: “What is the outcome this team needs to make happen this sprint?” The goal was meant to be a non-trivial thing, a meaningful step forward, an outcome, perhaps a challenge, certainly a rallying point.

However the sprint goal fell into disuse. When I used to run teams I never used it – partly because my teams have never used strict Scrum but also because most of the teams I worked with had multiple things happening. The teams were expected to make progress across a broad front. Conversely the sprint goal focuses the team on a single thing.

My experience was far from unique. And, if I’m being honest, in the days when I gave agile training regularly I never talked about it much. Again, most of the teams I encountered were expected to “deliver stuff” it was more a case of “burning down the backlog.”

When I did see the sprint goal used it was normally used in reverse. Rather than teams setting a goal and asking “What do we need to do to make this happen?” teams would decide on a collection of stories from the backlog and then ask “What is the goal we can write that describes this collection of items?” In such cases the goal might as well be “Do stuff” or perhaps “Do the collection of stories we think we can do.”

The goal was meaningless so why bother?

Yet I detect a change in the air. In the last few years I’ve heard the sprint goal talked about more and I’ve observed teams setting a goal more often. Plus, as I wrote in Succeeding with OKRs in Agile, a sprint goal sits well with OKRs – it also provides a way to cut through the tyranny of the backlog.

Unfortunately I have to report the teams I see setting sprint goals are still setting goals about “Do these stories from the backlog.”

Why is this?

Perhaps it is because the sprint goal is misunderstood or perhaps it is because people are aiming to tick off as many Scrum practices as they can, maybe they feel they must use the goal because Scrum lists it.

I’m sure both of these reasons are at play but I think the main reason is because of backlog fetish and the expectation that teams “do the backlog.” Teams – and especially product owners – don’t have the skills or aren’t being given the authority to make decisions about what to do based on fresh information arriving from customers, analytics and analysis.

That is: most teams are still expected to burn-down the backlog.

Well, it is one way of working, I understand the logic, and burning-down a backlog with Scrum is probably still better than ticking off use cases from a requirements document in a waterfall; but it still leaves so much opportunity unrealised. Things could be so much better if teams really worked to sprint goals and OKRs rather than labouring under the tyranny of the backlog.

So if you want some practical advice: if you are setting sprint goals in reverse just give up, accept that you “do backlog items” and save yourself the time of inventing a goal.

And if you are not setting a sprint goal: have a serious talk about it as a team, examine what having a sprint goal would mean and how you might work differently. Then experiment with using a sprint goal for a few sprints.

This advice goes doubly if you are a Product Owner, seriously using sprint goals is going to relieve you of a lot of backlog administration but means you will need to think hard about goals and what will really improve your product.


Subscribe to my blog newsletter and download Continuous Digital for free

(normal price $9.99/£9.95/€9.95)

Unplanned work after the sprint starts?

“Should unplanned work be allowed after the sprint starts?”

One of those questions which comes up again and again. And it came up last week when I visited a clients offices – yes I actually visited a client! The answer to this question is, as often happens: It depends. So let me give you my thinking.

First, many teams have a rule that work must be scheduled in the sprint planning meeting, after which this is fixed. Teams have a right to make this rule so if this is a team rule – what Kanban folk call a policy – then work is not allowed in.

This rule is based on a strict interpretation of Scrum. The thinking – particularly in early implemenations of Scrum – was that changing priorities was a big problem for teams and therefore fixing the work to be done for a few weeks made sense. In the event of that things did change the team would declare an “abnormal termination of sprint” and move to start a new sprint with new priorities.

Now for some teams this makes complete sense. Barring work from entering the sprint after planning makes complete sense. Equally it makes sense for team members to only do work scheduled in the sprint and refuse all other work. So, it depends… when a team is troubled by new work appearing, priorities changing, and when a team are expected to deliver something new – when their overarching priority is not support but building something new – then this approach makes complete sense.

But, don’t follow this rule just because you think Scrum says so. I just had a quick look at the latest Scrum Guide doesn’t actually mention abnormal termination of sprint. It does say “No changes are made that would endanger the Sprint Goal” which then leads us into a conversation about the sprint goal but let’s hold that for now.

Now ring fencing the team and the sprint like this solves one set of problems but it creates another set. If the team are aiming to be reactive why would they not pick up work?

And as teams increasingly become DevOps or SecDevOps, or BizDev, or whatever, things get more complicated. It would be irresponsible to hold a “no work enters the sprint” if the live server was down or a security hole had been found. But at the same time, being hyper-reactive has a downside because the team would be constant distracted.

Ultimately it is the Product Owner who should have the final say on whether work is unplanned work is accepted or not but when you have a customer on the phone someone else may be forced into a decision.

I apply two tests: is the unplanned work really urgent? – or could it wait a few days and be considered in the next sprint. (Or even queued in the backlog for longer.)

Second, is the unplanned work valuable? – namely, is it more valuable than the work the team are doing and would be displaced by this work. Ideally it would valuable enough to justify the disruption it causes by late entry too.

Hence I like to talk about urgent but valuable unplanned work. Just because something appears after sprint planning doesn’t mean it is not valuable. If the work is urgent, and if it is valuable enough, then it deserves to enter the sprint and get done.

However I like to build in two feedback loops. First, as the work arises, does the person raising the work understand the disruption this will cause? Are they prepared to accept that some other work may not get done?

I like to make this real: pull up your board and show the requester the consequences. Let them prioritise the work against the current planned work. This can make the unplanned work go away.

Second, mark the late-breaking work so you can track it through the system – on a physical board I would use a yellow card. At the end of the sprint review how many yellow cards you have and talk about whether the right decision was made.

Over time, as you build up your data – and stock of done yellow cards – you can reason about the cards and decide your long term action. For example,

You might want to make an allowance in sprint planning for unplanned work: suppose your team averages 3 yellow cards a sprint, then, when you are planning the sprint allow space for them.

If you have many yellow cards regularly you might even want to move to a Kanban model or split the team.

Review the requests, what are the common themes? – is there a module which is particularly troublesome, would some remedial work help reduce the unplanned work.

Or is there someone in particular who raises unplanned work? Should the team leaders talk to this person and see if they could change their behaviour, perhaps they could make their requests a few days earlier.

Maybe you want to ring-fence a team member to deal with unplanned work while the rest of the team pushes on with the main work.

As I said at the top of this post, the unplanned work question comes up a lot. I discussed it in Xanpan so if you want more examples go there. And if you have any other suggestions please comment on this post.


Subscribe to my blog newsletter and download Continuous Digital for free

Analyse your Jira data? (for free)

Photo by Tyler Easton on Unsplash

Send me your data!

Think of this as a free offer, let me look at your data and I’ll tel you if I find anything interesting.

When I work with clients I often download the Jira data and crunch the data in Excel to see if I can find any patterns or any information in the mass of tickets and dates. I know there are tools out there which will do this but I’m never quite sure what these tools are telling me so I like to do it myself. Also its a bit of a “fishing trip” – I don’t know what I might find. Having done this a few time I’ve developed a bit of a pattern myself – nothing i can describe yet but who knows.

So, if you would like me to crunch your data please send it over. I say Jira but I’m happy to work with data from any other systems – I’ll learn something new

You will need to export all the issues as a CSV or Excel file. And I suggest you anonymise the data, just delete the columns with names and even delete the card description. The more you can send me the better, but the columns that interest me most have to do with dates (created and closed), ticket types (story, bug, task/sub-task, etc.), status and, if they are recorded, estimates and actual times.

I won’t share the data with anyone else – I’ll even delete it when I am finished if you wish. I would like to document some of my findings in a blog post but I can give you first sight if you like.

Apart from find patterns and perhaps learning something what interests me is what I might be able to tell about a team I know nothing about. It is an experiment. I’m allan AT allankelly.net – or use the contact page.

Why on-ramps and off-ramps are more important than highways

It begins with a simple request: “we need to know when it will be done.” Or, when there is an agile-savvy manager, “our velocity needs to be higher.” But the more I look the more it appears the dev team aren’t really that bad, in fact they might actually be good. And, if you doubled team productivity overnight it wouldn’t make a big difference. The problem is elsewhere.

Sure the dev team could be better in many ways but simply coding faster isn’t going to solve the problem. The on-ramp and the off-ramp are in need of improvement: the work intake and the work delivery mechanism – entry ramp (getting stuff in processed) and exit ramp (getting it out the door) are often more imporant.

As they say: its déjà vu all over again. I see this again and again. In my mind’s eye turning requests into working software is a freeway, a motorway, an autobahn – a controlled-access highway to use a technical term. Each piece of work is a car.

Most of our attention goes on the cars/work speeding down the lanes, that is where we assume time is spent. That is where most of the team are working, that is where we direct people to look for problems. If all goes well the work/car moves rapidly from one place to another. Sure things go wrong on that journey, in coding, sometimes other pieces of work get in the way, sometimes something goes wrong and there is a pile up with work/cars queuing behind. And sometimes the best way of improving the overall flow is to limit work in progress or reduce speed limits.

But, the actual speeding down the highway part is but one of three essential elements. Frequently this is not where time is lost, and even though work can be unpredictable it is not the most unpredictable part of the work.

It is fairly common for work to spend most of its life waiting to enter the system – the on-ramp, how cars get on the highway and how work enters the development processes.

And there is the off-ramp – how does work leave the system and reach the customer? – after cars only join the highway when they are coming from one place and want to get to another.

Most people working in the system see their job as driving cars and ensuring that a particular payload is delivered to the destination. Who looks at the overall system? Who manages the highway? Who optimises the flow? This is where I see my work. It is not enough to ensure a piece of work is delivered, it is not enough to ensure the cars are going fast, one has to see the whole system. Usually the on-ramp and off-ramp require far more work than the actual highway itself.

In other words: it is not about ensuring any one car arrives once. It is about ensuring the system for delivering cars works effectively. While the highway journey gets the attention the on-ramp and off-ramp are often far more important.

Consider the off-ramp: it is very common to find that development teams are working pretty well, but when work is “done coding” it queues to get through testing, queues to get into a release and queues to be released. In fact, it is almost normal in teams that work spends longer “getting out the door” than it spends being done.

The continuous delivery movement has done a lot to improve this and the best teams have streamlined and automated this part but problems remain. I’ll just mention two.

One: I just said “the best teams.” The best teams are few and far between. Yes they get lots of attention but most teams are a long way from this. It is not uncommon to find that teams consider some continuous delivery processes madness. I floated the idea of branchless development to a team this year and they took it as a sign that I didn’t understand their work. The idea that you might not use source control branches appeared like a naive beginners mistake.

Two: where do you put testing? If testing is considered a special activity that must happen as part of a release process then it occurs on the off-ramp. That off-ramp has limited capacity and any problems have big knock on effects – it is very risky.

However, testing can be considered part of the main highway experience. Developers can work to a high standard an incorporate practices like TDD and BDD which lesson the need for testing. Formal testing – probably by professional testers- can be positioned before the off-ramp if you design the highway/workflow correctly.

Now consider the on-ramp: the intake process, the requirements process, the work-before coding, work that is normally done by Product Owners/Product Managers or Business Analysts. This can cause even bigger hold-ups than the off-ramp.

I’ve written before about the fear many organizations have of actually coding. As a result work is held in perpetual review, estimation and planning before it is allowed anywhere near a coder.

Another cause of delay is the product backlog: in many places this is a bottomless pit of work to do. Every few weeks the Product Owner shifts through the backlog selecting a few pieces of work to get done. Most work doesn’t get done and falls to the bottom. It is unlikely to be done but gets in the way and distorts metrics. As a result most work spends most of its life cycle waiting to be done, waiting to get onto the highway.

There is a natural (and good) tendency to focus on the work in hand, to think “if I can only get this piece of work done…” Like Orwell’s Boxer pledging “I will work harder” to any problem. (There are plenty of none team members prepared to stand on the sidelines saying “If only they did work harder.”)

It is not enough for any one person to work harder. It is a system: the an on-ramp, a highway and and off-ramp all need to work together. Only looking at the whole do these things become clear. Improving this flow requires a different set of skills to those of writing code and testing – of course there is overlap in skills and of course people can learn; but again, if one simply pledges to “work harder” and write better code the improvement will be marginal.

Seeing the highway – the work flow – is something I would expect a development manager to do, and if not a development manager than the person I call and Agile Guide and most of the rest of the industry calls an Agile Coach.


Subscribe to my blog newsletter and download Continuous Digital for free – normal price $9.99/£9.95/€9.95

Have Google made a $billion skateboard mistake?

How do you design a car? – It is one of the most famous diagrams in the agile world drawn by Henrik Kniberg.

I’m guessing many readers know this already: one approach (top of the diagram) is to iteratively design and build all the pieces, put them together and you have a car. This is one interpretation of iterative but until you put the pieces together there is no feedback and no value.

After all, who wants half a car? We know what we want from a car, right? Who needs feedback? Who wants a car with three wheels? – Why waste time experimenting?

The alternative, advocated by minimally viable product people everywhere is to redefine the problem; we don’t want a car so much as transportation, we could start with a very simple – and quick to deliver – transportation system (the skateboard). Because it is delivered sooner we get feedback sooner. We see how people use it, and evolve it over a series of iterations into a motorised car.

Since I know this, you know this and it is on the back of every agile cereal packet one can rest assured that Larry Page, Tim Cook and Jeff Bezos know this, right?

Well no, not according to the Financial Times recently. The FT carried a piece about the development of autonomous cars – “Robotaxis: have Google and Amazon backed the wrong technology?” – paywalled. Since we already have the sort taxi someone drives the development effort goes into advancement.

For the last few years Google/Waymo, Apple and others have sunk billions of dollar – yes billions – into developing self-driving cars. And of course, we all know what we want from a car, even a self-driving car so this is an engineering problem.

These cars now work but there are a number of challenges before the achieve world dominance. Most of the challenges now are less to do with the technology and more to do with the market: customer acceptance, insurance, pricing, etc. Still, billions more are needed before any return can be achieved. In other words: Google etc. took the first approach, build the pieces.

What is less well know, and what the FT writes about, is that another group of companies has take the other approach. Component suppliers like Bosch and Magna, and tech companies like Mobileye (Intel) have been developing discrate technologies that can be incorporated into existing cars which evolve towards self-driving dream. Not only is this cheaper but it is easier to market and clear regulatory hurdles because humans are assisted in driving not replaced. (Tesla is also in this camp as they add more and more capabilities to their auto-pilot features and have been getting feedback from real customers for years.)

Now it seems evolutionary approach may win-out against the big clean sheet of paper. The race is not over yet but the evolutionary suppliers are making money while the new designer are still burning cash. The evolutionary suppliers are integrating their tech into cars and getting regulatory approach while the new designers have to navigate many regulators.

Engineers often object to the evolutionary model because: “we need to see the big picture”, “you need an architecture”, “you can’t evolve a skateboard into a car”. And indeed, one of the Google engineers, quoted in the FT saying:

“Conventional wisdom would say that we’ll just take driver assistance systems and we’ll kind of push them and … over time, they’ll turn into self-driving cars … well that’s like saying that if I work really hard at jumping, one day I’ll be able to fly.”

Chris Urmson, 2015

When you consider the problem purely in engineering terms this rings true, but while one needs to respect engineering it is not the only frame of reference. One need to consider the commercial and marketing aspects, as well as customer acceptance and other factors. To give any single line of thought a privileged position is to expose yourself to risks from the others.

At the end of the day, as I have argued repeatedly: engineering is about creating solutions within a context, within constraints. To any given problem there are many potential solutions – many ways to slice the onion. The “best” solution is the solution which best fits those constraints.

The evolutionary approach allows you get feedback sooner which allows you to uncover those constrains sooner, that allows for course corrections and it also means less time and money has been spent on solutions which don’t meet the constraints.


Subscribe to my blog newsletter and download Continuous Digital for free – normal price $9.99/£9.95/€9.95

Documentation, another case of rapidly diminishing returns

No body writes documentation in startups.

Writing documents is a luxury only established companies can afford. One might ask: are companies successful because they write documents or do they write documents because they are successful? But I’d be hard pushed to find anyone to argue the latter.

Back in 1996 I worked on the ill-fated UK rail privatisation. I was part of the team writing a new computer system to create a new rail timetable – one that would allow competition in train services. In the average week I spent half my time coding and half my time documenting.

For every program there was a program specification – what the program had to do. And a functional specification – saying what the program did. I had to update both. And these had to align with the system architecture, which was a multi-volume beast I wasn’t allowed to touch.

Additionally I had to create a unit test plan for each program or set of changes. Unit tests were manual and the test plan was made up of two Word documents. One was a big table and the other had one test per page. As I conducted the test I had to update the plan with success or fail – and fail of course mean rework so we alwys made sure it worked before hand.

When my program was ready to check-in I had to fill in a source code control form so my team leader knew which files to check in. If I added a new code file to the system I needed to complete an additional form to explain what the file was. On one occasion I got fed up of a 2,000 line C++ file and refactored it as 4 smaller files. Being C++ his meant 4 pairs of .h and .cpp files, so 8 new files each with a form. My team leader was quite clear: I was never to do this again, it made too much work for her.

Did it help? – I find it hard to imagine it did. In fact I started my own hidden documentation, the “Rough Guide to …” which told me (and other devs) things we actually needed to know.

There are two types of companies: those that don’t write documents – and where many many people are asking for documentation. Documentation is seen as a solution to (almost) any problem. And yet people don’t write documents, then they feel guilty about not writing them.

The second type of company is overrun with documentation. There seem to be armies of people writing documentation: architects and business analysts seem particularly keen to write documents. These are often prose, they are about as readable as your average Shakespeare play is to a 15 year old and read about as often. Project managers are also prone to documentation but they don’t write prose; instead, project plans and progress presentations are documents in another medium.

In the first type of company nobody reads documents because there are none. In the second type of company its debatable how many of those documents are read. Many of them are so utterly boring that it is hard to stay awake reading them. As I’m often heard to say:

The bigger a document is, the less likely it is to be read.

And if it is read, the bigger a document is the less the reader will remember.

In one eye-and-out-the-other.

It’s probably just as well because documentation rapidly becomes out of date unless copious amounts of time are invested in keeping it up to date. Actually, it is good that documentation is seldom read because reading it is almost as expensive as writing it and, in theory at least, it should be read far more often.

True, “documentation” covers a very wide range of things. From project plans to release notes, from user guides to architecture diagrams. Some has more readers more than others – user guides compared to functional specifications. But then, who has read their iPhone manual? Does such a document even exist? Arguably the best products don’t need documentation.

In both types of companies I hear complaints about the lack of communication – actually, I don’t think I’ve ever visited a client were people didn’t complain about the lack of communication. But only in the first type of company do people think that documentation will cure the problem.

In such companies documentation is seen as the solution to almost every problem. Programmers complain they haven’t been given written requirements and specifications, they complain the user designers aren’t giving them documents of is expected, and most of all they complain the developers who came before them did not document what they did. Equally testers demand the same requirements and specification but also want programmers to provide written descriptions of what the program does. Project managers want written reports of what was done and so on.

Nobody ever got fired for asking for more documentation, but I’m not so sure about writing it.

While I have sympathy that these people want information I don’t believe documentation will solve the problems – perhaps because I’ve never seen a development effort with “Goldilocks documentation” – not too much and not too little. One thing I do know is that if everyone wrote the documentation they thought was needed, and what others wanted from them, then little would get done.

What I fear is a descent into documentation as everyone sets about communicating through documentation and not talking.

Because documentation takes time and money to create, it takes time and money to read, it takes time and money to update and keep current. And all the time and money spent on documentation is time and money not being spent on developing products and testing products in the market.

Worst still documentation becomes a hinderance to change. On multiple occasions I have met companies that do not want to change their products or processes because the cost of updating the documentation is too great.

I’ve nothing against documentation itself, I simply lament the time and money that could be better spent elsewhere, I regret the missed opportunities for real communication and belief that something has been communicated; and I fear the limitations that documentation brings once in place.

To answer my own question above. I don’t believe companies that write documents are successful, documentation does not determine success – if it did many many projects would have succeeded instead of failing. That documentation is so often lacking but products are successful actually goes to prove that is not essential.

Nor do I really believe companies write documents because they are successful. They write documents because they are successful enough to be able to afford to write documents but those same documents inhibit future success.

Before I close, I can almost hear people rushing to their keyboards to tell me of occasions were a system document saved their life,

“If Sam hadn’t left behind a document that told me the function was connected to the reactor core…”

While I’m sure such cases exists I’m not convinced the they justify the vast amounts of cost of writing a document against doing something else. If Sam hadn’t written that document what might Sam have done with the time instead? Possibly something even more valuable.

As with planning documentation exhibits rapidly diminishing returns on investment.

Photo by Glenn Carstens-Peters on Unsplash


Subscribe to my blog newsletter and download Continuous Digital for free – normal price $9.99 / £9.95 / €9.95

User Stories by Example part 3 (Refactoring)

The latest instalment of my online User Stories tutorial is now available online, User Stories by Example: Refactoring.

It takes as its starting point some existing stories and reworks them to convey their message more clearly. In the process I discuss:

The use of time-boxed spikes.

The naming of team members in user stories, e.g. “As a developer I want …” – and why this isn’t a good idea.

Rewriting user stories and breaking them down into more smaller stories. (More on this in the next tutorial.)

Why more smaller stories is better than a fewer larger stories.

How acceptance criteria can be used to split stories into smaller pieces.

and a brief look at dealing with dependencies.

Videos are intersperced with exercises and quizzes. My guess is this tutorial will take two to three hours to complete – which can be all in one go or split over days or weeks to suit yourself. As with the earlier tutorials I work through real life user stories to illustrate and draw lessons.

This is the third tutorial, it joins User Stories by Example part 1 Starting with Stories and part 2 Acceptance Criteria. The next module will look at splitting stories in more detail.

The tutorial this carries the introductory price of $49. In time this price will probably rise and I’ll introduce a combined option to buy all the courses in one go.

Please e-mail me with your comments and suggestions.

Software is not a lump or work to do (and the laws which prove it)

Workers digging hole

Building software is not like digging a hole in the ground: your target changes as you dig, the people and machines you use change the outcome beyond the original idea, and you never really know when to stop digging.

Years ago I was hired by Reuters to build an interface connector between the Liffe trading exchange and their data systems. Another developer started a few weeks after me. In the end we produced about half a dozen modules that worked together to make the connection. But we only produced that many because we were over staffed. In reality the one module I wrote handled 90% of the work required, the other modules were largely superfluous. And certainly the extra time which would have been required to make my module do 100% of the work was less than the time I spent to making sure it worked with the other modules.

Regular readers have probably already recognised Kelly’s Second Law of software: Inside every large development effort there is a small one struggling to get out

This itself is an example of Parkinson’s Law: work expands so as to fill the time available for its completion.

Actually you might restate my second law as Parkinson’s Law in reverse: constraining the capacity to do work reduces the amount of work to do. If you think about it this is a result of Conway’s Law: system designs copy the communication channels in the organization which creates the system.

Economists see a related phenomenon as the Lump of Work fallacy: people assume there is a fixed lump of work to do. If there are more people to do the work (say from immigration) then there will be unemployment – or possibly wages will be forced down and same work is distributed between more people. However at the level of a national economy it doesn’t work like that. More people means more demand for food, houses, healthcare, schools and so on. What is true in the small is not true in the large. The net effect can be positive for countries but exactly how and by how much is hotly debated.

Software development has its own lump of work fallacy: There isn’t a fixed amount of work to do. Rather than saying “How many people will it take and how long will it take?” It is better to say “If we have five people working on this for three months what progress can we make?”

Writing software is not like digging a hole in the ground: the work to do is neither really known in advance nor is it fixed. Adding people actually increases the amount of work to do – Brooks’s Law.

At the start nobody really knows what it required, they may get lucky but more often that not once the thing is put in front of clients and (potential) customers the ask changes. I’ve heard this called Humphrey’s Law (after Watts Humphrey) although that name is not in common usage and there is another Humphrey’s Law from the world of psychology – which is connected with time estimation discussions for different reasons.

When you put Parkinson’s Law together with this “don’t know until I see it” law you get Hofstadter’s law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.

Lets assume for the moment that one can know what is wanted in advance. There is another problem: there are multiple ways to achieve the same result. There is no one true way in software development, there are always multiple ways to achieve the same end result.

Even if one was to fix the big questions – the OS, the delivery platform, the programming language, the database and so on – then you can still create very different implementations for the same thing. Or as I usually put it “there are many ways to dice the onion” – my Crown Jewels post describes how I can get wildly different time and money estimates for what is basically the same piece of work.

Of course this has big consequence for effort estimation – how can anyone estimate effort and costs in these circumstances? Let me go further and suggest that the process of estimating the work to do is more likely than not to increase the amount of work to do. Not only does estimation take time to do itself, but there when estimating there is a tendency to “play safe” and favour larger estimates even thought these estimates are likely to themselves be underestimates (see Vierordt’s law.)

Sometimes it feels like quantum physics: when one parameter is measured another changes, we can know the speed but not the direction, or the direction but not the speed.

I’m not sure I have a complete answer but I have some of the pieces.

Start new work with a Minimally Viable Team: task the team to start immediately, coding starts on day one and in parallel the team work to understand what is needed and create potential solutions.

Keep teams stable: Teams and staffing containing a lot of variables, keeping the team stable (but not static) provides some past performance data to include in calculations. It will at times be necessary for the teams to call in more help – pull more skills and resources as needed.

By working with existing and minimally viable teams the problems are partially constrained: the technologies available are mostly the technologies the team knows already, the number of people available to work on the work is the number of people on the team. In time you may change both these parameters but initially they are constraints to work within.

Lastely, use active portfolio management and governance to kill work which is under performing or escalating beyond expectations. You may want to engage in set-based engineering to increase the chances of success.

The next time someone says “Building software should be like building a house” please remind them you aren’t building a house. What software engineers do is massively more variable and complex than building another example of the same thing.

Back at Reuters, the bright side was that the over engineered system we built wasn’t just used for Liffe, it was used to connect 2 or 3 other exchanges too so maybe the over staffing and over engineering was worth it. Except I don’t believe it really made economic sense, it would have been better to get the one exchange working and only add the minimum to the system when the second and third exchanges came along – diseconomies of scale again.


Subscribe to my blog and download Continuous Digital for free – normal price $9.99/£9.95/€9.95