Online User Stories tutorials now complete

Better User Stories
As a Product Owner I want to write better stories

I’m pleased to announce I’ve released the last of my online User Stories tutorials (part 5: Workflow and Lifecycle) and with that the whole series is complete. You can now buy the entire User Stories set of 5 tutorials as one package at a 40% discount to buying the tutorials individually.

The package includes over six hours of video commentary, exercises, quizzes, downloads and both ebook and audio book versions of Little Book of Requirements and User Stories.

Blog readers can get a further 25% off the price with the code: “blogoct21” until the end of this month, October 2021.

In addition, the first 3 people to use that code will receive a free print copy of The Art of Agile Product Ownership.

The 5 tutorials are:

These tutorials turned ouit to be a lot more work than I expected (where have i heard that before?). The core material is based on the Requirements, Backlogs and User Stories workshop that I have been running for a few years and last year converted to a series of online webinars. In the process the material has become a lot more focused.

Please, let me know what you think, in the comments section below or in the feedback forms at the end of each tutorial in the series.

7 habits of highly effective software development?

“Most of us think we don’t have enough time to exercise. What a distorted paradigm! We don’t have not to.” Stephen R. Covey

Try reading that quote again and substituting the word “refactor” for “exercise.” Or try substitute the words “test first”, or “technical excellence” for “exercise.”

It was Craig Girvan of Head Forwards at Agile on the Beach this month who pointed out to me that one of the most famous management books of all time actually contains an edict to pursue technical excellence and refactoring.

Read this snippet form The 7 Habits of Highly Effective People and as you do so think about software development, and specifically the technical quality of the code being cut:

“We are instruments of our own performance, and to be effective, we need to recognize the importance of taking time to sharpen the saw”

Whether you think of the skills of the engineers building the system, the system itself, the technology which powers the system or the process that build the code into an executable there is a resonance.

On of the things Covey emphasis in the book is that the be effective one needs not just productive capacity (“PC”, the capability to produce something) but to maintain that ability and enhance that capacity. Hence his advice to: sharpen the saw. That means using some of your PC to create more PC in future, grow the pie if you like. This is a theme he returns too many times in the book.

And that is exactly the thinking we need to put behind our software development teams: it is not just about producing something for today, it is about increasing our capability for tomorrow. Of course there is a balance, one needs to both produce today and enhance for tomorrow, find the sweat spot is hard.

In the race to deliver value today we sometimes loose that. We forget that enhancing our capability creates value because it helps us create more value tomorrow – that capability is itself valuable. In part the problem is because investing in capability enhancements has a longer payback period, the return on those investments will not be seen immediately while directing our efforts to today will deliver returns real soon.

The old “jam today or more jam tomorrow” problem. It is a balance, and getting that balance right is incredibly difficult.

But it is exactly that philosophy that lead Microsoft and Amazon to reinvest all their profits for many years. Rather than pay shareholders dividends they prefer to invest in themselves so their future capacity is greater. And because of that promise their share rise in value and shareholders benefit more than if they had paid dividends.

It’s nearly 20 years since I read 7 Habits but after Craig’s observation I fished out my copy. What struck me was how the 7 Habits themselves could be seen as a software development method in their own right:

  1. Be proactive: teams are experts in delivering useful digital products, they should be finding what is needed and working to deliver it. Simply doing what you are told is not enough. The days of sitting around waiting for a requirements document or specification are over.
  2. Begin with the end in mind: what could be more outcome focused than that? It’s not about the myriad of stories and features you will develop, its the ultimate goals that is important
  3. Put first things first: some will see this as a mandate to design the thing up front, I don’t, I see this as an instruction to start delivering value and testing hypothesis immediately. For Covey it is simply about prioritisation, “organize and execute around priorities” – simply decide what are the priorities today and get on with them.
  4. Think Win/Win: too often in development we frame decisions in confrontational terms, “Users v. developers”, “The business v. coders”, “Programmers v. testers”. One side “wins” at the others expense. We need to stop that, and we need to stop seeing the divisions. (Similar to David Cote’s argument in Winning now, Winning later.)
  5. Seek to understand first, then be understood: what a brilliant way of saying “Listen to customers” and then frame technical discussions in terms the audience will understand.
  6. Synergize: this overlaps with #4 in my mind. Covey says it is about recognising that the whole is greater than the sum of its parts. What better description of a software system could you want? All those little parts, functions, classes, modules, all working together to produce a useful whole. Yet this is one of the most difficult problems in software engineering, approaching it from either the parts or the whole creates problems. Instead we need to build something small that works, some parts that work together, and then grow it, organically.
  7. Sharpen the saw: work to have more capability tomorrow than you have today, which is where we came in

So maybe 7 Habits is a development method in disguise, or maybe its a way of thinking that should inform our approach. In fact, as I said, I read the book nearly 20 years ago, I suspect much of this has seeped into my general thinking and, without me knowing it, informed my approach.

The book may have become a cliche itself but I would still recommend reading 7 Habits.

Saw images from Luke Milburn on Wikicommons, Creative Commons License.

Subscribe to my blog newsletter and download Continuous Digital for free – normal price $9.99/£9.95/€9.95

Questioning the great work from home experiment

18 months ago everyone who worked in an office was sent home, and told “work from home.” Suddenly even the most anti-work from home companies and bosses had to accept it. Even the slowest, bureaucratic, IT departments had to support remote working.

In many ways it has been a great experiment – an experiment that is judged a success because people have been more productive than expected. And look… the western economies are still here. Indeed, some businesses have boomed.

But, I don’t think that was the great experiment. To my mind the great experiment starts now – now that people are getting the option to work in the office or work from home, now that big-bad-managers can lean on people to go into the office and start using “presenteeism” again.

There are a few of points about the great work from home which I think are generally missed. It is because these points no longer apply which makes me thing now is the true experiment. I’ve also got a bunch of concerns, I’m not convinced that surviving the last 18 months by working-from-home is the way we should carry on.

First, the great work-from-home was egalitarian: it wasn’t a privilege or a punishment. Nor was it self-selecting: whether you wanted to or not you did it.

Second, everyone had to do it so everything was online. There were no meetings with some people in one room and others hanging on the end of a telephone, the kitchen was no longer a place for side chats and the smokers couldn’t have their own meetings outside the back door.

Third there was no end date: there was no option to say “we’ll decide it when we are all in the office.”

Finally, last but not least: at least initially it was existing people and teams so relationships and social-capital already existed People were used to working together already.

Bringing in new people, “onboarding” and “enculturing” is always hard, its a lot harder online – and being honest, one of the things I find hardest is finding my way into a new team when everything is online.

As work-from-home has gone on teams have had to learn to recruit and incorporate new people online.

Now, as people drift back to the office – at different paces in different places – these points don’t hold. Now being in the office or not is often a choice, it is no longer whole teams so those side conversations are back.

To my mind this is an even bigger experiment than the great work-from-home; and I think it is going to prove more difficult for companies and teams to navigate.

Personally, I’m really lucky, I’ve had my office in my home for years so I’m all set up for it. But I’m sick of working in the same place day-after-day for months on end. While I’m sure many people never want to see an office again I think there will be many others who are keen to go back to the office.

Next, I have a few fears about the extended remote working model. First off is mental health: without an office, without the journey to and from the office, without the change of clothes that all involves there is less separation between work and home. Without dividing lines that means work impinges on home time and it is difficult to escape work stress.

Second, online working is far more dependent on the written form. That means that those of us who struggle with the written form are at a disadvantage.

Yes I know I’m a bad example, I write too much. But look at all my grammatical and spelling foibles – I’m dyslexic. I’ve learned to be good at writing but really I’d rather be talking.

As e-mail replaces the telephone, Slack replaces the talking across the desk, WhatsApp replaces coffee conversations those of us who struggle with writing are disadvantaged.

Next, I’m concerned for younger people, those who only entered the work force recently. How are they to learn about their job? How are they to learn work culture, let along specific company culture if there is no office?

And most those same people are often in a more difficult position. They are stilling with their parents in their childhood bedroom, or they are living in a house share with problematic internet. In short, younger people need the office more and they need older people as role models and teachers.

Finally, I feel a lot of technology people – programmers specifically – are very keen to push back on any suggestion that face-to-face communication, and co-location, has any advantages. It seems very acceptable to say “We can do everything online, there is no difference.”

I beg to differ. If only because without body language, without facial expressions, without seeing how people react then communication is less – the old “80% of communication is non-verbal” idea. You can do a lot online, but you can also do a lot in person.

Unfortunately I don’t see this debate. I don’t see people discussing “what is best done online” and “what is best done in person.” I see my fellow digital workers being very quick to push back on anyone that questions online working.

Perhaps it’s me, but I feel the technology community is so anti-office work that I have hesitated even to voice these concerns. Anyone else share these views? Or am I persona non-grata?

Subscribe to my blog and download Continuous Digital for free – normal price $9.99/£9.95/€9.95

Splitting & slicing user stories

I’m please to announce the fourth part of my User Stories tutorial is now available.

User Stories by example, part 4: Splitting stories

In this tutorial I look at 10 days to split a story and illustrate each with examples and exercises.

I have one more part of this tutorial series to deliver, Workflow and Lifecycle, hopefully I’ll have that out in the next month.

Until then please try the tutorial and let me know what you think.

The Sprint Goal?

Hi Allan, what do you think of this as a Spring Goal?
  • Prototype store locator
  • Deploy product selector to live
  • Fix accessibility defects identified by client
  • Complete visual design of search feature
  • Security fixes & updates
  • Team improvement: refactor VX tables, page template processor”

I answer:

“This looks more like a sprint backlog than a sprint goal.”

This e-mail exchange sums up the problem with the sprint goal, or rather, the sprint goal as it so often ends up being used.

The sprint goal has always been part of Scrum even if it has often been forgotten. The idea behind it was to say: “What is the outcome this team needs to make happen this sprint?” The goal was meant to be a non-trivial thing, a meaningful step forward, an outcome, perhaps a challenge, certainly a rallying point.

However the sprint goal fell into disuse. When I used to run teams I never used it – partly because my teams have never used strict Scrum but also because most of the teams I worked with had multiple things happening. The teams were expected to make progress across a broad front. Conversely the sprint goal focuses the team on a single thing.

My experience was far from unique. And, if I’m being honest, in the days when I gave agile training regularly I never talked about it much. Again, most of the teams I encountered were expected to “deliver stuff” it was more a case of “burning down the backlog.”

When I did see the sprint goal used it was normally used in reverse. Rather than teams setting a goal and asking “What do we need to do to make this happen?” teams would decide on a collection of stories from the backlog and then ask “What is the goal we can write that describes this collection of items?” In such cases the goal might as well be “Do stuff” or perhaps “Do the collection of stories we think we can do.”

The goal was meaningless so why bother?

Yet I detect a change in the air. In the last few years I’ve heard the sprint goal talked about more and I’ve observed teams setting a goal more often. Plus, as I wrote in Succeeding with OKRs in Agile, a sprint goal sits well with OKRs – it also provides a way to cut through the tyranny of the backlog.

Unfortunately I have to report the teams I see setting sprint goals are still setting goals about “Do these stories from the backlog.”

Why is this?

Perhaps it is because the sprint goal is misunderstood or perhaps it is because people are aiming to tick off as many Scrum practices as they can, maybe they feel they must use the goal because Scrum lists it.

I’m sure both of these reasons are at play but I think the main reason is because of backlog fetish and the expectation that teams “do the backlog.” Teams – and especially product owners – don’t have the skills or aren’t being given the authority to make decisions about what to do based on fresh information arriving from customers, analytics and analysis.

That is: most teams are still expected to burn-down the backlog.

Well, it is one way of working, I understand the logic, and burning-down a backlog with Scrum is probably still better than ticking off use cases from a requirements document in a waterfall; but it still leaves so much opportunity unrealised. Things could be so much better if teams really worked to sprint goals and OKRs rather than labouring under the tyranny of the backlog.

So if you want some practical advice: if you are setting sprint goals in reverse just give up, accept that you “do backlog items” and save yourself the time of inventing a goal.

And if you are not setting a sprint goal: have a serious talk about it as a team, examine what having a sprint goal would mean and how you might work differently. Then experiment with using a sprint goal for a few sprints.

This advice goes doubly if you are a Product Owner, seriously using sprint goals is going to relieve you of a lot of backlog administration but means you will need to think hard about goals and what will really improve your product.

Subscribe to my blog newsletter and download Continuous Digital for free

(normal price $9.99/£9.95/€9.95)

Unplanned work after the sprint starts?

“Should unplanned work be allowed after the sprint starts?”

One of those questions which comes up again and again. And it came up last week when I visited a clients offices – yes I actually visited a client! The answer to this question is, as often happens: It depends. So let me give you my thinking.

First, many teams have a rule that work must be scheduled in the sprint planning meeting, after which this is fixed. Teams have a right to make this rule so if this is a team rule – what Kanban folk call a policy – then work is not allowed in.

This rule is based on a strict interpretation of Scrum. The thinking – particularly in early implemenations of Scrum – was that changing priorities was a big problem for teams and therefore fixing the work to be done for a few weeks made sense. In the event of that things did change the team would declare an “abnormal termination of sprint” and move to start a new sprint with new priorities.

Now for some teams this makes complete sense. Barring work from entering the sprint after planning makes complete sense. Equally it makes sense for team members to only do work scheduled in the sprint and refuse all other work. So, it depends… when a team is troubled by new work appearing, priorities changing, and when a team are expected to deliver something new – when their overarching priority is not support but building something new – then this approach makes complete sense.

But, don’t follow this rule just because you think Scrum says so. I just had a quick look at the latest Scrum Guide doesn’t actually mention abnormal termination of sprint. It does say “No changes are made that would endanger the Sprint Goal” which then leads us into a conversation about the sprint goal but let’s hold that for now.

Now ring fencing the team and the sprint like this solves one set of problems but it creates another set. If the team are aiming to be reactive why would they not pick up work?

And as teams increasingly become DevOps or SecDevOps, or BizDev, or whatever, things get more complicated. It would be irresponsible to hold a “no work enters the sprint” if the live server was down or a security hole had been found. But at the same time, being hyper-reactive has a downside because the team would be constant distracted.

Ultimately it is the Product Owner who should have the final say on whether work is unplanned work is accepted or not but when you have a customer on the phone someone else may be forced into a decision.

I apply two tests: is the unplanned work really urgent? – or could it wait a few days and be considered in the next sprint. (Or even queued in the backlog for longer.)

Second, is the unplanned work valuable? – namely, is it more valuable than the work the team are doing and would be displaced by this work. Ideally it would valuable enough to justify the disruption it causes by late entry too.

Hence I like to talk about urgent but valuable unplanned work. Just because something appears after sprint planning doesn’t mean it is not valuable. If the work is urgent, and if it is valuable enough, then it deserves to enter the sprint and get done.

However I like to build in two feedback loops. First, as the work arises, does the person raising the work understand the disruption this will cause? Are they prepared to accept that some other work may not get done?

I like to make this real: pull up your board and show the requester the consequences. Let them prioritise the work against the current planned work. This can make the unplanned work go away.

Second, mark the late-breaking work so you can track it through the system – on a physical board I would use a yellow card. At the end of the sprint review how many yellow cards you have and talk about whether the right decision was made.

Over time, as you build up your data – and stock of done yellow cards – you can reason about the cards and decide your long term action. For example,

You might want to make an allowance in sprint planning for unplanned work: suppose your team averages 3 yellow cards a sprint, then, when you are planning the sprint allow space for them.

If you have many yellow cards regularly you might even want to move to a Kanban model or split the team.

Review the requests, what are the common themes? – is there a module which is particularly troublesome, would some remedial work help reduce the unplanned work.

Or is there someone in particular who raises unplanned work? Should the team leaders talk to this person and see if they could change their behaviour, perhaps they could make their requests a few days earlier.

Maybe you want to ring-fence a team member to deal with unplanned work while the rest of the team pushes on with the main work.

As I said at the top of this post, the unplanned work question comes up a lot. I discussed it in Xanpan so if you want more examples go there. And if you have any other suggestions please comment on this post.

Subscribe to my blog newsletter and download Continuous Digital for free

Analyse your Jira data? (for free)

Photo by Tyler Easton on Unsplash

Send me your data!

Think of this as a free offer, let me look at your data and I’ll tel you if I find anything interesting.

When I work with clients I often download the Jira data and crunch the data in Excel to see if I can find any patterns or any information in the mass of tickets and dates. I know there are tools out there which will do this but I’m never quite sure what these tools are telling me so I like to do it myself. Also its a bit of a “fishing trip” – I don’t know what I might find. Having done this a few time I’ve developed a bit of a pattern myself – nothing i can describe yet but who knows.

So, if you would like me to crunch your data please send it over. I say Jira but I’m happy to work with data from any other systems – I’ll learn something new

You will need to export all the issues as a CSV or Excel file. And I suggest you anonymise the data, just delete the columns with names and even delete the card description. The more you can send me the better, but the columns that interest me most have to do with dates (created and closed), ticket types (story, bug, task/sub-task, etc.), status and, if they are recorded, estimates and actual times.

I won’t share the data with anyone else – I’ll even delete it when I am finished if you wish. I would like to document some of my findings in a blog post but I can give you first sight if you like.

Apart from find patterns and perhaps learning something what interests me is what I might be able to tell about a team I know nothing about. It is an experiment. I’m allan AT – or use the contact page.

Why on-ramps and off-ramps are more important than highways

It begins with a simple request: “we need to know when it will be done.” Or, when there is an agile-savvy manager, “our velocity needs to be higher.” But the more I look the more it appears the dev team aren’t really that bad, in fact they might actually be good. And, if you doubled team productivity overnight it wouldn’t make a big difference. The problem is elsewhere.

Sure the dev team could be better in many ways but simply coding faster isn’t going to solve the problem. The on-ramp and the off-ramp are in need of improvement: the work intake and the work delivery mechanism – entry ramp (getting stuff in processed) and exit ramp (getting it out the door) are often more imporant.

As they say: its déjà vu all over again. I see this again and again. In my mind’s eye turning requests into working software is a freeway, a motorway, an autobahn – a controlled-access highway to use a technical term. Each piece of work is a car.

Most of our attention goes on the cars/work speeding down the lanes, that is where we assume time is spent. That is where most of the team are working, that is where we direct people to look for problems. If all goes well the work/car moves rapidly from one place to another. Sure things go wrong on that journey, in coding, sometimes other pieces of work get in the way, sometimes something goes wrong and there is a pile up with work/cars queuing behind. And sometimes the best way of improving the overall flow is to limit work in progress or reduce speed limits.

But, the actual speeding down the highway part is but one of three essential elements. Frequently this is not where time is lost, and even though work can be unpredictable it is not the most unpredictable part of the work.

It is fairly common for work to spend most of its life waiting to enter the system – the on-ramp, how cars get on the highway and how work enters the development processes.

And there is the off-ramp – how does work leave the system and reach the customer? – after cars only join the highway when they are coming from one place and want to get to another.

Most people working in the system see their job as driving cars and ensuring that a particular payload is delivered to the destination. Who looks at the overall system? Who manages the highway? Who optimises the flow? This is where I see my work. It is not enough to ensure a piece of work is delivered, it is not enough to ensure the cars are going fast, one has to see the whole system. Usually the on-ramp and off-ramp require far more work than the actual highway itself.

In other words: it is not about ensuring any one car arrives once. It is about ensuring the system for delivering cars works effectively. While the highway journey gets the attention the on-ramp and off-ramp are often far more important.

Consider the off-ramp: it is very common to find that development teams are working pretty well, but when work is “done coding” it queues to get through testing, queues to get into a release and queues to be released. In fact, it is almost normal in teams that work spends longer “getting out the door” than it spends being done.

The continuous delivery movement has done a lot to improve this and the best teams have streamlined and automated this part but problems remain. I’ll just mention two.

One: I just said “the best teams.” The best teams are few and far between. Yes they get lots of attention but most teams are a long way from this. It is not uncommon to find that teams consider some continuous delivery processes madness. I floated the idea of branchless development to a team this year and they took it as a sign that I didn’t understand their work. The idea that you might not use source control branches appeared like a naive beginners mistake.

Two: where do you put testing? If testing is considered a special activity that must happen as part of a release process then it occurs on the off-ramp. That off-ramp has limited capacity and any problems have big knock on effects – it is very risky.

However, testing can be considered part of the main highway experience. Developers can work to a high standard an incorporate practices like TDD and BDD which lesson the need for testing. Formal testing – probably by professional testers- can be positioned before the off-ramp if you design the highway/workflow correctly.

Now consider the on-ramp: the intake process, the requirements process, the work-before coding, work that is normally done by Product Owners/Product Managers or Business Analysts. This can cause even bigger hold-ups than the off-ramp.

I’ve written before about the fear many organizations have of actually coding. As a result work is held in perpetual review, estimation and planning before it is allowed anywhere near a coder.

Another cause of delay is the product backlog: in many places this is a bottomless pit of work to do. Every few weeks the Product Owner shifts through the backlog selecting a few pieces of work to get done. Most work doesn’t get done and falls to the bottom. It is unlikely to be done but gets in the way and distorts metrics. As a result most work spends most of its life cycle waiting to be done, waiting to get onto the highway.

There is a natural (and good) tendency to focus on the work in hand, to think “if I can only get this piece of work done…” Like Orwell’s Boxer pledging “I will work harder” to any problem. (There are plenty of none team members prepared to stand on the sidelines saying “If only they did work harder.”)

It is not enough for any one person to work harder. It is a system: the an on-ramp, a highway and and off-ramp all need to work together. Only looking at the whole do these things become clear. Improving this flow requires a different set of skills to those of writing code and testing – of course there is overlap in skills and of course people can learn; but again, if one simply pledges to “work harder” and write better code the improvement will be marginal.

Seeing the highway – the work flow – is something I would expect a development manager to do, and if not a development manager than the person I call and Agile Guide and most of the rest of the industry calls an Agile Coach.

Subscribe to my blog newsletter and download Continuous Digital for free – normal price $9.99/£9.95/€9.95

Have Google made a $billion skateboard mistake?

How do you design a car? – It is one of the most famous diagrams in the agile world drawn by Henrik Kniberg.

I’m guessing many readers know this already: one approach (top of the diagram) is to iteratively design and build all the pieces, put them together and you have a car. This is one interpretation of iterative but until you put the pieces together there is no feedback and no value.

After all, who wants half a car? We know what we want from a car, right? Who needs feedback? Who wants a car with three wheels? – Why waste time experimenting?

The alternative, advocated by minimally viable product people everywhere is to redefine the problem; we don’t want a car so much as transportation, we could start with a very simple – and quick to deliver – transportation system (the skateboard). Because it is delivered sooner we get feedback sooner. We see how people use it, and evolve it over a series of iterations into a motorised car.

Since I know this, you know this and it is on the back of every agile cereal packet one can rest assured that Larry Page, Tim Cook and Jeff Bezos know this, right?

Well no, not according to the Financial Times recently. The FT carried a piece about the development of autonomous cars – “Robotaxis: have Google and Amazon backed the wrong technology?” – paywalled. Since we already have the sort taxi someone drives the development effort goes into advancement.

For the last few years Google/Waymo, Apple and others have sunk billions of dollar – yes billions – into developing self-driving cars. And of course, we all know what we want from a car, even a self-driving car so this is an engineering problem.

These cars now work but there are a number of challenges before the achieve world dominance. Most of the challenges now are less to do with the technology and more to do with the market: customer acceptance, insurance, pricing, etc. Still, billions more are needed before any return can be achieved. In other words: Google etc. took the first approach, build the pieces.

What is less well know, and what the FT writes about, is that another group of companies has take the other approach. Component suppliers like Bosch and Magna, and tech companies like Mobileye (Intel) have been developing discrate technologies that can be incorporated into existing cars which evolve towards self-driving dream. Not only is this cheaper but it is easier to market and clear regulatory hurdles because humans are assisted in driving not replaced. (Tesla is also in this camp as they add more and more capabilities to their auto-pilot features and have been getting feedback from real customers for years.)

Now it seems evolutionary approach may win-out against the big clean sheet of paper. The race is not over yet but the evolutionary suppliers are making money while the new designer are still burning cash. The evolutionary suppliers are integrating their tech into cars and getting regulatory approach while the new designers have to navigate many regulators.

Engineers often object to the evolutionary model because: “we need to see the big picture”, “you need an architecture”, “you can’t evolve a skateboard into a car”. And indeed, one of the Google engineers, quoted in the FT saying:

“Conventional wisdom would say that we’ll just take driver assistance systems and we’ll kind of push them and … over time, they’ll turn into self-driving cars … well that’s like saying that if I work really hard at jumping, one day I’ll be able to fly.”

Chris Urmson, 2015

When you consider the problem purely in engineering terms this rings true, but while one needs to respect engineering it is not the only frame of reference. One need to consider the commercial and marketing aspects, as well as customer acceptance and other factors. To give any single line of thought a privileged position is to expose yourself to risks from the others.

At the end of the day, as I have argued repeatedly: engineering is about creating solutions within a context, within constraints. To any given problem there are many potential solutions – many ways to slice the onion. The “best” solution is the solution which best fits those constraints.

The evolutionary approach allows you get feedback sooner which allows you to uncover those constrains sooner, that allows for course corrections and it also means less time and money has been spent on solutions which don’t meet the constraints.

Subscribe to my blog newsletter and download Continuous Digital for free – normal price $9.99/£9.95/€9.95

Documentation, another case of rapidly diminishing returns

No body writes documentation in startups.

Writing documents is a luxury only established companies can afford. One might ask: are companies successful because they write documents or do they write documents because they are successful? But I’d be hard pushed to find anyone to argue the latter.

Back in 1996 I worked on the ill-fated UK rail privatisation. I was part of the team writing a new computer system to create a new rail timetable – one that would allow competition in train services. In the average week I spent half my time coding and half my time documenting.

For every program there was a program specification – what the program had to do. And a functional specification – saying what the program did. I had to update both. And these had to align with the system architecture, which was a multi-volume beast I wasn’t allowed to touch.

Additionally I had to create a unit test plan for each program or set of changes. Unit tests were manual and the test plan was made up of two Word documents. One was a big table and the other had one test per page. As I conducted the test I had to update the plan with success or fail – and fail of course mean rework so we alwys made sure it worked before hand.

When my program was ready to check-in I had to fill in a source code control form so my team leader knew which files to check in. If I added a new code file to the system I needed to complete an additional form to explain what the file was. On one occasion I got fed up of a 2,000 line C++ file and refactored it as 4 smaller files. Being C++ his meant 4 pairs of .h and .cpp files, so 8 new files each with a form. My team leader was quite clear: I was never to do this again, it made too much work for her.

Did it help? – I find it hard to imagine it did. In fact I started my own hidden documentation, the “Rough Guide to …” which told me (and other devs) things we actually needed to know.

There are two types of companies: those that don’t write documents – and where many many people are asking for documentation. Documentation is seen as a solution to (almost) any problem. And yet people don’t write documents, then they feel guilty about not writing them.

The second type of company is overrun with documentation. There seem to be armies of people writing documentation: architects and business analysts seem particularly keen to write documents. These are often prose, they are about as readable as your average Shakespeare play is to a 15 year old and read about as often. Project managers are also prone to documentation but they don’t write prose; instead, project plans and progress presentations are documents in another medium.

In the first type of company nobody reads documents because there are none. In the second type of company its debatable how many of those documents are read. Many of them are so utterly boring that it is hard to stay awake reading them. As I’m often heard to say:

The bigger a document is, the less likely it is to be read.

And if it is read, the bigger a document is the less the reader will remember.

In one eye-and-out-the-other.

It’s probably just as well because documentation rapidly becomes out of date unless copious amounts of time are invested in keeping it up to date. Actually, it is good that documentation is seldom read because reading it is almost as expensive as writing it and, in theory at least, it should be read far more often.

True, “documentation” covers a very wide range of things. From project plans to release notes, from user guides to architecture diagrams. Some has more readers more than others – user guides compared to functional specifications. But then, who has read their iPhone manual? Does such a document even exist? Arguably the best products don’t need documentation.

In both types of companies I hear complaints about the lack of communication – actually, I don’t think I’ve ever visited a client were people didn’t complain about the lack of communication. But only in the first type of company do people think that documentation will cure the problem.

In such companies documentation is seen as the solution to almost every problem. Programmers complain they haven’t been given written requirements and specifications, they complain the user designers aren’t giving them documents of is expected, and most of all they complain the developers who came before them did not document what they did. Equally testers demand the same requirements and specification but also want programmers to provide written descriptions of what the program does. Project managers want written reports of what was done and so on.

Nobody ever got fired for asking for more documentation, but I’m not so sure about writing it.

While I have sympathy that these people want information I don’t believe documentation will solve the problems – perhaps because I’ve never seen a development effort with “Goldilocks documentation” – not too much and not too little. One thing I do know is that if everyone wrote the documentation they thought was needed, and what others wanted from them, then little would get done.

What I fear is a descent into documentation as everyone sets about communicating through documentation and not talking.

Because documentation takes time and money to create, it takes time and money to read, it takes time and money to update and keep current. And all the time and money spent on documentation is time and money not being spent on developing products and testing products in the market.

Worst still documentation becomes a hinderance to change. On multiple occasions I have met companies that do not want to change their products or processes because the cost of updating the documentation is too great.

I’ve nothing against documentation itself, I simply lament the time and money that could be better spent elsewhere, I regret the missed opportunities for real communication and belief that something has been communicated; and I fear the limitations that documentation brings once in place.

To answer my own question above. I don’t believe companies that write documents are successful, documentation does not determine success – if it did many many projects would have succeeded instead of failing. That documentation is so often lacking but products are successful actually goes to prove that is not essential.

Nor do I really believe companies write documents because they are successful. They write documents because they are successful enough to be able to afford to write documents but those same documents inhibit future success.

Before I close, I can almost hear people rushing to their keyboards to tell me of occasions were a system document saved their life,

“If Sam hadn’t left behind a document that told me the function was connected to the reactor core…”

While I’m sure such cases exists I’m not convinced the they justify the vast amounts of cost of writing a document against doing something else. If Sam hadn’t written that document what might Sam have done with the time instead? Possibly something even more valuable.

As with planning documentation exhibits rapidly diminishing returns on investment.

Photo by Glenn Carstens-Peters on Unsplash

Subscribe to my blog newsletter and download Continuous Digital for free – normal price $9.99 / £9.95 / €9.95