What is a software quality?

If any of you have heard me speak in a training session or conference you’ll know I am found of quoting Philip Crosby: “Quality is free!”. Crosby was talking from a background in missile production but the message was picked up by the car industry and silicon chip industry (“The Anderson Bombshell” in 1980 explained how Japanese RAM manufacturers were cheaper than Americans and better quality). I am quite fond of applying the argument to software.

I like to cite Capers Jones: “projects with low defect potentials and high defect remove efficiency also have the shortest schedules, lowest costs, and best customer satisfaction levels.” (Capers Jones, Applied Software Measurement, 2008)

He’s not alone in this, Tom Gilb says: “The reduction in defects … saves ‘rework’, which otherwise is about half of all effort in software projects.” (Tom Gilb, Competitive Engineering, 2005).

Jon Jagger pulled me up on some sloppy discussion of software quality, defects and rework in Xanpan. So I’m trying to come up with a (concise) definition of what I mean by software quality and it turns out that this more difficult than you might think.

To my disappointment Jones doesn’t give a definition.

Gilb offers “Defect: a failure to observe a formal, written, required rule. It is not a personal opinion or personal taste. It is a failure to observe a group norm, or required best practice.”

That sounds good until you take the statement to pieces:

  • What if rules are informal? Well I suppose we can allow informal, tacit, rules, because they are “group norms”.
  • “written” assumes there is something written which isn’t an assumption I can accept. Jones points out that documentation is the second most expensive activity after fixing defects, so I’d hate to eliminate defects at the expense of increase writing costs.
  • “personal opinion or taste” seems fair enough but putting this into practice can be incredibly difficult. I know plenty of times when I would call a defect a personal taste but the person raising the issue wouldn’t
  • “group norm” is particularly difficult when you are developing products which will change group norms
  • And “best practice” …. who says it is best practice? who says it can’t be bettered?

I like Gilb’s definition but I don’t think is enough. Crucially even in saying “not a personal opinion” it does nothing to avoid the “one man’s bug is another man’s feature” problem.

What can we say about software quality and defects?

  • Software quality is inversely proportion to the number of defects in the system: high quality implies few defects and vice versa
  • Defects have undesirable consequences
  • Defects incur costs, in all likelihood financial costs but there are others, time in particular. Even if defects are not fixed they will incur costs, e.g. over payments from a financial system or people ringing the helpline to report a spelling mistake
  • Removing defects requires rework and rework costs time and money

This is probably the start of a longer list, what I am describing are the attributes – or qualities – I attribute to “high quality software”.

The list is also self fulfilling: everything I have said so far implies that low quality, lots of defects, will increase costs, so the quotes from Crosby, Jones and Gilb all become self fulfilling. Perhaps this isn’t a problem, perhaps the quality attributes we want from our software is that costs are kept down.

But there is another quality I would like from high quality software which is insidious. High quality software should be changeable – actually all software is changeable (its soft!) but some code is easier to change than other code.

High quality software as easy to change

Lets leave to one side a definition of easy, I agree it should be quantified but not right now.

What do I mean when I say “change” ? I think the spirit is captured by an old John Vlissidees quote I’ve long been fond of:

“A hallmark – if not the hallmark – of good object oriented design is that you can modify and extend a system by adding code rather than hacking it…. In short, change is additive, not invasive. Additive change is potentially easier, more localized, less error-prone, and ultimately more maintainable than invasive change.” John Vlissides, The C++ Report, February 1998

I’m prepared to generalise this to all software, not just OO software. I might even go as far as focusing on the “rather than hacking it” – although one then needs to define “hacking”. Good software needs to allow for change rather than having change forced into it.

Actually this quote also provides the attributes we need to define “Easy to change”

  • Change is localized
  • Change is less error-prone – perhaps better stated as “change does not inject defect” (Somewhere in Jones writing he suggests 7% of defect fixes inject new defects so high quality software would have a bad fix injection rate less than 7%)
  • Change is more maintainable, i.e. changing software does not detract from the changeability of the software

If have these attributes (qualities if you prefer) then software quality is high and as a result change is cheap(er). The relationship between quality and costs appears again.

But the way I describe the quality-cost link is the reverse of the way many people perceive it: the stereotypical Project Manager views quality as an attribute that can be reduced in order to accelerate development and reduce costs. I have to say I have difficulty in actually understanding this point of view but perhaps its because of the way I am defining quality.

Apart from that there is another danger of approaching: Over engineering.

Given all we have said so far you could make an argument for spending a lot of time designing your software to exhibit all these attributes. You could seek to build, design, software which would not require the design itself to be revisited. That after all is rework isn’t it?

Now I’ve long believed there is rework and there is rework:

  • Rework to fix bugs, defects, is bad and wasteful because you shouldn’t have put the bug in there in the first place.
  • Rework to change software for new requirements, even if that means reworking (refactoring) the design is good, or at least acceptable, because you couldn’t know about this up front therefore any effort to cater for this requirement might be misplaced and could actually end up complicating the design. In other words it is self defeating.

As I see it there is a question of knowledge here: you need to engineer within your knowledge, if you know, or could easily find out, some piece of information which would cause you to work differently then you should. But if there is information you don’t know, and would be time consuming/costly, or even impossible, to find out then it is acceptable to defer knowing and accept rework will be required later.

So the question starts to become one of knowledge acquisition. One way of acquiring more knowledge is through feedback, when feedback is rapid, timely and cheap to get we can rapidly expand the knowledge we are working with.

High quality software should be as free as possible from deficiencies given the current knowledge of what is required, but open to change when new knowledge becomes available which necessitates a change.

It’s tempting to write this as:

Quality(T) = Changeability(T) / Known defects(T)

Where T represents some point in time, as time progresses onwards changeability may well decrease which known defects may go up or down.

Notice I’ve said: “Known defects” not “Known changes”. For a piece of living, successful, software there will be a list of changes people would like made to the software. The existence of this list actually demonstrates another attribute of quality software: people use it and value changes. (Low quality software on the other hand may be so buggy that people avoid using it and thus don’t request changes.)

Excluding non-defect changes like this does leave open the problem of whether a defect report (a bug) is actually a defect report or a request for change. In some organisations such debates are heated but usually they are pointless. Sometimes they are really a Cap Ex v. Op Ex discussion, sometimes they are a “Who will do it?” or a “When will it be done?” discussion, sometimes they are a “Who will pay?” discussion. All these, and more, problems get in the way of this measurement.

While I would like to throw the door open and say: “Its all work to be done, one backlog” to do so would be to blow the equation and argument out of the water because, as I just said, high quality software may well have a longer list of change requests than low quality software.

So now we have to consider the argument about internal v. external quality…. but this blog entry is all ready too long.

Does any of this make sense?

Does any of this help?

Am I any closer to defining software quality?

Perhaps but I don’t think I’ve answered my own question yet!

Writing this entry has helped me. I think I’ve found a possible definition of quality, although I still need a definition of defect. I think we need to consider the attributes of both quality and defects. I think there is a temporal issue here related to knowledge (but I don’t know how to model or define that.) I’m even more confused then ever about the relationship between cost and quality because it appear circular.

Anyone got any better ideas? – or just comments?

Scaling Agile Heuristics – SAFe or not!

If I’m being honest, I feel as if I don’t know much about scaling agile. But when I think about it I think the issue is really: What do you mean when you say “scaling agile?”


It seems to me you might mean one of three things:

  • How do you make agile work with a big team? Not just 7 people but 27 people, or 270 people
  • How do you make agile work with multiple teams within an organisation? i.e. if you have one team working agile how do you make another 2, or 20 or another 200?
  • How do you make a large organisation work agile? – it’s not enough to have a team, even a large team working agile if the governance and budgeting systems them work within are decidedly old-school

When I rephrase the question like that I think I do have some experience and something to say about each of these. Maybe I’ll elaborate.

But first an aside: why have I been thinking about this question?

Well David Anderson pushed out a blog about the Scales Agile Framework (SAFe) which prompted Schalk Cronje to ask what I thought – and at first I didn’t have anything to say! But then Schalk pointed out that Ken Schwaber has blogged about the SAFe too. It’s not often that David and Ken find themselves on the same side of the argument… well, actually… they probably do but too many people are willing to see Kanban and Scrum as sworn rivals. I digress, back to SAFe and scaling.

I still don’t have anything original to say about SAFe, I simply don’t know much about it. However the points David and Ken make would worry me too.

I’m not about to rush out and read SAFe. I find I’m more likely to be told by my clients: “I can see how agile works in big companies, but we are a small company, we can’t devote people like that.” And while I do have, and have had, some larger clients I think that statement says a lot.

I have over the years built up some rules-of-thumb, heuristics if you prefer a fancy term, for “scaling agile”. I’ve never set them down so completely before so here you go:

  1. Inside every large work effort there is a small one struggling to get out: find it, work with it
  2. Make agile work in the small before you attempt to make it work at scale; if you can’t get a team of 5 to work then you won’t find it any easier to get a team of 10 or 100 working. Get a small team working, learn from it, and grow it…
  3. Use piecemeal growth wherever possible: start with something small and grow it
  4. Don’t expect multiple teams to work the same way – one size does not fit all! A new project team might be perfectly suited to Scrum, a maintenance team to Kanban and a BAU+Project team to Xanpan. Forcing one process, one method, one approach one different teams is a sure way to fail.
  5. Manage teams as atomic units, aim for team stability, minimise moves between teams
  6. Split big teams into multiple small, independent, teams with their own dedicated work streams, product focuses and even code bases        
  7. Trust in the teams, delegate as far down as you can; give teams as much independence as possible; staff teams with all the skills they need – vertical teams, include testers, requirements people and anyone else
  8. Minimise dependencies between teams; decouple deliveries, decouple teams and again, vertical teams, staff the team so they don’t need to depend on other teams
  9. Use technical practices to automate as much of the dependencies between teams as possible. e.g. a good TDD suit and frequent CI will by themselves allow two related teams to work much closer together
  10. Overstaff in some roles to reduce dependencies – overstaffing will pay back in terms of reduced dependency management work        
  11. Learn to see Supply and Demand (a future blog entry is in the works on this): supply is limited and is hard to increase, you need to work on the demand side too
  12. Recognise Conway’s Law: work with it or set out to use it; again piecemeal growth and start as small as possible will help
  13. Use Portfolio Management to assess teams, measure them by value added against cost. Put in place a governance structure that expects failure and use portfolio management to fail fast, fail cheap, fail often
  14. Ultimately embrace Beyond Budgeting and change your financial practices
  15. Big projects, big teams are high risk and likely to fail whatever you do; some things are too big to try. Some big projects just shouldn’t be done

    
There is no method, framework, tool out there that will be your silver bullet, but if you think for yourself, and if your people are allowed to think and act then you might just be able to create something for yourself.

Doing back to the three questions I opened with:

  • How do you make agile work with a big team? When the team gets too big split it along product lines or product subsystems; if you can’t then don’t do anything that big
  • How do you make agile work with multiple teams within an organisation? Use multiple independent teams, minimise dependencies, decouple and use technical practices
  • How do you make a large organisation work agile? Portfolio management and beyond budgeting

And remember: Don’t do big projects, do small ones and grow success. If that is not an option for you then brace.

Coach or Consultant? Agile or not? What am I?

I am: a Software Development Consultant who specialises in Agile techniques

Or maybe: I am an Agile Consultant who specialise in Software Development.

There was a fascinating thread on Twitter this morning started by Marcin Floryan when he asked: “What are your views on a difference between coaching and mentoring?”. Tweets were coming thick and fast from John McFadyen, Rachel Davies, myself with George Dinwiddie and Andy Longshaw bring up the rear.

For me the difference between Coaching and Mentoring is the different between non-directive coaching and directive coaching. i.e. a true coach is non-directive. Using this definition an awful lot of what passes for Coaching – both in software teams and sports – is actually closer to Mentoring.

Of course things aren’t always that clear and as John pointed out the Coaching industry doesn’t agree on these definitions itself. And sometimes mentoring takes on coaching dimensions. And in fairness sports coaches might not recognise that distinction and it might be that what some people call “Agile coaching” is actually based on sport coaching rather than business coaching.

Still it got me thinking about why I increasingly shun the title “Agile Coach” and bill myself as an Agile Consultant. Hence the statement above. I used to call myself a coach but in the last year I’ve slowly backed away from that term.

I say consultant not coach because while I have studied coaching a little and read excellent books by Whitman and Downey on coaching I’m conscious of what don’t know about coaching. I’m conscious about how much time and effort real coaches put into becoming coaches. And indeed being an opinionated sod I don’t think I can practice true non-directive coaching.

Thats not to say I don’t use coaching techniques, I do but thats not all I do. I give advice, I’m directive – particularly when I first engage with software teams.

Anyway, I know lots and lots of software development – both technical and management – and I can’t help feeling that my clients don’t get the value of my knowledge if they employ me as a non-directive coach. Besides, very few clients I meet want me to be non-directive: they want my knowledge.

So perhaps I am: A Software Development Consultant who specialises in Agile techniques and uses some coaching techniques. (And is prepared to go off-piste and work with non-software teams if you ask nicely.)

But then the word Consultant is pretty nebulous and – in my opinion – abused a lot. The dictionary on my Mac defines Consultant as “a person who provides expert advice professionally.” Thats a definition I can associate with. The thesaurus gives synonyms as “adviser, guide, counsellor; expert, specialist, authority, pundit; informal ace, whizz, wizard, hotshot.” I’m happy with them too.

And to complicate things the work Agile is even more difficult to pin down. The name “Agile” now has a lot of problems. I openly use the term when I’m marketing myself but as soon as I’m inside a company I often find myself distancing myself from “Agile”. Defining just what is agile – and more importantly what is not agile – is increasingly difficult. Hidden away on my website is an unfinished piece “What is Agile?” that says more.

Actually I’d like to have nothing to do with Agile, what we refer to as Agile is really just the best way we know to develop software. Its not about Agile its about Software Development. But try Googling for that, introduce the work Agile and it narrows the field considerably.

Then there is Lean and Kanban. I am – because I paid for it – a Kanban Coaching Professional. You have to respect the way Kanban University has sidestepped the issue of what is coaching by not designating people as a coach but rather a coaching profession.

But is Kanban Agile? A few Kanban people shun the word Agile and the Agile community. Does that mean I should say: “Agile and Kanban” or “Agile and Lean”. (Actually I think the “Kanban is not Agile” tendency has declined recently, it was a growth phase Kanban went through and I think most people accept that while Kanban is not Scrum it falls under the broad Agile umbrella.)

Which would make me: A Software Development Consultant who specialises in Agile and Kanban techniques and uses some coaching techniques.

A bit of a mouthful.

So what am I?

I know a few things about software development. Some people agree with my views and a few people even pay me money to coach/consult/mentor/train/advise them on the subject.

I think many of the things I know about processes in the context of software development can be extended to related domains in technology and elsewhere. But what is the limit? I don’t know.

Xanpan – now available

A bit over two years ago I started using the term Xanpan to describe the style of agile I advocate and help teams implement. If it isn’t obvious Xanpan – pronounced “Zan-pan” – is a cross between Kent Beck’s Extreme Programming (XP) and David Anderson’s Kanban.

At first I used the term Xanpan to myself, then I started using it in public, then people started telling me they wanted to know more. While I’ve mentioned Xanpan in this blog a few times I’ve never really described it as a whole. That has now changed.

A few days ago I made “Xanpan – reflections on Agile and Software Development” available on LeanPub. This is a short e-book which described Xanpan. In the best traditions of LeanPub publishing the book is going to evolve and change.

Right now I’m reading it end-to-end and fixing a few small things. After this I’d like to write a section on requirements/need/product ownership and another on management. Plus – and this is one of the advantages of LeanPub – I want to get feedback on what I’ve written.

A few people have already seen drafts and given me some feedback but I’m hoping to get more. I’m also hoping for a special type of feedback which is very meaningful: money.

On LeanPub the book is available for a small sum of money. If people are prepared to pay then I know its an endeavour worth continuing.

In addition, I’ve added a Xanpan page to my own website which provides some other links about Xanpan – mainly pieces in this blog.

Please buy Xanpan today! That will give me immediate feedback in dollars, then send me your comments – feedback tomorrow.