I'm on Software Engineering Radio

For those who don’t know Software Engineering Radio is a series of pod casts about, well, software engineering. It was founded by Markus Völter and most of the team are other German software engineers. Don’t worry, its all in English!

Markus and the team are very well connected so they get access to most of the best people in the field.

So, its with a little awe that I am now joining in this collected wisdom. Yes, you can here me talking about Software Development as Learning on Software Engineering Radio. If you’ve ever wondered just what I sound like now’s your chance to find out.

The interview was recorded months ago and focuses on the material in my book (Changing Software Development).

SE-Radio is a totally not for profit organization so you might like to make a small donation if you find it useful.

ACCU conference 1 of many (Wednesday)

Another year, another great ACCU conference. I counted 9 languages on the conference program, everything from Scala to C++ – yes there was more C++ that anything else but it was far from the only thing going on. Languages plus sessions on design, pattern, management, Agile and neuroscience.

Normally some of the most insightful comments come not in the scheduled sessions but in the discussions over coffee and in the bar afterwards. Sometimes you don’t know these are insightful until after the event when you’ve left and had time to think it all through.

This will always be a special ACCU conference for me because I was the keynote speaker on Saturday. My keynote was well received and the slides ….

So here is a brain dump.

The opening keynote on Wednesday was from Bob Martin – “Uncle Bob” – about software craftsmanship. (I’m watching the software craftsmanship movement with interest and will write a blog about it in future.) Bob suggested that the most successful Agile method – Scrum – was the accidental “winner” of the Snowbird summit in 2001. This was because the Agile manifesto and Agile values said nothing about technical practices.

Scrum, as is well know, does not contain any technical practices. Teams often borrow these from XP. Because of this Bob believes Scrum is a subset of XP – a subset which just deals with project management. Without the technical practices productivity falls off and teams make a “big mess faster.”

Bob’s answer is a renewed emphasis on technical practices – hence software craftsmanship.

Bob is a great showman and his talk was very entraining, if you get the chance to see him do so. Along the way made several Scrum jokes and was quite critical of Scrum Master Certification.

Someone like Bob can do this, Bob has the reputation and standing to be able to take pot shots from a safe(ish) position. This is good because someone needs act as a foil.

Jutta Eckstein’s session on transparency in Agile was interesting. It reminded me of the session I ran here a few years ago called “Exposing problems / Creating awareness.” The these was similar: “Agile as a problem detector.” The session was largely a discussion between Jutta and the various people in the room.

Keith Braithwaite presented the latest instalment of his “Measuring the effects of TDD” research project. I’ve seen Keith talk about this subject before and over time his presentation is getting more and more compelling. His sample size in increasing and its looking like a more convincing case.

If you don’t know the work have a look at Keith’s blog. To summarise it: you can measure code complexity using a measure called cyclomatic complexity. Keith’s research shows there is a correlation between complexity and automated until tests. Namely: code with a high level of automated unit tests exhibits lower complexity metrics and people comment they are “happier” with the code.

Having discovered the metric Keith is still trying to understand the cause-and-effect relationship and, perhaps, more importantly, uncover how this relationship can be used to improve matters.

Nicolai Josuttis finished the day with a second keynote. You could call it the antidote to software craftsmanship: Crappy code.

Although Nico made it jovially it had a serious point: there is a lot of bad code out there and it is likely to be around for a while to come. The economics of software development and business tend to perpetuate this situation. Therefore, we need to get used to crappy code and find ways of dealing with it.

While I agree with Nico I can’t help feeling it was a little defeatist. If we accept this poor state of affairs do we still have hope of a better tomorrow?

Between them Bob and Nico certainly set a debate raging – Good code, or Crappy code?

Wednesday finished with 40 of us in Chutney’s curry house.

New writing and events: Downturn & Product Management

Its been a busy few weeks with presentations to BCS Bristol, BCS Kingston & Croydon, Hewlett-Packard and a trip to Switzerland to deliver some Scrum training and coaching – not to mention a family holiday.

The couple of weeks is also busy with some public events coming up:
ACCU Conference: I’m delivering a keynote, a regular session and co-ordinating a set of pattern talks with Kevlin Henney. (This starts on Wednesday next week and I believe there are a few places remaining.)
Brighton Agile Forum have invited me to speak on 11 May
• ACCU London is hosting Jeff Sutherland on 21 May

I’ve had some new articles published elsewhere recently:

Agile in the Downturn is in this months Agile Journal
ACCU Overload 90 April 2009 (whole journal download) continues my On Management series with a look at the Product Manager role (article download)
• A New Approach to Software Project Management in CEO Today: this is one of those virtual magazine formats so you have to flick through the pages. And for reasons I don’t entirely understand my company name (Software Strategy) and not my name has been used as the author. Can’t say I’m entirely happy the way this piece has worked out, its an experiment in reaching a different audience.

Actually, if anything I’ve been writing less recently but these three have been in the works for a while.

New writing, new patterns online

Two new pieces on my website for anyone who’s interested.

I’ve kicked off a new series of articles in the pages of ACCU Overload this month entitled On Management. The first of these pieces, Triangle of Constraints is now on my website. Alternatively it is available in the Overload August download.

After post conference editing my patterns from EuroPLoP 2008 are now online. These four patterns continue my series of business strategy design patterns for software companies. They are: Single Product Company, Whole Product, Product Portfolio and Product Roadmap.

I deliberately entitle these patterns “for software companies” although many people are quick to point out these patterns are applicable to many technology companies and to many companies in general. While I recognise this I stick to the “for software companies” formula for two reasons. First I write from experience, I won’t want to step too far outside my experience zone. Second to do proper justice to the wider context I would need to write longer patterns. I wont to keep these patterns short and readable – I aim for two pages and most of them end up at 4 pages!

Actually there is a third reason: a challenge to the reader. I leave some blanks for the reader to fill in, so the reader can make the pattern their own. I drop hints – some of the pictures and examples deliberately come from other industries – but leave it to the reader to bridge the gap.

Make strategy like you make software?

 

There is an interesting piece in the latest issue of the MIT Sloan Review entitled: Should you build strategy like you build software?

I can imagine some managers initial reaction: What? IT is such a total disaster why would we want to make strategy the same way?

After all, we are regularly told that 70% of IT projects fail, and a few months ago the same journal ran a piece damning software development and specifically ERP systems: The Trouble with Enterprise Software

So why would anyone want to copy what IT does?

Well, believe it or not there is something interesting what the author, Keith R. McFarland is arguing is: many of the practices and techniques used in Agile software development can be applied to strategy formation and execution. McFarland focus on techniques such as small iterations, collective ownership, overlapping phases, direction changes (i.e. refactoring), organising around people not tools and abolishing big up front design.

To some degree I think he’s a little behind the curve. The myth about long range / strategic planning in companies was exploded a long time ago. Sure some companies still do it but that doesn’t mean it works. Henry Minztberg’s Rise and Fall of Strategic Planning explains why it just doesn’t work and why its a myth.

It is not only software development were managers and companies have suffered from the Illusion of Control it occurs in strategy formation and planning. Strategy formation is an emergent process, in the same way that software design is emergent.

Big strategic planning has been out of fashion for a while but its far from clear what should replace it. McFarland seems to be suggesting the answer is Agile Strategy.

There is an irony in McFarland’s work – and that of Donald Sull at LBS – is that while businesses thinkers, and some businesses, are looking to Agile Software Development for a paradigm of devising strategy too many businesses are still resisting Agile development or finding it impossible to implement.

Agile development is still largely a bottom-up change exercise. Not enough senior managers are embracing and backing Agile, they cling to the illusion of control. Too many people still say ‘Agile is unproven’. The fact that McFarland and Sull are prepared to embrace these ideas for strategy formation shows that Agile ideas are valid.

Regular readers will know that I have been arguing that there are close parallels between business strategy and software development for some years now. This argument is embedded throughout Changing Software Development. It is most clearly seen in the early chapters were I discuss the knowledge and learning aspects of development work. (The argument is also embedded in my MBA Dissertation “Software Development as Organizational Learning” if you want a less polished – more academic – version.)

And I have written in Overload about it – An Alternative view of design (and planning).

Most directly I’ve blogged about it on and off, specifically last August
Agile software and strategy and Thoughts on strategy and software development.

But this is more than just a parallel: for companies which use a lot of technology software and strategy are increasingly converging. Ultimately your software is your strategy – so much so that I sometimes image software code as liquid strategy.

There are two forces at work here. Firstly, as much IT – including software – becomes commoditised (Nicholas Carrs argument) what remains is strategic. Your Word Processor is a commodity, but your inventory management system is strategic. if you inventory management system is a commodity then your customer management system is strategic, and so on. Because IT is so varied, and software so capable – limited by your imagination – it is used to embed our knowledge.

Which is our second force: what we know. We embed our knowledge in our code so our organisations can operate: whether it is the Galileo booking system, Google’s Adwords or Unilever’s ERP system the capabilities and limitations of our IT systems are also the capabilities and limitations of our organizations.

As Cynthia Rettig argued in her Sloan Review piece, the limitations imposed by an ERP system impose costs on organization and limits on what they can do. That in turn limits on strategy options.

At the same time software systems open up opportunities and create strategy options for companies. We see this most directly with companies like Microsoft and Amazon but it is also true of companies we might not expect. Companies like FedEx depend on software in order to be able to execute strategies like delivering packages on time.

Banks are on the cutting edge of this convergence, unfortunately that doesn’t mean they do it well, but they are converging. Go to the financial centre of London or New York and you’ll probably find more software developers than investment bankers. Trading derivatives, managing risk and countless other banking activities would be impossible today without the sofwtare.

So where does this leave us?

Companies which embrace Agile in software development can learn a new way of working which will help them elsewhere – specifically in strategy formation

Companies which embrace Agile will be better positioned to as software and strategy converge

No company has to embrace Agile, you can do it the old fashioned – see IT as a cost outsource it, run it like a train timetable. But those companies who embrace Agile will win. The more you embrace it the more you win.

 

Heathrow part 2 – a major learning failure

 

I’m really disappointed, for years I’ve been pointing to Terminal 5 construction as a great example of lean ideas at work. It gets delivered on schedule and then what? Mess. Seems everyone will forget it was on time and remember the opening week nightmares.

How come after being an exemplar of Lean techniques during construction Terminal 5 was such a mess at opening?

To recap: BAA decided that T5 was such a big project, and so strategically important, that they couldn’t follow “traditional” construction practice and outsource it to the cheapest bidder. (A quick look at the Wembly Stadium debacle is enlightening.) So, BAA more or less re-invented construction project management along lean principle and delivered their new terminal on time and on budget.

Then BAA handed over T5 to British Airways who messed it up. Weeks of passenger delays, cancelled flights, lost baggage and humiliating publicity which some have called a “National disgrace”.

Most of the problems seem to have been on the BA end but BAA is not in the clear. BAA treated T5 construction as a project, the project (mostly) ended when T5 was given to BA. At that point the bad management, sloppy thinking, and lack of customer respect endemic in the rest of BAA came to T5.

The BAA side of things is an example of how the Project Approach is wrong. BAA should have taken a Product Approach. The product needs to be fully in use, product development continues.

Lesson for software companies: It is a Product, Not a Project.

Now to BA.

From the reports I have read BA has confrontational management. Managers were scared to report problems up the chain so senior managers never heard of problems.

Lesson for software companies: Managers need to stay in contact with what happens on the ground. How actual employee are doing their jobs. Macho management is bad management.

Moving into T5 stretched BA’s resources. Staff from T1 had to move to T5 and find their way around a new terminal with new machines. They had only had limited time to familiarise themselves with the terminal and equipment. In fact, during the learning process they needed more time than usual.

Lesson for software companies: Allow time for learning, don’t skimp; over-staff your departments when big changes are happening. The time of change is not the time to extract savings. That comes later.

Not only is BA confrontational but it has poor employee relations. I suspect the flow of ideas between employees and managers is poor. Employees probably aren’t willing to trust managers and give extra performance. Why should they be?

Lesson for software companies: Trust your employees, help them trust you.

Rightly BA didn’t attempt to move all their flights in one go. However, they did move an awful lot in one wave, and at the same time moved other flights from Gatwick to Heathrow. This wasn’t an incremental delivery.

Lesson for software companies: Incremental delivery means many small deliveries. Not two big ones.

It seems we have a major learning failure at T5. BAA construction teams learned completely new ways of working to deliver on time. But this learning failed to cross the boundary to BAA operations and BA. How often have we heard that before?

 

Lean Product Development references

After yesterdays blog posting I had a comment from ‘Brave Heart’ asking for some references on Lean Product Development. I’m more than happy to help and normally I’d reply direct to Brave Heart. But… I can’t find your e-mail address on your posting or blog. So that means everyone is going to see my recommendations…

The only book I’ve read which is dedicated to Lean Product Development is Michael Kennedy’s Product Development for the Lean Enterprise. There is some additional information in Machine that Changes the World (little bit dated now), Lean Solutions and Thinking Beyond Lean (also getting a bit old) but not a lot. I believe there are a couple of other books dedicated to the Toyota Product Development System but I haven’t read them.

I think you would benefit from looking at some of the literature from the software community. In particular Mary and Tom Poppendieck’s books Lean Software Development and Implementing Lean Software Development.

From there you have to decide if you want to look at Agile software development. I think of Agile as an application of Lean, which I talk about in my own book, Changing Software Development. If you do then SCRUM is going to be of more interest than XP or some of the others.

This is because SCRUM has its roots in the knowledge management – specifically the theories of Nonaka – and these link up with the knowledge thinking behind Lean Product Development. Which brings us full circle back to Kennedy. As Kennedy points out, Lean Product Development might be better described as Knowledge Based Product Development.

As for research, you might want to go and watch some Agile or Lean teams in action. Its probably easier to find teams which consider themselves Agile than Lean. You’ll want to look at how their activities differ from ‘traditional’ product development.

I hope that helps.

Brave Heart, please drop me e-mail (with your address) if I can be of any more help.

Developers are not the only fruit

 

My younger self would be horrified to hear me say this but, when you develop software you need people who are not software engineers.

Don’t get me wrong, coders/engineers/programmers are the most important people because these are the people who actually produce the thing. As such they are central to any development effort, you can’t write software with a collection of managers, analysts and testers any more than you can build a ship without shipwrights.

Coding is where the metal is bent, shaped and welded. When coding there is no room for ambiguity or fudge, the code doesn’t lie and it doesn’t take to ambiguity. Thus there is a lot of details in coding which aren’t obvious before you start. So your developers need to be able to call on Product Managers for more detail and, if you have any architects, you need them at the code face coding with your developers.

See Jim Coplien and Neil Harrison’s patterns Work Flows Inward and Architect Also Implements for more discussion of this topic.

One of the worst projects I ever worked on was completely top heavy with managers (programme, project, customer, delivery, team – you name it we had a manager for it), testers, analysis, admin staff – yes you needed a lot of admin staff when the project was over 120 people. But only about 12 actual coders. Crazy. The overhead was there because… well, the ISO-9000 procedures needed a lot of managers, and it gave the consultancy implementing the project a lot of bodies to bill for, and it gave the Government (the people who were ultimately paying for it all) a lot of reassurance that it would be done one time, to budget and spec. Except, it was late, the spec trimmed left, right and centre, and if there ever was a budget we blew it.

So you see I’m no fan of carry extra weight on a project.

But, you need people who don’t code.

I’ve written before about the role of the Product Manager and why product management is important, and why you need them. So lets say we accept that Product Manager are needed.

In an idea world we wouldn’t need software testers either. And I expect one day to work somewhere where there are no testers. I don’t believe this is an impossible dream. But with the current state of play we need software testers in most organizations.

One role I don’t think we need is Software Architect. On the whole I think most architects are mislabelled. They should be Product Managers, Business Analysts, Senior Software Designers/Engineers or even Project Managers. Sometimes this is because the organization misuses the title, sometimes its because they don’t understand it (after all, what is software architecture?) but most of the time its because of title inflation. You couldn’t give someone a pay rise so you gave them a nice title.

Although I don’t read Joel Spolsky very often I do agree with him about Astronaut Architects. The moment your architects stops coding its time to fire them. Once they stop coding they are a burden not an asset.

Project Managers are another group I don’t think we need. I accept teams need some leadership but Project Managers are completely the wrong people to. Project Managers are trained to analyse, dissect, use whips (badly) and emit hot air. Their are not leaders by nature or training.

The Agile world is re-inventing project management as coaching but unfortunately the Agile world is pretty messed up about coaching. Coaches are useful when you they get it right and they are genuinely improving the team rather than acting as Project Managers with a different title.

But, and this is where my younger self will have a fit: you do need someone to oversee and manage the whole development process. The big problem is very very few people can do this job right.

By way of explanation to my younger self: done badly this role is worse than useless, done right it can really improve things. Unfortunately most people do it badly.

Its done badly for several reasons.

Firstly, most Software Managers have never been trained in the role – or management at all. They ‘make it up as they go along.’

Second, some of these managers don’t understand software development, they come from outside the domain. Such people try and make software development fit their existing model of management but it never will. Therefore they are constantly trying to make a square fit in a round hole.

Third, many of the managers who come from inside the domain, and know how to develop software can’t give up coding software. To be a good software manager you have to stop being a coder. You have to learn to look at the world differently, you have to realise you can’t intervene to fix problems, you can only arrange things so other people can fix them.

When I was younger I once worked for a bank. The code I was working on was terrible and I had no support from the organization. I really really wanted my manager, and his manager, to come and code with me. I thought their big mistake was to stop coding. I was wrong.

Their biggest mistake was not to stop coding but to stop communicating with those who did code. The big mistake was to put too much space between themselves and the building process. Had they involved themselves more they would have better understood the problems and would have been able to arrange things so I could fix them.

If you are managing a software team you should be talking to them every day. You should be improving their working conditions every day. You should be allowing work to flow to them.

When you move from coding to management you have to give up part of yourself, you have to trust the people you manage. It is their code now, not yours.

So, in conclusion: it is wrong to think you only need developers to create software. It is also wrong to think you can code and do other things – you need the separation. But it is also wrong to have too big a separation.

Make any of those mistakes and you just moved one more duck out of line.

 

New pieces on the website

 

I’ve been holding out, waiting until I have the time to write a really great start-the-year blog entry. Well I’ve not had the time and I’m about to go on holiday for a week. Still, just time to say I’ve posted some new pieces on my website.

• Session notes from Creating Awareness / Exposing Problems is the result of my ACCU 2007 conference session. These notes were published in ACCU Overload last month and are now on the website.

• My patterns from VikingPLoP 2007 (More) Patterns for Software Companies were uploaded just before Christmas but never announced here.

And what for this year?

Well I think I’m going to try and blog less. Yes less. Two reasons really.

Firstly I have plenty of ideas for blog pieces but quite a few of them would be long entries. Some of these I will write up but I am included to place the majority in ACCU Overload. When a pieces runs to several pages I think this is a better forum. I’ll still upload them to the website and announce them here but by their nature they require more time. More time on these means less time elsewhere.

Mentally I’ve accepted that I am going to write a second book. This will be concerned with, well, Business Patterns and Strategy for Software Companies (umm, that could be a good title.) Don’t expect it much before 2010, I intend to avoid signing a contract for a while. I’ll divert some of my material to this instead of the blog.

Secondly, I’ve spent most of the weeks before and during Christmas researching a new business idea and developing a business plan. Very few people know what I’m planning but this select group increased by one at lunch time.

That also means my consultancy work may have to take a back seat when things get going.

(By the way, if you know anyone who would like to invest please let me know. Current projections suggest I need about £100k).

In the meantime, if your missing this blog you can buy the book. Changing Software Development is out in a few days, I heard today that Amazon are already publicising it in e-mail.

And if that is not enough I’ll be appearing in Cambridge later this month, at SPA and at ACCU 2008 – events list here.