Blog

Agile and the Demand Curve

In the last two posts I’ve taken two different views on what might happen to the Software Demand curve when we introduce Agile:

When we discussed how Agile effects the software supply curve things were pretty clear cut. Agile, in the long term boosts supply, and may even flatten the curve – making it more responsive to price changes, i.e. more elastic. Unfortunately things are not so clear cut with the demand curve.

Good news: There is a story Agile people – OK me! – sometimes tell – about what “should” happen with demand in an Agile environment.

Bad news: The is another story about demand in an Agile environment, one that Classicist (i.e. believer in “Waterfall” style up front requirements and planning) fear. And I’ve seen that too. It ain’t pretty.

But, the key point is: Few of the Agile tools work directly on the demand curve. After all, Agile is about software and the demand for software is derived from something else so why would they?

Let me say that again in other words because it is important: Nobody wants software for the sake of software, the demand for software is derived from some other demand, e.g. a retailer wants to sell online, that means they need software.

The current Agile tool set (e.g. User Stories, Acceptance Criteria, Specification by Example, etc.) which touches on demand – call it need, want, requirements, specification, whatever you want – does not address the underlying demand. These tools operate with the demand as presented.

Sure there can be feedback from the tools and Agile deliveries but feedback is often absent. Even when feedback is present those responsible for the underlying demand may not want to hear – or act – on the feedback. And in an environment where success is measured by criteria like “On Budget, On Schedule, On Features” acting on feedback may be impossible.

Which brings us back to the 3 Styles of Agile: Iterative, Incremental and Evolutionary (part 1) and 3 Styles of Agile part 2. (At some point in the future I will reprise those three styles in the light of this analysis.)

One of the key factors in determining whether the feedback can affect underlying demand is whether those responsible (e.g. senior managers) who largely originate the underlying demand are open to feedback. Which brings us to another story I need to tell, another time…

Where does this leave us then?

If the current Agile toolset cannot address the underlying demand we need new tools. We might choose to add those tools to the thing we call Agile or we might choose to put them in another category.

Personally I believe many of those tools do exist but they exist outside the Agile space. They are fellow travellers if you will. For example: Beyond Budgeting is clearly one of those tools. However, many of those tools have their own names, they are not Agile, they are what they are (e.g. Beyond Budgetting). They fit very nicely with Agile but they are not.

However that is my personal view. I also think that Agile is a marketing bang-wagon and those tools will be conscripted to the Agile course and in many cases put under the Agile umbrella.

Either way the problem we need to recognise is: addressing the software demand curve Agile is necessary but not sufficient by itself.

The bad Agile demand curve

In explaining the Agile Demand curve I told a good news story in the previous entry. This time I want to tell a bad news story. This is every classical Project Managers’ fear about Agile, its story we don’t tell often but it can happen. Once I’ve told this story we can proceed to the proper analysis of the Agile Demand curve.

So a classical project manager looks at Agile and he sees his greatest fear: No Scope limit.

There are no signatures against exactly what is to be build, not even in pencil, let alone blood. There are some loose “Story things.” But he is told it is Agile and Agile works this way so against his better judgement he gets stuck in – after all, he needs to get Agile on his CV sometime.

And to some degree the classical Project Manager is right to have this fear.

After years of being pushed around by IT and IT’s sub-contractors the business hears of Agile and thing “Yes! – we are finally going to get what we want. No more messing about with long documents, no longer waiting for ever and no more bugs.”

These days when I go to talk to a company about “going Agile” the Agile publicity machine has beaten me to it. Unfortunately for some on the business side they believe that “going Agile” means they can have whatever they want whenever they want it. If the development team are Agile, so the logic goes, then they will deliver fast and will deliver exactly what is wanted even if the business keeps changing their mind.

You can hardly blame the business side from believing this, after all, it is how Agile presents itself. Unfortunately Agile as “all things to all people” and “I don’t need to change what I’m doing” is more prevalent than is healthy.

The next effect of this attitude is to boost the Demand curve significantly, and if it does affect elasticity it probably reduces it further.

DemandRises-2013-12-23-12-03.png

The demand curve moves from D to D’. At every price point the business wants more. Of course the business doesn’t think of it that way. They are just asking for what they always wanted but now, because it is “Agile” they are not restricted.

Previously, the need to put every request in writing – voluminous writing – and argue it with various managers meant making the request required effort (a price) which they were not willing to pay. No with Agile all that is gone and demand is unbounded.

There is one “project” I know which is quite large, we are talking dozens of people all counted. Demand is rampant, the business keeps on thinking of new ideas, analysts keep finding more work in what has been asked for, developers find more work and the testers more (because initial quality is not what it could be.)

In short: I’ve seen rampant demand happen and I think Agile makes it more likely because it removes many of the restrictions previously used to control demand.

If the team don’t deliver on (often unreasonable) expectations “Agile” becomes another tool with which to beat the team.

It is worth noting here: that as demand is derived much of that demand always existed but was hidden. What is a more interesting question to ask is: which elements of this demand are worth doing? That question will need to wait.

We need to add supply to our diagram above. There are two scenarios we need to consider here and shows in this diagram:

DemandRisesSupply-2013-12-23-12-03.png

Scenario 1: Demand moves to D’ but the developers are still operating on the original Supply curve S. In which case the amount and the price is going to rise in both dimensions. The next diagram shows this, the yellow area is the pre-Agile cost while green area shows the additional cost.

DemandRisesSupplyColours-2013-12-23-12-03.png

Scenario 2: Demand moves to D’ again but this time the team are operating on the Sa (Supply Agile) curve. Thus we get this diagram:

DemandRisesSupplyColours2-2013-12-23-12-03.png

Things are more complex now: because supply is elastic and responds to price more software is produced than at the beginning but the unit price is lower. So the blue are represents increased cost but the purple area represents money saved.

The immediate question I bet you are asking is: have they saved more than they increased?

As it happens yes. On my graphing tool the purpose area takes up 45 grid squares and the blue 46. So they have saved, just.

But, those curves were placed by my for readability without any data for any actual team. The difference is small and demonstrates things could go either way.

A more important question which should be asked is: are the team delivering more benefit? More value?

When I started this blog entry my aim was to show that talking Agile could by itself increase demand. The problem I want to highlight is rampant demand, and this is not confined to Agile. Although I think Agile can make it worse.

What I have also shown is that if the development team really is Agile they can handle this but it isn’t clear.

The good Agile demand curve

I’ve been slow with this entry on Agile and the Demand Curve, apologies. Part of the reason for the delay is software demand is not simple.

In my last entry on software economics I discussed why the Software Demand curve for custom software is both high (i.e. people want a lot of software) and inelastic (i.e. increases in costs don’t reduce demand by much.) This time I want to look at how applying Agile changes the demand curve.

In order to explain what happens to the demand for software in an Agile environment we need to consider different possibilities. So lets start with a good news story – the bad new story and analysis will come in later blog posts.

This story is about how, using Agile software development, a team reduced the demand for software and shipped a product sooner. This is the story most Agile advocates will associate with and happily promote. This is the way Agile is supposed to work….

There was once a team who adopted Agile software development. At first they were able to demo their enhanced software to others in the business on a more regular schedule. They made a big thing about this and invited others to come and look. This generated feedback on the thing they were creating: some good, some not so good, some requests for more and some cancellations of things which had been asked for.

A little while later they stated making more regular releases. And because the software was “ready to release” more often the business could decide: “Do we want it now – with any pain that might bring? Or do we want to wait a little longer for more?” (A bird in the hand is worth two in the bush as they say.)

Simultaneously the team were adopting technical practices which improved the quality of their code – they also had the side effect of reducing the amount of code. Not only did this make the team more productive because they had less code to maintain and bugs to fix but it also meant the business had fewer reasons to complain about bugs.

(In my experience, for some teams when the conversations about “Have you fixed it yet?” and “Can we release it yet?” start to disappear it turns out business don’t know what to ask for. They’ve forgotten or never knew what they really wanted.)

The team also adopted User Stories at some point. As they got better at this technique they were challenged to think about “Who actually wants this story?”. With a little time stories advanced from “As a User” to something more specific.

And one day the “Product Owner” realised: they were not building one product but two.

The Product Backlog of work to do would be better throughout of as two backlogs for two products, each with its own user/customer base. User Stories had helped him see this.

When he restructured the backlog he found he had three piles. Product A, Product B and not justified. A chunk of the backlog, maybe as much as a third, didn’t fit in either product. They were nice ideas, things people had asked for, but when you came to analyse them they didn’t belong anywhere and justification was weak.

ThreeDemandCurves-2013-12-21-12-39.png

This diagram shows the Demand Curve graphically. The team start with demand D, Agile supply is Sa. After the restructuring to separate products appear with their own demand curves Da and Db. Each on of these can be reasoned about independently.

At any price point the combined demand of A plus B is less than D (the original total) because some work has gone away. Note: I have not suggested any changes to the slope of the curves, it is possible that different products will have different elastics but that is another discussion.

While the team had been battling lots of bugs and their output was a black hole to the rest of the organization, and while they thought they were building one big product this insight had been hidden from them.

That is the way Agile is supposed to work on the demand side. By being clear, by being delivery focused, by feeding back to users and customers people get what they want.

But, do you know what? All the members of this team are still employed and still work on software. In fact I think the team might have grown a bit. There is still work to do, there is still demand.

Agile reduced the demand in one part but there is still plenty that company wants to do, and much of that means software. Getting demand reduced in one place just leaves the team more time to do other work, other demand.

(OK, if I’m being honest while I have one team in mind while I was retelling this story I’ve simplified it (knocked a few rough edges off) and composited it a bit to include other examples. I don’t think that invalidates the story as this is how it is supposed to be.)

Honoured – I’m on a 100-top lists

Bit thank yo to the guys at ProgramCreek.com for including this blog on their list of “100 High-Quality Java Developers’ Blogs”.

Strictly speaking this is not a Java blog but most of the issues I talk about do effect Java developers.

It also seems that in listing this blog at number 55 (yeah, still room for improvement!) the ProgramCreek.com team have set up a little competition between me and my #NoProjects conspiracist Steve Smith who is just two places behind me 🙂

(Whoops, should have posted this when it was announced! – better late than never)

Software demand curve

Returning to my series of posts applying the tools of economics to software development – Supply & Demand in software development, Software supply over time and Software supply & demand – this time its Agile – it is time to turn our attention to the demand curve. First a reminder of how things start…

SoftwareMarket-2013-12-3-19-08.png

First I need to explain why I believe there is so much demand for software and why I believe the curve is inelastic, i.e. why a higher price doesn’t reduce demand very much. The reasoning here breaks into five groups: technology progress, separation of demand from cost, separation of benefit from delivery, lack of evaluation and the effect of “fixed” work – although the last in that list doesn’t always play a part.

It is important to remember in all this discussion that software is a derived demand. Nobody wants software for its own right, they want it to achieve some other aim.

Technology Progress

Moore’s Law implies that processor power doubles every two years (give or take a bit). Consequently the ability of computers to tackle ever more complex problems and fill more needs constantly increases. Supply creates demand, we want it because we can.

For example, I’m hoping Farther Christmas will bring me an activity tracker for Christmas, I didn’t know such devices existed until recently but I can see it solving one “problem” in my life. These devices didn’t exist until recently, I didn’t need one because they didn’t exist. The underlying problem existed but had been put in the “thats life” bin.

At a company level this means the benefits computers can bring to our companies are constantly increasing and there are new opportunities to save costs or generate revenue. And if the incumbent company doesn’t take these opportunities then Amazon, Google or some start-up will.

Moore’s Law is not alone, Metcalfe’s Law compounds the effect. The more devices we enable and the more devices we connect to the internet the greater the value and opportunities from connecting these and more devices.

Separation of Demand from Cost

Traditional development process – and even Agile processes – tend to separate the demand for software technology from the supply, consequently those requesting technology are isolated from the cost of that technology.

Many practices in software development make this problem worse. We send Business Analysts to talk to “users” about “What they Want” i.e. what is their demand. These BAs are sometime little more than order takers, waiters. People are encouraged to ask for more and more. Indeed, some of our practices incentivise people to ask for more and punish them for not asking for it.

Even if “users” don’t ask for it Analysts may be encouraged to include any need they can comprehend. I once had a BA tell me “If we miss a something in the requirements document it is seen as a black mark against the BA.”

Separation of Benefits from Delivery

To make things worse its not just demand which is separated from costs but so too are the benefits. Nobody wants IT for IT’s sake, they want it for some benefits. But achieving the benefits may well take time, it might be months or years after an IT system is delivered before the benefits are seen.

That is, if the benefits are ever seen, IT alone is not enough to bring about benefit. Users may need training, changes to processes may be needed, customer agreements may need to be changed and so on.

Although it is a few years old now Wired for Innovation contains a good, short, discussion of why IT benefits don’t appear in the way we would like them too (i.e. quickly and easily) and what can be done to help.

Lack of evaluation

Ideally companies would evaluate the benefits delivered by IT work but all too often this fails to happen. Instead companies may rely on the original claimed benefits. But this might the result of optimistic thinking itself: “38% of businesses openly admit benefits are overstates in business cases in order to obtain project funding” – from a Cranfield Business School study I reported in October 2010.

And when one manager exaggerates benefit to get his project funded it creates an incentive for the next manager to exaggerate the benefit of the next project and so on. Benefit inflation cascades through the organisation.

This isn’t helped by vendors who claim benefits for their product which might not apply to a particular business or in a particular context. If company A used product B to save $C millions then surely company X can use product B to save about $C millions, right?

In our personal lives we have become accustomed to technology getting cheaper and more powerful and we expect the same in business. But consumer technology is paid for by millions of people buying millions of products while business technology may have one customer. Consequently we don’t appreciate the costs involved in creating new software.

Goal Displacement: Floorded contracts, projects and governance

In an effort to control IT, to maximise the return organizations frequently fixate on controlling the costs. They demand things like price, time and scope are decided in advance and fixed. Sometimes this is done internally – projects are approved and must be finished according to some set budget or date. And sometimes it is done by finding a supplier who will agree to fixed price/time work. (I’ve written about this before in Dear Customer on this blog and on Agile Connection/TechWell.)

The problem is, when you put these arrangements in place the objective becomes: meeting the fixed criteria, not delivering value. It is what psychologists and management students call Goal Displacement.

Options to reduce scope, time and cost are lost in the process. The things which were meant to be maximums, ceilings, become minimums, floors. Opportunities to reduce costs are lost to more features and more software.

Net effect

Taken individually, together and in combination with other forces we end up with a demand curve which is very high and inelastic. Changing the price of the technology doesn’t reduce the demand by much, Moore and Metcalfs Law mean there are always new opportunities, failure to capture original benefits mean they are still there for the taking, and lack of evaluation means nobody is counting, boys will always make arguments for more toys.

In the next instalment of this series I intend to look at what Agile can do about this and whether Agile makes things better or worse.

Agile is like sex

I’ll get back to software development economics, there are more posts in the works, but right now I need to say something….

I have blogged before on the subject of “What is Agile” – I’ve even expanded on that blog in an unfinished piece of writing called “What is Agile? Perspectives on Agile” – but sometimes I think its just sex….

(OK, this blog just got filtered out of lots of feeds and stopped by lots of firewalls but lets continue.)

In the software development world there are, broadly speaking, two groups of people: those who create the software (coders, testers, etc.) and those who manage the process (project managers, development managers, etc.). When discussing “Agile” I find that both sides think the problem is with the other.

To put it another way, if I’m talking to developers they think its managers who are the block to adopting more Agile techniques and improvement. But when I’m talking to managers they say its the developers who resist Agile and other improvements.

Which always reminds me of the old Philip Crosby quote:

“Quality has much in common with sex. Everyone is for it. (Under certain conditions, of course.) Everyone feels they understand it. (Even though they wouldn’t want to explain it.) Everyone thinks execution is only a matter of following natural inclinations. (After all, we do get along somehow.) And, of course, most people feel that all problems in these areas are caused by other people.” Philip Crosby

Lets bring it up to date and make it Agile specific, substitute the work ‘Agile’ for ‘Quality’:

“Agile has much in common with sex. Everyone is for it. (Under certain conditions, of course.) Everyone feels they understand it. (Even though they wouldn’t want to explain it.) Everyone thinks execution is only a matter of following natural inclinations. (After all, we do get along somehow.) And, of course, most people feel that all problems in these areas are caused by other people.” Allan Kelly after Philip Crosby

You see? It fits.

Agile isn’t someone else’s thing, it doesn’t belong to another person: Its yours. You are as likely to be the block as the person you think is the block.

Software supply & demand – this time its Agile

Carrying on from my previous posts applying the economists tools to thinking about software development (Supply & Demand in software development and Software supply over time). In this post I want to see what happens when we apply Agile…

We start with our supply and demand curves as before:

SoftwareMarket-2013-11-19-11-25.png

First we need to define Agile, not a trivial matter. Lets simplify this by saying “Agile practices”, specifically:

  • Short iterations and regular releases (or at least demos), e.g. every two weeks
  • Planning meetings, planning poker estimation (or similar, or #NoEstimates), velocity (or commitment if you prefer)
  • User Stories, Epics, task breakdown (if you do this sort of thing)
  • Stand-up meetings
  • Team retrospectives and team planning boards or electronic equivalent
  • Test driven development and continuous integration
  • Pair programming, code review and shared code-ownership
  • Acceptance Test Driven Development (ATDD), Behaviour Driven Development (BDD), Specification by Example (SbE), and other automated testing mechanisms

The first thing to note about all these techniques – and indeed the majority of what we tend to call “Agile” – is that these techniques work on the supply side. They operate on the supply curve, the net effect is to increase supply.

Possibly, if used in the right fashion, BDD and SbE might effect the supply side. But since most of the efforts to use BDD I encounter are little more than super-charged TDD we’ll leave it there for now – see Dan North’s blog for a better discussion BDD v. TDD.

Projects – specifically projects but other work initiatives too – are often still judged on the basis of: on schedule, on feature and on budget. Agile in this fashion is what I have called Iterative Agile, or at best Incremental. The team is still seeking to deliver a fixed quantity of software. Thus the demand curve is unchanged. (We will return to the demand curve in another post and discuss how Agile techniques might affect it.)

While Agile tools promise increased performance this is only in the long run. In the short run we can expect teams to slow down as they adopt these tools. We can also expect costs to go up as companies to spend money to buy in this expertise.

Maybe they hire the likes of myself to supply training and consulting in adoption (yes! I’m for hire, see my company website Software Strategy). Even if they don’t pay the likes of me costs will go up because engineers need to learn these skills. Companies rationale for hiring me is that I will help reduce the period of time for which they are slower and reduce the risk. The deal is: hire me and I will help you get to increased productivity faster (and with less risk).

Putting this in terms of curves:

SupplyAgAdop-2013-11-19-11-25.png

A team starting on Ss can expect to move left (1) to Sasr (agile short run) before moving to the right (2) – Salr (agile long run) – reduced supply before greater supply. In the first instance the price per unit increases while the quantity delivered falls but after the second move the price per unit falls and the team produces more software.

Leave consultants like me to one side for a moment and simplify it. Imagine a team which “just does it”. Assume for a moment that the team members stay the same and the amount they are paid each week stays the same. When the team start doing something new they slow down, they produce less software each week so the average price of per unit increases. As they get better the same team better their original performance, earn the same money, produce more software and as a result the average price per unit falls.

Hopefully that all makes sense and isn’t controversial.

But I think something else happens. This is a hypothesis, at the moment I can’t measure it so please tell me – in the comments on this blog – whether you agree or think I’m wrong.

Staying with the supply curve, consider again the tools we were looking at:

  • Short iterations and regular releases make it cheaper to make small changes. Therefore the supply of the small quantities of can should take place at a lower cost. And since the delivery – or at least a demo – is earlier the whole “project” could be cancelled if things don’t work out. Thus the bottom of the curve moves left over time.
  • Automated testing (TDD, ATDD, etc.) reduce the quantity of faulty software produced and increase the amount of useful software increased at any point thereby raising all points on the curve. With less “technical debt” teams can deliver more functionality more cheaply.
  • The same techniques – TDD, ATDD, etc. – and shared code ownership also make it easier for new engineers to become productive because a) there are more code examples of how the system works and b) places a safety net under any work they do, allowing them to experiment and learn faster, and get their changes to production sooner.

The net consequence of these changes is to flatten the curve, not entirely but certainly reduce of an angle. In economics terms we increase the elasticity. Changes in price – adding or removing a developer – causes a bigger effect on the quantity supplied than before.

SupplyAvlr-2013-11-19-11-25.png

Supply moves from the initial Ss curve to the Savlr (Supply Agile Very Long Run). As the chosen name implies, this takes some time to occur.

Now there is a catch. Consider the total amount of software bought and the total price paid (price x quantity.) To do this look at the next two diagram.

SupplyAvlrGreen-2013-11-19-11-25.png

The green area represents the total paid for the software. You can’t see it here but on my graphics tool I can see price is 10 high and quantity 7 wide, so 70 (whatever) units.

SupplyAvlrBlue-2013-11-19-11-25.png

In this diagram, on the new supply curve, the blue area represents the total spend. Again you can’t see it but on OmniGraffle it is 5 high and slightly more than 9 wide, almost 45 units of total spend.

Now if you are producing software for your own company – either to resell (e.g. Microsoft) or to consume yourself (e.g. a bank) – this model is good news. You get more software for a lower overall spend.

But, if you are an outsourced provider of software selling custom software to a client this is bad news. You now deliver more software for less money, about 36% less in total. If profit is tied to revenue (time and materials) then this is bad.

No wonder outsourced providers can be reluctant to embrace Agile or to do it badly. If you are a consumer of outsourced software development services you might want to think about that.

Next time I want to turn my attention to the software demand curve.

(As a footnote, using the same analysis techniques it is possible to show that reducing supply of software – moving the supply curve left, either because the software ages or because Agile adoption is slowing things down – will result in a greater overall spend for less software. This might be good for outsourcer suppliers but isn’t so good for in-house development.)

Blog revisions: economics

Although I don’t think I’ve ever stated it I operate a “no revisions” policy to this blog. I don’t take things down, and I rarely modify past entries. OK, I might change it in the 10 minutes or so after I first post to fix typos and gaffes I see once I see it online but otherwise I avoid doing so.

But my recent posts on software economics, supply and demand, contained inaccuracies. Or rather the graphs I published didn’t follow economics convention. The logic was (mostly) right but the graphs were wrong. I rushed out one fix and breathed a sigh of relief.

Then last night on my way back from Riga I noticed another flaw. I won’t go into details but I’m now fixing/fixed it.

Several times in the past I’ve attempted to capture my thoughts on software economics but nothing I’ve ever written has seen the light of day. As I continue this analysis I’m finding more insights I hope to share. I’m also – possibly – making errors.

I hope other readers who think they know economics can help debug my posts. They will be better for it.

In the meantime, sorry.

Software supply over time

Having outlined the theory of supply and demand in the software market I want to spend this, and the next few posts examining what this analysis can teach us. I’m starting with the supply curve and will look at the demand curve later.

First I want to consider what happens to a software system over time if nothing else is done. (Consider this as a small detour before we consider Agile.)

Over time changes to software systems will add code, this means there is more code to maintain, more code for engineers to understand, more code for testers to test. Internally code quality tends to decay, some people call it software entropy.

Engineers leave the team and new engineers (might) arrive thereby loosing knowledge of how the system works. At least the knowledge is diluted. And if maintenance is outsourced or offshore such knowledge may be destroyed altogether. The new, cheaper, outsourced developers need to relearn.

Meanwhile the business environment seldom becomes any simpler. New business rules get added one place or another – increasing complexity. New systems need to be connected – either as data sources or destinations – which again increase complexity.

This all gives rise to the Big Ball of Mud pattern.

Finally technology changes. The operating system and databases which form the foundations of the system may be upgraded or replaced. In the worse cases the language the system is written in dies.

The net result of all of this is that the system becomes harder to maintain. Harder translates to more time and more time translates to more money. Thus the system becomes more expensive to maintain. Consequently the supply curve for the system moves to the left. For the same price we can now get less software.

Consider the graph below. We have the Supply of Software (Ss) and Demand for Software (Ds) curves as before. Now we add the Supply of Software over Time (Sst) curve. Over time, of nothing is done, the curve itself moves, all other things being equal we do not move to a new position on the curve. We get an entirely new curve.

SupplyDoNothing-2013-11-13-09-35.png

Now it is possible to fight against this trend. And certainly there are plenty of tool vendors out there who will happily sell you a tools which they claim will move the curve in the opposite direction. (In economics terms we break the “all other things being equal” assumption.”)

But either way: unless you fight it things will get more expensive for the same quantity of software – or you get less software for the same money.

Supply & Demand in software development

(Whoops, I’m publishing a revision of this blog entry – something I don’t normally do. I realised after publication that I had mislabelled the graphs – they were correct but did not follow economists convention hence this revision. The core argument still stands although I have had to modify the reasoning slightly. Sorry.)

I’m proud to say I am, or at least was, a Software Engineer. I’m proud to say that I’m a third generation Engineer. My farther and grandfather got their hands dirty with the oily sort of engineering. But I’m also a frustrated Economist. Economics forms part of both of my degrees and on a couple of occasionsI’ve been temped to become a real Economist.

So I am always disappointed when someone talks about Software Economics. I have yet to find a serious study of software economics. Most of the studies, books and papers about “software economics” would be better called “software accounting.” They are about apportioning costs and adding up numbers. Economics is not Accountancy.

I have long believed software engineers and their managers would benefit from a better understanding of economics. In fact many of my clients will be used to hearing me refer to the supply side (development) and demand side (requirements, analysis, etc.).

I find it very useful to think of software development in supply and demand terms and use the tools of economists to understand what is happening. Usually thinking like this shows there supply and demand to be mismatched.

(This blog post that has been a long time coming. Several previous attempts have floundered because there is so much I want to say about software economics – and thinking in supply and demand terms. So while there might be little in this post which is earth shattering – especially to those of you who have studied economics – later posts will build on these ideas and analysis.)

Lets start with the fundamental problem which economic theory addresses: the allocation of scarce resources. After all we have scarce resources in software development so lets use an Economists tools: supply and demand.

Lets have a look at the supply curve:

Supply-2013-11-10-18-11.png

The line marked Sn is a normal supply curve. When price is a bit greater than zero individuals will start supplying their services and companies will come into being to supply the market. As the price increases more individuals and companies will be attracted to the market and supply will increase, more resources are brought to play.

Note two things here:

  1. Economists usually draw these curves for a product within a market. Here I am not examining the supply of multiple copies of a software product, rather I am examining the supply of software development capacity for a unique product. The marginal costs of supplying existing software are as close to zero as to be zero, the marginal costs of supplying new software products – i.e. the capacity to create a product – is significantly larger than zero and that is what we are examining here. In software terms we need to consider a software product, perhaps a single program, perhaps a “project” or a small system.
  2. If you are not an economist don’t worry about the fact that these “curves” are straight lines, the basic analysis still holds, we hold actual curves for an advanced class.

The software supply curve – Ss – starts the same way. When price is zero nobody supplies – perhaps not entirely true if we consider open source but close enough. As price increases more resources are brought into play and supply increases.

Software development is not cheap. If you are buying software developer services they are rarely cheap. The only time you get “cheap” software is when you buy (some) off the shelf stuff. Consequently when the price is low there is no supply of custom software.

But – and this is the BIG BUT – supply raises very slowly. Two people might write twice as much code as one but when they work together the generalised form of Brook’s Law comes into play: adding people to a software development effort slows a team down. Therefore supply responds slowly to price increases – what economists call inelastic.

Now consider the demand for new software:

Demand-2013-11-10-18-11.png

Here Dn represents demand for a normal product, when the price is high there is little demand for the product. As the price falls the demand increases because more people can afford it. (Remember an Economist’s definition of “Demand” is “A want with the ability to pay”. I may want a new iPad Air but if it is more than I can pay there is no demand – some might call this “latent demand.”)

For normal products, in time, the market will “clear” and balance will be reached when the curves meet:

NormalMarket-2013-11-10-18-11.png

At some price (Pn) the quantity supplied (Qn) will match that demanded. Now economists accept that this might not happen instantly – and there are various models for that – but in the scheme of things the time it takes for the market to settle is not a big issue. It will happen eventually.

Until that happens all sorts of problems will ensue, some people might make a lot of profit, some people might go without and some people might have to pay a non-financial price, e.g. queues will form. All good stuff we could consider but lets leave that for another day.

Returning to the demand for software. People and companies do not want software for its own sake, they want software because of what it allows them to do. What Economists call a “derived demand”.

The problem is, and the reason I’ve drawn the the Ds curve as highly inelastic is that we are faced with an ever increasing demand for software. Processor power doubles every 18 months to two years (Moore’s Law) which means the things our computers are capable of doing – with the right software – is forever expanding. When we get this right the benefits can be massive. Which increases the demand for software…

  • How many readers wanted to read newspapers on their telephone three years ago?
  • How many readers knew they needed to watch TV on an iPad five years ago?
  • How many readers took digital pictures with their telephone and posted them online on a social network to instantly share them with friends and relatives seven years ago?

So lets look at the software supply and demand curves together:

SoftwareMarket-2013-11-10-18-11.png

For me, the problem in software economics:

  • Demand is rampant but inelastic so as price rises it does not fall off very fast
  • Supply is severely constrained and is very difficult to increase (particularly in the short run) so is also inelastic
  • Consequently the market only clears at high prices and slowly: consequently markets take a long time to reach equilibrium, which in turn means we need to consider the tensions arising in a market which spends so much time out of equilibrium. Thus the time lags which normally only play a small part in classical economic analysis are more significant.

Each of these statements deserves further examination and explanation in their own right – future blogs, maybe.

More importantly, and also deserving of its own blog entry, is the question: what can be done to bring this market into equilibrium?

That is the question I find myself wrestling with on behalf of clients, and it is one I will return to.