What if it is all random?

iStock-512907832Small-2017-08-10-12-37.jpg

What if success in digital business, and in software development, is random? What if one cannot tell in advance what will succeed and what will fail?

My cynical side sometimes thinks everything is random. I don’t want to believe my cynical side but…

All those minimally viable products, some work, some fail.

All those stand-up meetings, do they really make a difference?

All those big requirements documents, just sometimes they work.

How can I even say this? – I’ve written books on how to “do it right.”
I advise companies on how to improve “processes.” I’ve helped individuals do better work.

And just last month I was at a patterns conference trying to spot reoccurring patterns and why they are patterns.

So let me pause my rational side and indulge my cynical side, what if it is all random?

If it is all random what we have to ask is: What would we do in a random world?

Imagine for a moment success is like making a bet at roulette and spinning the wheel.

Surely we would want to both minimise losses (small bets) and maximise wheel spins: try lots, remove the failures quickly and expand the successes (if we can).

I suggested “its all random” to someone the other day and he replied “It is not random, its complex.” And we were into Cynefin before you could say “spin the wheel.”

Dave Snowden’s Cynefin model attempts to help us understand complexity and the complex. Faced with complexity Cynefin says we should probe. That is, try lots of experiments so we can understand, learn from the experiments and adjust.

If the experiment “succeeds” we understand more and can grow that learning. Where the experiment “fails” we have still learned but we will try a different avenue next time.

Look! – it is the same approach, the same result, complexity, Cynefin or just random: try a lot, remove failure and build on success. And hang on, where have I heard that before, … Darwin and evolution; random gene mutations which give benefit get propagated and in time others die out.

It is just possible that Dave is right, Darwin is right and I am right…

Today most of the world’s mobile/cell telephone systems are built on CDMA technology. CDMA is super complex maths but it basically works by encoding a signal (sequence of numbers, your voice digitised) and injecting it into a random number stream (radio spectrum), provided you know the encoding you can retrieve the signal out of the randomness. Quite amazing really.

Further, provided the number sequences are sufficiently different they are in effect random so you can inject more signal into the same space.

That is why we can all use our mobile phones at the same time.

Put it another way: you walk into a party in London, in the room are Poles, Lebanese, Germans, Argentinians and the odd Brit. They are all talking in their own language to fellow speakers. Somehow you can hear your own language and join the right conversation. Everything else is random background noise.

Maybe the same is true in digital business and software development…

Perhaps it is all complex but it is so complex that we will never be able to follow all the cause and effect chains, it is so complex that it looks random. Dave is right with Cynefin but maybe there is so much complexity that we might as well treat it as random and save our time.

Back to CDMA and London parties, faced with apparent randomness there are useful strategies and signals can still be extracted.

Perhaps the way to deal with this complexity is not to try and understand it but to treat it as random. Rather than expend energy and time on a (possibly) impossible task accept it as random and apply appropriate strategies.

After all, if we have learned anything from statistical distributions it is that faced with actual and apparent randomness we can still find patterns, we can still learn and we can still work with, well, randomness.

Programmer’s Rorschach test

The picture above, I recently added this picture to Continuous Digital for a discussion of teams. When you look at it what do you see:

An old style structure chart, or an organization chart?

It could be either and anyone who knows of Conway’s Law shouldn’t be surprised.

When I was taught Modula-2 at college these sort of structure charts were considered superior to the older flow charts. This is functional decomposition, take a problem, break it down to smaller parts and implement them.

And that is the same idea behind traditional hierarchical organizational structure. An executive heads a division, he has a number of managers under him who manage work, each one of these manage several people who actually do the work (or perhaps manage more manager who manage the people who do the work!)

Most organizations are still set up this way. It is probably unsurprising that 50 years ago computer programmers copied this model when designing their systems – Conway’s Law, the system is a copy of the organization.

Fast forward to today, we use object oriented languages and design but most of our organizations are still constrained by hierarchical structure, that creates a conflict. The company is structurally decomposed but our code is object oriented.

The result is conflict and in many cases the organization wins – who hasn’t seen an object oriented system that is designed in layers?

While the conflict exists both system and organization under perform because time and energy are spent living the conflict, managing the conflict, overcoming the conflict.

What would the object-oriented company look like?

If we accept that object oriented design and programming are superior to procedural programming (and in general I do although I miss Modula-2) then it becomes necessary to change the organization to match the software design – reverse Conway’s Law or Yawnoc. That means we need teams which look and behave like objects:

  • Teams are highly cohesive (staffed with various skills) and lightly coupled (dependencies are minimised and the team take responsibility)
  • Teams are responsible for a discrete part of the system end-to-end
  • Teams add value in their own right
  • Teams are free to vary organizational implementation behind well defined interface
  • Teams are tested, debugged and maintained: they have been through the storming phase, are now performing and are kept together

There are probably some more attributes I could add here, please make your own suggestions in the comments below.

To regular readers this should all sound familiar, I’ve been exposing these ideas for a while, they draw on software design and Amoeba management, I discuss them at greater length Xanpan, The Xanpan Appendix and now Continuous Digital – actually, Continuous Digital directly updates some chapters from the Appendix.

And like so many programmers have found over the years, classes which are named “Manager” are more than likely poorly named and poorly designed. Manager is a catch all name, the class might well be doing something very useful but it can be named better. If it isn’t doing anything useful, then maybe it should be refactored into something that is. Most likely the ManagerClass is doing a lot of useful stuff but it is far from clear that it all belongs together. (See the management mini-series.)

Sometimes managers or manager classes  make sense, however both deserve closer examination. Are they vestige from the hierarchal world? Do they perform specialist functions which could be packaged and named better? Perhaps they are necessary, perhaps they are necessary for smoothing the conflict between the hierarchal organization and object oriented world.

Transaction costs can explain both managers and manager classes. There are various tasks which require knowledge of other tasks, or access to the same data. It is cheaper, and perhaps simpler, to put these diverse things together rather than pay the cost of spreading access out.

Of course if you accept the symbiosis of organization and code design then one should ask: what should the organization look like when the code is functional? What does the Lisp, Clojure or F# organization look like?

And, for that matter, what does the organization look like when you program in Prolog? What does a declarative organization look like?

Finally, I was about to ask about SQL, what does the relational organization look like, but I think you’ve already guessed the answer to this one: a matrix, probably a dysfunctional matrix.