Complexity

We’ve run through a couple models of analysis on the nature of Agile/fluid systems.  Specifically, we’ve looked at that Method Grid, and then recently, we were trying to address which question needed asked.  I think we’ve made a breakthrough in our analysis.

Question:  Why is it that Hero and Director are more natural states than Messiah and Wedding Planner?

It seems as if there’s some natural synchronicity between Independent work and Individual responsibility, and also a natural synchronicity between planning and management.

Perhaps calling them out as separate axes is not the right thing to do…maybe there’s a deeper, underlying question, which is being answered in the same way on different axes that leads to this synchronicity.  And once we looked, we did indeed find that to be true.

The new claim:   There is one question at the heart of Agile.  The Agile manifesto gropes blindly for the question, but seems clearly to be looking for it.  The fluid principles are each a manifestation of the Agile question, in a different domain.  What is the question?

Q:  How do you handle complexity?

And at that same deep level, there are three basic paradigmatic responses.

Paradigm 1:  Simplicity.
A:  We don’t.  Our problem isn’t that complex.
Why do startups run in a profoundly different fashion than large companies?  Because their problems aren’t big enough yet.  They’re applying simple solutions (frequently from a brilliant insight) to simple problems.  In applying simple solutions to simple problems, they don’t address complexity at all.  And that’s wonderful.  For the domains that it works in.

Paradigm 2:  Control.  (aka Mechanical, Resisting, Rigid, Structured, Design, Managed)
A:  In the face of complexity, we attempt to control it.
Once problems become difficult enough, the Simplicity paradigm is insufficient to handle the problem.  We design a complexity management system.  More than a couple dozen people in an organization, and the group dynamics need management.  More than a couple dozen modules in your code, and you need to start doing design.  More than a couple dozen weeks in your project, and you need to lay out a plan.  More than a couple priorities, and you need a schedule.  More than a couple risks, and you need a process.   Everyone knows this system, as every large business we’ve seen has followed this mode.

Paradigm 3:  Fluid (aka Biological, Accepting, Organic, Nudge)
A:  In the face of complexity, we travel with it.
At some point after problems become difficult enough that the Simplicity paradigm is insufficient to handle the problem, so too is the Control paradigm unable to handle the complexity of the system.  Nature is complex enough we cannot direct it.  Rather, we need to work with nature, rather than against it.  In the southwest USA, build from brick and adobe to handle the heat.  In California, build flexible, wooden structures to handle the earthquakes.  In Galveston, build on stilts, to handle the Surges.  Nature will not be commanded…but in flowing with nature, we can get some of what we want.

In the face of hundreds of people in an organization, we need to abolish the formal organizational structure that fights nature, and instead allow natural social systems to emerge.  Tribes sit at the 150-person level.  Hunting parties are near 10.  Pairs work together naturally as well.  Natural social systems don’t sit well with authority.

In the face of thousands of components in a system, and fluid requirements, we need to abandon the notion that a design will capture what we need.  Rather, we need test-driven design and an aggressively maintainable system in order to build good enough architectures to survive the natural change cycle.

In the face of hundreds of competing priorities, we need to abolish the formal method of scheduling, and focus instead.  Biological systems like human beings and human organizations have limited bandwidth.  Scheduling 6 things at once doesn’t get 6 things done faster.  Rather, it usually means that at least 5 things get done slower than they would have,  and usually aggressive prioritization/queuing/focus gets 9 things done faster than 6 things in a scheduled fashion.

In the face of innumerable risks, process cannot save us.  We need a better answer.  A natural, flowing with human patterns instead of against them answer.  The answer that gets us there is ceremony.

In the face of more than a couple encounters with changing reality, planning systems crumble under the weight of replanning.  Instead, we need fluid, biological responses to changing conditions.  We need the ur-mechanism for responding to reality.  Feedback systems are the only answer.

There is only one question:

How complex is your problem?

If you have easy problems, a simple paradigm can solve them.
For mildly difficult problems, the control paradigm is standard because it handles them.
But for the actually hard problems, fluid systems are the only answer.

It’s interesting also to consider how many problems fall into the moderately difficult category…and how much overlap there is between the problem-solving capacity of control vs. fluid systems?  How many problems can control solve (well) that fluid can’t?