I think it is not possible to implement a sustainable Agile practice thinking only in terms of teams of 5-10 people, especially a sustainable Agile practice in large companies. I have nothing against small teams – quite the contrary! But there are distinct advantages to including our teams in a larger organism that supports and sustains the Agile teams.

We need to think in terms of Tribes.

I have been familiar with the ideas around Tribe for a very long time, and have been part of creating Tribe on many occasions.  Dedicating people to a Tribe and allowing them to move around to different teams within the Tribe provides distinct advantages without the thrashing and loss of velocity that occurs when we move people between teams that are not part of a Tribe.

What is a Tribe?  A Tribe is a group of around 100 people who are dedicated and co-located, and who have a commonality of interests. A Tribe develops its own language, culture, and rituals distinct from other Tribes. A Tribe contains people of many ages and experience levels, with the most knowledgeable people acting as “elders” who guide and mentor those with less experience. People in the Tribe work together, play together, and eat together.

Why about 100 people? It is large enough to get a varied pool of talent and skills, and small enough that everyone can get to know everyone else. These are very important elements to creating a stable Tribe.

Doing paired development is the best way I know of to help a Tribe form. Not just paired programming, but pair all the tasks of the Tribe. As people move around pairing with a variety of other people within the Tribe, we get terrific knowledge sharing, and people get to know each other and how to work together.  It is this broad knowledge sharing and knowing how to work together that lets us have fluid teams without the disruption we usually see when people are moved on and off a team.

Advantages to a Tribe are many:

1. Allows Tribal members the opportunity to broaden and deepen their experience by working on a variety of teams with a variety of people
2. Allows for load balancing of work among a number of teams without the disruption that typically occurs when people move between teams
3. Knowledge is spread among many people so you do not risk losing the one person who knows X
4. It is easy to bring new people into the Tribe and get them quickly up-to-speed
5. If your company has pools of specialists who are stretched thin trying to help so many teams, a Tribe of 100 may be large enough that one specialist can have enough work to be dedicated to the Tribe so the Tribe does not have to wait for their time.

Many companies do form Tribes, though they probably do not call it that. They dedicate a group of analysts, testers, developers, technical writers, etc. to one product line, and within that product group, individual teams work on specific work packages. It works quite well, and fits nicely with the Agile focus on Products rather than Projects.

We can do more to promote the idea of Tribe as a purposeful construct, deliberately forming a Tribe by co-locating people with many different skills and experience levels, dedicating them to the Tribe, pairing all tasks to share professional knowledge and get to know how to work together, and encouraging the most knowledgeable people to mentor those less skilled.

Hero is one of several methods we have of solving problems.  And, as you can tell, gentle reader, it’s not a particularly Agile method of solving problems.  Indeed, the term was originally coined in order to be used as a foil to carefully differentiate between a few standard problem-solving methods.

However, in our exploration, we discovered a few more methods, and discovered that you could categorize them neatly.  Let’s begin with hero.  When we explored the characteristics of a heroic approach, we identified two factors as being the primary signifiers.

First, hero is about independent actors, each doing what they do well.  Perhaps they’re in a group, and perhaps not, but they remain independent.  Dirty Harry and John McClane (Die Hard) are iconic individual movie heroes.  The more recent Avengers movie show us a group heroic effort.  Each hero does his or her own thing, brilliantly, and independently (except for a few touching scenes in which they help one another for a few seconds).  But the independence is central to the heroics.  Each participant helps, but they each work solo.

Second, we might call out that the heroic approach relies on expert intuition to solve problems.  Each participant is required to be expert in order to solve the problem.  And it is by virtue of their expertise that the problem gets solved.  In the case of the two cop-hero movies, it is their grit, determination, individual excellence, and their long careers as cops that make them succeed.  In the Avengers movie, it’s their individual superpowers (a very impressive form of expertise).

If we were drawing a diagram, we might see hero sitting at the intersection of independent and intuition:

With hero as the first method, the second method we named is now generally referred to as Director.  Director is modeled after the approach of any legendary movie director.  Steven Speilburg is among the best of the modern examples.  How does Spielburg develop a movie?  He does it very differently than a hero would, both on the grouping axis and on the optimizing axis.

On the optimizing axis, Spielburg makes a plan.  Not only does he make a plan, he takes years to make a plan.  Get a thousand scripts.  Read them all.  Throw them all away, except the best two.  Rewrite/improve those scripts 30 times each.  Pick the better of the two.  Rewrite it another 30 times.  Nail down every shot location, every line, and every look in the movie.  And then?  Hire some actors to make it work.  Spielburg is a planner.  A darn good planner, but his fundamental method for excellence is nonetheless to plan exceptionally well.

On the grouping axis, Speilburg manages his group.   Not only does he have a plan, and share his plan with the team, but when the team doesn’t follow his plan to the tee, he cajoles sometimes, yells sometimes, fires someone even sometimes, and then gets back to the business of managing the details of all 7000 participants in the movie-making process.

A director, to be successful, plans and manages his team.  A heroic group operates with intuition and independence.  These are hugely different.  Putting these two into the grid, we now have this:

Once we see this relationship, it becomes pretty easy to fill in the other two quadrants.  Intuitive-Managed is Supernanny or Gordon Ramsay or any other Managing Expert approach, including essentially all management consulting.

Planned independent is a lone caterer, prepping a meal for 500.  Alternately, an individual carefully planning his or her stops on a cross-country driving vacation.

Our graph has grown

This exhausts most normal methods for solving problems.  However, we assert there are more, and that they fit nicely into our grid.

Let’s take another method, and see if we can place it.

The notion of a high-performing team (HPT) has been discussed in management literature for the last 60 years.  What are the characteristics?  The three primaries are group-decision-making, and highly competent individuals who flow into one another’s roles as needed.  Rather clearly, they optimize their decisions based on intuition and expertise, but they organize in a completely different fashion from the independent and managed groups.    The simplest way to describe their method of organization is tribal.  The team forms a common bond with a common purpose, and in a very egalitarian fashion (while recognizing individual expertise) moves towards a solution as a tribe.

This expands the grid to look like this:

But wait, there’s more (It’s not sold in any store).   We know of another method for solving problems besides intuition and planning.  What does a scientist do?  A scientist proposes a hypothesis, and then runs an experiment to determine if (s)he is right or not.  Rather than planning a correct result, or relying on years of expertise and the hard-won intuition, a scientist gets to the truth via feedback.  Michaelson-Morley figured they’d measure the directional flow of the ether.  Turns out that they couldn’t find one.  Indeed, that experiment in which they attempted to measure something turned into one of the foundations of Einstein’s theory of relativity.  Run an experiment.  Measure the results.  Decide what to test next.  Run another experiment.  Eventually we converge upon the truth, but we get there only via feedback systems.   Expert intuition, or even perfect planning are simply insufficient.

Note: I am posting this in the off chance you haven’t read something like it before.

Gordon Moore¹ observed in 1965 that the surface area of a transistor was being reduced by 50% each year. The press called this observation Moore’s Law. It is especially significant to us because the transistor is the fundamental building block of computation technology, the integrated circuit.

He predicted in 1975 that, for the foreseeable future, computer chip density would double every two years and with it computer power. At the same time, Moore observed that the cost to manufacture the computer chips was remaining relatively constant. If you bought your first new microcomputer in 1975, according to Moore’s Law you have observed the following: 

1977 – New computers are 2 times faster than mine in 1975
1979 – New computers are 4 times faster than mine in 1975
1981 – New computers are 8 times faster than mine in 1975
1983 – New computers are 16 times faster than mine in 1975
1985 – New computers are 32 times faster than mine in 1975
1987 – New computers are 64 times faster than mine in 1975
1989 – New computers are 128 times faster than mine in 1975
1991 – New computers are 256 times faster than mine in 1975
1993 – New computers are 512 times faster than mine in 1975
1995 – New computers are 1024 times faster than mine in 1975
1997 – New computers are 2048 times faster than mine in 1975
1999 – New computers are 4096 times faster than mine in 1975
2001 – New computers are 8192 times faster than mine in 1975
2003 – New computers are 16,000 times faster than mine in 1975
2005 – New computers are 32,000 times faster than mine in 1975
2007 – New computers are 64,000 times faster than mine in 1975
2009 – New computers are 128,000 times faster than mine in 1975
2011 – New computers are 256,000 times faster than mine in 1975
2013 – New computers are 512,000 times faster than mine in 1975

Do you have enough computational power to solve your business problems yet? 

Practically speaking, Moore’s Law has been in operation throughout the entire careers of most people in the software development industry. Our computer hardware is threatened with obsolescence every few years.

You can also think of Moore’s Law another way. In 1975 you would have paid $67,000 for computational ability you can buy today for a few cents. In the world of computers the golden triangle of faster, smaller, and cheaper is actually true.

Moore’s Law impacts us in ways we often don’t first realize. For example, answer the following question: How many computers do you own? Before you jump to a quick answer think for a moment. How many computers are in your car? In your home? In your kitchen alone (the microwave, stove, refrigerator, toaster oven, etc. all likely have computers inside)?  How many computers are you carrying on you right now?

Moore’s law affects a lot more than just the “personal computer.” It has created a demand for customized computers and software for just about every electronic device imaginable; more devices, more computers, more software, more complexity in the technology that runs our society.

 Are you developing software to run on your customers’ current computers? How many changes of operating systems and hardware should your software be designed to survive?  What will it look like in the new iPad, on a larger cell phone, in Google Glass?

To survive in this environment you need to make and continually revisit strategic business decisions including which new technology to adopt, how much and when. Yesterday’s answers may not be at all appropriate today for the simple reason that you can no longer even acquire yesterday’s technology. This is a game you cannot get out of, some of us have tried. 

Context matters. The context for developing software products is a hyper-accelerated society. Technology advances at an ever accelerating rate and business is a direct beneficiary, and sometimes casualty.  The world has seen more technological innovations in the past 50 years than in all the years of previous human history combined. And there is no sign that the rate of change is letting up. In my lifetime, I’ve watched our society embrace and adopt the following significant new technologies in an increasing accelerated pace:

• Tablet Computers: 3.5 years to adoption by ¼ of U.S. population
•  

• The Internet Browser: 7 years to adoption by ¼ of U.S. population

• Cellular Telephones: 14 years to adoption by ¼ of U.S. population
• Personal Computers: 16 years to adoption by ¼ of U.S. population

Do you see the trend? It is important to understand the context of the society we find ourselves in, an accelerated society. The following three principles help capture the this context:

• Moore’s Law
• Metcalfe’s Useful Equation
• Disruptive vs. Sustaining Technology

I will post on them separately, or if you are impatient you can Google them yourself. If you are going to do agile development, these are worth studying and understanding.

The Department of Homeland Security, U.S. Cert, and other private organization continue to raise concerns about the significant vulnerabilities that exist in U.S. Information Technology (IT) infrastructure (e.g. computers, operating systems, phones, software, servers, databases, and networks).

Our economy has become significantly dependent on our IT infrastructure to conduct almost all business and this trend continues to expand. Unfortunately, there is reason to believe that a highly coordinated and sophisticated attempt to disrupt the operations of this infrastructure could succeed. Clearly, our networks and computers are vulnerable to attacks where even unsophisticated high schools students can inflict more economic costs than a Florida hurricane.

Attacks come in many forms. Hacking is where a user gains direct control over a computer, usually by thwarting the log-in and firewall mechanisms. Viruses are self replicating code fragments that infect computers automatically. Trojan horses are social tricks, where the user is tricked into executing hostile code by appearing to be something else.

The current responses to these security threats is a reactive one, typified by updating virus software and downloading various application and operating system software patches. The weakness with a reactive response is that it typically occurs only after an attack has been successful. It takes time to identify new attacks, it takes time to update the virus filters and security holes in the operating systems and other software, and it takes time to distribute these new updates to all of the computers on the network. During all of this time, significant damage is occurring to the economy.

The reactive response does provide increased security but at great risk. Reactive responses require that the filters be continually updated, and because each new attack requires a customized response, even hundreds of people can’t properly keep up with all of the attacks. These reactive programs, in an attempt to combat the attacks, continue to grow in complexity and size causing a signification reduction in machine performance, plus they tend to introduce new defects and more vulnerabilities, and negatively impact worker productivity.

We have been fortunate that no enemy has really released a truly morphing virus, which continually changes form and method of attack. Such a virus resists all standard filter attempts. We have been fortunate that no enemy has tried more subtle attacks; such as changing just a few of the numbers in every spreadsheet on a machine and then deleting itself. These types of attacks could bring a halt to the information economy as companies spend trillions of dollars trying to sort out good data from compromised data.

So, what conditions have most led to our current crisis of vulnerability? Interestingly enough it is our historic strengths in mass production and uniformity that cause these vulnerabilities. Currently, all of our machines are fundamentally the same. If you can successfully infect one machine you can successfully infect most of the machines. Hundreds of millions of machines in government, business, and private homes all have the same software installed, the exact same version, they look exactly alike. If you break one machine, you have successfully broken a hundred million machines.

Note, these vulnerabilities are almost impossible to eliminate simply by better programming.

We need a new solution. A solution that takes the hundreds of millions of existing machines that all are exactly alike and makes them all dramatically different—automatically. We need a proactive active strategy to protect against infection, before a new attack is even conceived. A proactive solution does not wait to see how computers are compromised and then add a new filter to stop that attack. Instead, the system leverage the best of encryption, advanced pattern detection, and proprietary polymorphic behavior (i.e. continually changing forms) to insure that a virus, hacker, or Trojan has no place to go.

A proactive solution continues to work even if a machine in successfully attacked, that machine automatically identifies the attack and actively attempts to remove it and restore operations.

A proactive security world has all of the machines expressing in different forms. If any individual machine is compromised, it is highly unlikely that that technique can be used to attack any other machine. A system acting more like a human body, responding automatically to infection by changing its defensive forms until the hostile code can no longer survive. Our machines must actively fight infection.

This will only be accomplished by taking a fundamentally different approach to the problem. We must leveraging the power of the processors already in the computer to full advantage as security processors. Security must become a fundamental property of the machine, not an afterthought downloaded from a virus software vendor. No virus updates and search patterns to download. No large teams attempting to respond to the latest attack. No single points of failure.

In a way, the machine is becoming more self-aware. Using the resources of the computer and the processors to build, monitor, and continually change defenses on that machine that are totally unique to that machine. In this way, almost all of the security holes and vulnerabilities that currently exist in our IT infrastructure can be closed.

Of course, these technique must run quickly and compactly and not require a lot of additional resources. The techniques must scale both up and down the spectrum, allowing for a style of security to be applied to all computational devices from mainframes, to servers, to desktops, to laptop computers, to hand-held devices and cell phones.

Our economy has become too dependent on IT infrastructure to allow security to be handled haphazardly.

It is time for drastic change.

The coming software productivity revolution will be based in the rigorous application of mathematics, not in clever programming tricks, “enterprise” Java or language du jour  ”integrated” development tools.

MIT’s Technology Magazine reported that IT teams spend up to 80% of their budgets removing defects they themselves introduced into the code.¹ Imagine the possible savings if a software product could be produced defect free the very first time. The only way to achieve this is to have a mathematically rigorous process of creating software, a mathematically rigorous process of turning business needs into executable systems.

Although loath to admit it, most software developers will confess that the internals of their software systems have much more in common with a Rube Goldberg cartoon than a mathematical equation. This is unfortunate, for only the rigorous application of mathematics enables the rapid production of error-free software systems.

I’ve seen it done, repeatedly.

The day is coming, burning like a furnace, when traditional development will be chaff; that day will set it ablaze, leaving neither root nor branch.²

I look forward to that day.

-Tom

Automation changes the nature of work. It improves productivity and significantly reduces defects, by reducing opportunities for human error. Automation improves quality, while decreasing costs.

This is true for manufacturing, it is also true for software. New software products can be produced for significantly less money, in dramatically less time, with little or no defects–through extremely aggressive automation.

Automation is the future of software products as sure as it has been the path for all other commodity industries. If you are not actively engaged in discovering how to make your products a part of the software productivity revolution then it is definitely time to begin.

I’ve watched large corporate client after large corporate client offshore software development to reduce costs. May I make a suggestion?

Don’t offshore, automate. The answer to building software products faster, better, and less expensively is not cheap labor. The answer is in eliminating most costly and error prone manual labor altogether.

A study of over 30,000 software development projects reported that two-thirds experience major problems and over one-quarter fail outright. In one recent year alone over 30,000 projects failed wasting over 56 billion investment dollars¹. The rate of failure is so high and the variation so great that the success or failure of any given project is, to most managers, essentially random.

It is not surprising that sponsors are reticent to support software development initiatives. It is not surprising that so many companies are eager to send software projects overseas where at least they diminish the costs of failure.

Currently, market forces are acting on the belief that the future of software development is offshore cheap labor². The emphasis on the single characteristic of unit costs per programmer hour is tragically flawed and outdated.

Cheap labor diminishes costs, but it does not improve productivity or quality.

I am making a bold pronouncement, I say it is possible to eliminate 90% of the programming labor of most projects entirely, and I have the case studies to prove it.

Although cutting unit costs per programmer hour is a reasonable goal, the benefits gained from this approach are insignificant when compared to automating most of the programming and testing tasks and eliminating most manual labor entirely.

If you were going to dig a tunnel from England to France, would you seek to hire 5,000 Indian laborers and arm them with picks and shovels? They are really cheap per day!

No – it is an insane way to dig a tunnel. It’s an insane way to build software. Eventually the industry will wake up. But they haven’t yet. So if you learn the secrets of automation you can be way ahead of your competition.

Cheap labor WAS NOT the most efficient way to build the Chunnel.

Cheap labor IS NOT the most efficient way to build software.

Automation is.

Automation makes the current trends in off shoring software development irrelevant.

This is my notice to the software industry, it is time to seriously raise your game.

As Agilists, we all do standups.  Three questions.  What did you do since the last standup?  What will you do before the next one?  Do you have any impediments?  And maybe a fourth: Do I need to have a parking lot conversation after the standup about anything?

Or at least, we all try to do them.  Oftentimes, we run into difficulties.  Sometimes the scrum master tries to lead the standup, and take reports.  Sometimes the standup turns into a set of conversations.  Sometimes the team drones on.  Sometimes even the scrum master is unable to prevent herself from asking questions of the folks reporting.

Is there anything that can be done about these problems?  Funny you should ask.

In the last few months we’ve seen a few interesting experiments tried in our standups.  The simplest standard experiment in standup is to choose a talking token, and to require that the only person who gets to talk is the one with the talking token.  In general, this is a good practice, and if the standup has any tendency to revert to conversation rather than a short team-to-team reporting, the talking token is a generically good idea.

A step in that direction that went very well for some of our teams over the last year occurred when trying to get remote workers to report /participate during the standup.  We ended up having difficulties getting the polycom system to work so we ended up using a cell-phone, and calling the shared number.  Then, the person reporting would hold the cellphone, and speak into it, allowing the remote worker(s) to hear what was being said.  As a side effect, it was among the most well-obeyed talking tokens I’ve seen.

A different step in that direction came up when one of our most conscientious scrum masters found that she couldn’t prevent herself from asking questions during standup, and also determined that not everyone was following the rules.  Her response was to print up a set of four index cards, each with one of the 3+1 questions on it.  The new rule on that team is: not only can you not talk unless you’re holding the deck of cards, but you also can only talk about what card you’re showing.  It makes standups extremely focused.

Also, because one of our standup’s quality metrics is that we do 4-days a week of correct standup behavior, that same team has Cowboy fridays, in which they do a 90% correct standup, but relax a little bit, and allow themselves to not follow the rules perfectly.

But the last experiment may be the best.  An awful lot of scrum teams have a difficulty with just yammering on during the standup, rather than talking about what tasks they accomplished, and what tasks they will be working on.  The trick we’ve recently tried to address this concern (besides the normal bribery and threats) is to attempt silent standups.

The rules:  Do your standup correctly, but no spoken words (or sign language).  Move tasks, from WIP to complete, from not started to WIP, or update their hours.  Also, write up impediments, and parking lot items into the right locations.  We’ve tried a few.  They are incredibly good at forcing the team back to the task-focused nature of the standups.  Might be the best experiment we’ve tried related to standups.  Also, silent standups end up being really short.  A modification would be to just do Today/Tomorrow silent, and allow impediments and parking lot items to be vocalized.

While I wouldn’t suggest that silent becomes the norm, doing it once every week or two really reminds teams to focus on talking to tasks, skip talking if they have nothing to say, and to put tasks up that are being missed in normal discussion.

What else have you seen work to make standups better?