Category: Uncategorized

The coming software productivity revolution will be based in the rigorous application of mathematics, not in clever programming tricks, “enterprise” Java or language du jour  “integrated” development tools.

MIT’s Technology Magazine reported that IT teams spend up to 80% of their budgets removing defects they themselves introduced into the code.¹ Imagine the possible savings if a software product could be produced defect free the very first time. The only way to achieve this is to have a mathematically rigorous process of creating software, a mathematically rigorous process of turning business needs into executable systems.

Although loath to admit it, most software developers will confess that the internals of their software systems have much more in common with a Rube Goldberg cartoon than a mathematical equation. This is unfortunate, for only the rigorous application of mathematics enables the rapid production of error-free software systems.

I’ve seen it done, repeatedly.

The day is coming, burning like a furnace, when traditional development will be chaff; that day will set it ablaze, leaving neither root nor branch.²

I look forward to that day.

-Tom

Automation changes the nature of work. It improves productivity and significantly reduces defects, by reducing opportunities for human error. Automation improves quality, while decreasing costs.

This is true for manufacturing, it is also true for software. New software products can be produced for significantly less money, in dramatically less time, with little or no defects–through extremely aggressive automation.

Automation is the future of software products as sure as it has been the path for all other commodity industries. If you are not actively engaged in discovering how to make your products a part of the software productivity revolution then it is definitely time to begin.

I’ve watched large corporate client after large corporate client offshore software development to reduce costs. May I make a suggestion?

Don’t offshore, automate. The answer to building software products faster, better, and less expensively is not cheap labor. The answer is in eliminating most costly and error prone manual labor altogether.

A study of over 30,000 software development projects reported that two-thirds experience major problems and over one-quarter fail outright. In one recent year alone over 30,000 projects failed wasting over 56 billion investment dollars¹. The rate of failure is so high and the variation so great that the success or failure of any given project is, to most managers, essentially random.

It is not surprising that sponsors are reticent to support software development initiatives. It is not surprising that so many companies are eager to send software projects overseas where at least they diminish the costs of failure.

Currently, market forces are acting on the belief that the future of software development is offshore cheap labor². The emphasis on the single characteristic of unit costs per programmer hour is tragically flawed and outdated.

Cheap labor diminishes costs, but it does not improve productivity or quality.

I am making a bold pronouncement, I say it is possible to eliminate 90% of the programming labor of most projects entirely, and I have the case studies to prove it.

Although cutting unit costs per programmer hour is a reasonable goal, the benefits gained from this approach are insignificant when compared to automating most of the programming and testing tasks and eliminating most manual labor entirely.

If you were going to dig a tunnel from England to France, would you seek to hire 5,000 Indian laborers and arm them with picks and shovels? They are really cheap per day!

No – it is an insane way to dig a tunnel. It’s an insane way to build software. Eventually the industry will wake up. But they haven’t yet. So if you learn the secrets of automation you can be way ahead of your competition.

Cheap labor WAS NOT the most efficient way to build the Chunnel.

Cheap labor IS NOT the most efficient way to build software.

Automation is.

Automation makes the current trends in off shoring software development irrelevant.

This is my notice to the software industry, it is time to seriously raise your game.

As Agilists, we all do standups.  Three questions.  What did you do since the last standup?  What will you do before the next one?  Do you have any impediments?  And maybe a fourth: Do I need to have a parking lot conversation after the standup about anything?

Or at least, we all try to do them.  Oftentimes, we run into difficulties.  Sometimes the scrum master tries to lead the standup, and take reports.  Sometimes the standup turns into a set of conversations.  Sometimes the team drones on.  Sometimes even the scrum master is unable to prevent herself from asking questions of the folks reporting.

Is there anything that can be done about these problems?  Funny you should ask.

In the last few months we’ve seen a few interesting experiments tried in our standups.  The simplest standard experiment in standup is to choose a talking token, and to require that the only person who gets to talk is the one with the talking token.  In general, this is a good practice, and if the standup has any tendency to revert to conversation rather than a short team-to-team reporting, the talking token is a generically good idea.

A step in that direction that went very well for some of our teams over the last year occurred when trying to get remote workers to report /participate during the standup.  We ended up having difficulties getting the polycom system to work so we ended up using a cell-phone, and calling the shared number.  Then, the person reporting would hold the cellphone, and speak into it, allowing the remote worker(s) to hear what was being said.  As a side effect, it was among the most well-obeyed talking tokens I’ve seen.

A different step in that direction came up when one of our most conscientious scrum masters found that she couldn’t prevent herself from asking questions during standup, and also determined that not everyone was following the rules.  Her response was to print up a set of four index cards, each with one of the 3+1 questions on it.  The new rule on that team is: not only can you not talk unless you’re holding the deck of cards, but you also can only talk about what card you’re showing.  It makes standups extremely focused.

Also, because one of our standup’s quality metrics is that we do 4-days a week of correct standup behavior, that same team has Cowboy fridays, in which they do a 90% correct standup, but relax a little bit, and allow themselves to not follow the rules perfectly.

But the last experiment may be the best.  An awful lot of scrum teams have a difficulty with just yammering on during the standup, rather than talking about what tasks they accomplished, and what tasks they will be working on.  The trick we’ve recently tried to address this concern (besides the normal bribery and threats) is to attempt silent standups.

The rules:  Do your standup correctly, but no spoken words (or sign language).  Move tasks, from WIP to complete, from not started to WIP, or update their hours.  Also, write up impediments, and parking lot items into the right locations.  We’ve tried a few.  They are incredibly good at forcing the team back to the task-focused nature of the standups.  Might be the best experiment we’ve tried related to standups.  Also, silent standups end up being really short.  A modification would be to just do Today/Tomorrow silent, and allow impediments and parking lot items to be vocalized.

While I wouldn’t suggest that silent becomes the norm, doing it once every week or two really reminds teams to focus on talking to tasks, skip talking if they have nothing to say, and to put tasks up that are being missed in normal discussion.

What else have you seen work to make standups better?

It is an easy trap to fall into, a project somewhere in the company has a few struggles or even outright fails and management sends in a team to “fix the process.”

After studying the failure, the team suggests a new document or checklist to add so that other teams will not have this problem in the future too.

Problem solved.

Unfortunately, this solution likely simply exacerbates the problem. The solution was yet another checklist, yet another way to remind people there is something they need to be thinking about, yet another form to fill out so the team is reminded ”do not forget this important thing:.

The Agilist understands there is another way, a better way, to help teams be successful.

First, keep the tribe together dedicated and co-located so important knowledge about the system is maintained in the group automatically over time. Keeping the tribe together eliminates the need for a lot of documents and checklists.

Second, pair everybody. Ensure key knowledge is spread across the entire tribe by pairing and eliminating specialists, so everybody has an opportunity to be exposed to core issues.

Third, if it is so important you are inclined to put it on a check list to make the team constantly review it then AUTOMATE the test for it. If you are worried you need to remember something then eliminate the worry by automating the test for it.

UNTIL YOU HAVE DONE ALL THREE OF THE ABOUT DON’T CREATE NEW CHECKLISTS!

There is a really important reason why, care to guess what it is?

An extremely popular approach to business, almost iconic in the United States, is one we’ll call Hero. Hero is what most people actually are doing when they say they are being Agile. They sometimes look alike, but are fundamentally different.

Companies who have a Hero approach and who think they are doing Agile–will truly struggle to be Agile.

So what is Hero?

To illustrate Hero we use the 1971 Clint Eastwood motion picture, Dirty Harry.

In the movie, a killer threatens to randomly kill a person each day unless the city of San Francisco pays him a ransom.To lead the investigation the Chief of Police and the Mayor of San Francisco assign “Dirty” Harry Calaghan the task.

Why Dirty Harry?

Because he gets things done… he is a hero you can rely on. He doesn’t care about rights, doesn’t follow rules, breaks laws… in order to get the job done. A more recent example of this type of character in popular fiction is terrorist fighter “Jack Bauer” in the TV series 24.

The Hero Approach

The Hero is a maverick. Independent. Confident, yet highly skilled. He is driven by his own internal moral compass and doesn’t let rules get in his way.

Americans like Heroes, and our biggest grossing films almost always feature “Super Heroes.” A business philosophy built around heroes says we make our own rules to get the work done. We do not constrain our teams with bureaucracy and paperwork, but leave them to themselves to discover the best way to succeed.

Laws don’t really apply to super heroes. Or to software heroes.

It is a compelling way to work, and some of the most interesting American companies and products were birthed in a heroic fashion. Very significant companies such as Apple, Google, and Facebook followed a hero launch pattern.

What does Hero look like in business?

A small group of people working together in a small room or garage dedicated to a specific task. Apple and Google started in garages, Facebook in a dorm room.

Smart, dedicated, passionate people working closely on a project they love. Send pizza and Mt. Dew into the room and hope something great comes out.

Large businesses actually ask for this type of thing all the time: “Give me a war room and get out of my way.” The executives are known to say, when something they think is really important needs to be done.

Of course, the executives also steal the best people from every other project to put in their “war” room. Heroic methods require, no, demand heroic staff.

However, hero has some down side. Care to guess what they may be?

An attribute of Agile software development I have been pondering lately is uncertainty.

We might argue that uncertainty is just one of many sources of friction, but because
it is such a pervasive trait of software development, I will treat it singly.

All actions in our software development life cycle take place in an atmosphere of uncertainty. Uncertainty pervades our operations in the form of unknowns about the competition, about the environment, and even about our own business. While we try to reduce these unknowns by gathering information, we must realize that we cannot eliminate them—or even come close. The very nature of business makes certainty impossible; all actions in business will be based on incomplete, inaccurate, or even contradictory information.

Business is intrinsically unpredictable. At best, we can hope to determine possibilities and probabilities. This implies a certain standard of executive judgment:

What is possible and what is not?

What is probable and what is not?

By judging probability,we make an estimate of our competitor’s designs and act accordingly. Having said this, we realize that it is precisely those actions that seem improbable that often have the greatest impact on our outcomes.

Because we can never eliminate uncertainty, we must learn to act effectively despite it. We can do this by developing simple, flexible plans; planning for likely contingencies; developing standing operating procedures; and fostering initiative
among subordinates.

One important source of uncertainty is a property known as nonlinearity. Here the term describes systems in which causes and effects are disproportionate. Minor incidents or actions can have decisive effects. Outcomes of battles can hinge on the actions of a few individuals, and as Clausewitz observed, “issues can be decided by chances and incidents so minute as to figure in histories simply as anecdotes.”

By its nature, uncertainty invariably involves the estimation and acceptance of risk. Risk is inherent in business and is involved in every project. Risk is equally common to action and inaction. Risk may be related to gain; greater potential gain often requires greater risk. The practice of concentrating business resources toward the main effort necessitates the willingness to accept prudent risk elsewhere. However, we should clearly understand that the acceptance of risk does not equate to the imprudent willingness to gamble the entire likelihood of success on a single improbable event.

Part of uncertainty is the ungovernable element of chance.Chance is a universal characteristic of business and a continuous source of friction. Chance consists of turns of events that cannot reasonably be foreseen and over which we and our competitors
have no control.

The constant potential for chance to influence outcomes in our business initiatives, combined with the inability to prevent chance from impacting on plans and actions, creates psychological friction. However, we should remember that chance favors no one exclusively. Consequently, we must view chance not only as a threat but also as an opportunity which we must be ever ready to exploit.

(Note: This is an exercise in rewriting existing text created for another purpose. Any guess as to the source material for this post?)

Modern Control Systems – The Secret to Understanding Agile
We make a simple promise to you, read and understand this overview of modern control theory and you will understand how agile software development works, in a profound way. You will understand it better than many people who lecture on Agile techniques at conferences.

Open Loop Systems
We begin with a simple description of open loop systems. An open loop system is a modern control system without feedback. “What is that?” you ask. Fortunately, it is easy to understand, it is as easy as making toast.

A modern control systems can be described in three simple parts:

 Part 1: Desired Output

You begin  with an idea of a specific output you desire to create. If you are creating a piece of toast then you being with an image of what a perfect piece of toast may look like to you.

Having the idea of the desired output is the first part in modern control theory. You need some idea of what you want so you can discover a way to create it. That leads us to:

Part 2: Process

After you have a desired output you create a process to give you that desired output. If we want to make a process to give us the perfect piece of toast it will likely contain the following steps:

1. Acquire bread, remove from package.
2. Put bread into toaster.
3. Set desired darkness.
4. Press down control lever lowering toast into toaster and engaging heating elements.
5. Wait until toast pops up.
6. Remove toast.

You implement this process to create your desired output, having the desired output in mind helped us describe this process.

[Desired Output] →[Process]

Simple, even a child can do it. In fact, making toast is one of those childhood delights.

That leads us to:

Part 3: Desired Output 

At the end of the toasting cycle we removed the perfect piece of toast and enjoy. Our entire process looks like the following:

[Desired Output] →[Process]→[Actual Output]

In modern control systems we call this an “Open Loop” control. It is perhaps better described as a “No Loop” control. No loops because there is no feedback in this system. You set the controls, wait, and whatever you get is whatever you get. Hopefully, the actual output is close to the desired output and you have the perfect piece of toast.

Burnt Toast

Sometimes, however, the actual output looks more like this:

What happens if you accidentally create a piece of toast that looks like this?

Well, you can try to do additional processing and scrape the burnt part off in the sink. But that doesn’t create a very compelling piece of toast. The typical solution is to toss the toast into the trash and start the whole process all over again, setting the darkness knob on the toaster to a slightly lighter level.

The process we outlined for making toast contains no feedback (no loops). Step 1, led to step 2, led to step 3,… and so on.

1. Acquire bread, remove from package
2. Put bread into toaster
3. Set desired darkness control
4. Press down control lever lowering toast into toaster and engaging heating elements
5. Wait until toast pops up
6. Remove toast

Closes Loop Systems

Nowhere in the previous system do we use information while the toast is toasting to understand how well our process is working. What might it look like if we try to gather information while our toast is toasting?

“Why do I care?”  Andy asked, interrupting about 45 seconds into my Toast presentation.

“What?”

“Why do I care?” Andy asked again. “You haven’t told me why I should care enough to listen to this presentation.”

Andy is one of our Agile coaches.A maverick entrepreneur who has made, lost, and made again several small fortunes. He is used to being boss, and is usually about as tactful as a she bear whose cubs are being tortured.

Still, he was right. I hadn’t stated why someone should care. I hadn’t provided a hook, a reason to listen, a promise of an interesting story or a guarantee of a result. I hadn’t given my students a reason to actually listen closely, and for this presentation I really needed them to listen.

I added a new slide at the beginning of the Toast deck.  A picture of a giant set of keys.

The slide appears and I say, “In the next 20 minutes I’ll explain why and how agile works so clearly that you will understand it better than most people who speak at agile conferences. Listen for 20 minutes, and you will understand exactly how agile works and exactly how it differs from agile–guaranteed.”

A powerful promise. But we deliver. An nobody has ever tried to collect the guarantee, which is good, since I can’t really give them their 20 minutes back.

Still, I tell people clearly why they should care.

Today, we showed Andy a poster.

“Why do I care?” Andy said, reading the poster.

Sigh.

Dang-it, I had no idea.

When are you ever going to get this right?” He asked.

An alternative loopian dicationary.

loop: A completed feedback cycle.
loops: Plural of Loop. The name of the process.
looped: Past tense of Loop. The process of having feedback provided to you.

loopm : Someone providing manual feedback loops (loop em).
loopa: An automated process/daemon providing automated feedback (loop uhh).

Prefixes:

• s- loops that operate on a second by second basis.
• m- loops that operate on a minute by minute basis
• h- loops that operate on an hour by hour basis
• d- loops that operate on a day by day basis

Putting these definitions together we could get:

s-loopm:  Someone providing feedback on a second by second basis, usually through pairing.

s-loopa: Something providing feedback on a second by second basis.

Here, rewritten, are the loops listed in a previous post:

1. As a loopm,I want my design ideas sloopm so I do not get stuck and waste time.
2. As a loopm, I want my variable names sloopm for clarity so the code is easier to maintain the very first time it is checked in.
3. As a loopm, I want my class/function names sloopm for clarity so the code is easier to maintain the very first time it is checked in.
4. As a loopm, I want my each line of code sloopm for clarity so the code is easier to maintain the very first time it is checked in.
5. As a loopm, I want each line of code sloopm for logic errors to reduce errors in the code the very first time it is checked in.

Writing a few more loops this way:

1. As a loopm, I want my token of code sloopa as I  type for syntax errors so no syntax errors are introduced as I type.
2. As a loopm, I want each function of code I am editing sloopa as I type for unit correctness so I do not have to wait to run all the unit tests to see that this code is accurate. **
3. As a loopm, I want all units dependent on the code I am editing mloopa for unit correctness so I know if I am breaking code in dependent systems before I check the code into production.