5 Practices to Make Any Project More Agile

A core foundational concept in Agile is to get feedback from real users early enough to be able to address any issues that arise. In traditional Waterfall, we assume we can do enough work up front that there will be no issues at the end, but reality shows that this almost never happens. Instead, at the very end of the project, we do User Acceptance Testing and discover issues that we either rush to patch or we plan a follow up project to address them.

Even if your project cannot adopt an Agile methodology, there is a better way to do Waterfall that still lets you take advantage of all the experience you have doing Waterfall projects and requires very little change within your organization.

There are 5 practices we strongly recommend because they can be done by any project following any kind of methodology. We have in practice with real projects seen them dramatically improve the results of Waterfall projects, and in fact have found historical evidence that some of these practices were recommended in the past for Waterfall efforts. These 5 practices are:

Divide the work into increments
Plan, estimate, and prioritize each increment
Synchronize work including vendors
Demo each increment
Retrospective each increment

This is what as known as Incremental Waterfall with the addition of the Agile practices of work synchronization, demonstrations of progress, and multiple retrospectives. For Waterfall projects, these practices should not require much change to current processes. Some of the phase gates may need to be passed each increment, such as detailed requirements or design each increment, so some additional time should be allowed for in the schedule. The time is made up later when better quality leads to shorter test cycles and involvement of users means much less (if any) late rework. Big picture work such as overall planning, high level requirements, architecture, and solution selection can be done at the beginning because they are common across all increments and typically do not change much throughout the project lifetime.

In general the approach for a Waterfall project will be a relatively short up front planning, requirements, and architecture phase followed by a set of increments for implementation. Each increment will include the work of detailed planning, requirements, design, implementation, and testing. The complete release process may take place at each increment if desired, or may be delayed until the end of the project.

Each practice is described in more detail below.

Divide the work into increments

An increment is a working subset of the overall solution. This practice would mean examining a project to determine the best way to divide it into 2 or more incremental deliveries of working solution. Note that these deliveries do not have to be general availability, but rather could be collected in a holding area until the whole solution is complete. But each increment will go through a full test cycle including especially user acceptance testing.

The criteria for determining what goes into each increment should include business value (the most valuable work should be done earliest), risk (we should use feedback from real users to drive out risk as early as possible), and the ability to deliver a working subset of the overall solution (minimum viable product). This practice by itself has dramatically improved traditional Waterfall projects by providing feedback well before the end of the project.

In fact, this practice was proposed for Waterfall projects in 1970 by Dr. Winston W. Royce based on his previous long time experience with large Waterfall projects (Managing the Development of Large Software Systems, Proceedings IEEE Wescon August 1970, pages 1-9). His experience and research showed that there would almost always need to be at least a second increment of a Waterfall project to refine the solution based on lessons learned, and so teams should just plan for it. We see the same realization at most of our large customers, where large Waterfall projects have typically been followed by another improvement or fix-it project. We are suggesting to formalize this practice with multiple planned increments of any software project.

Plan, estimate, and prioritize each increment

No matter the length of the increment, the detailed work planning, refining of estimates, and prioritizing of work should happen not long before the start of the increment, instead of doing all of it up front for the whole project. This allows the flexibility to take advantage of lessons learned in one increment to apply to the planning for subsequent increments. It is also a good time to review project metrics to verify the work is going according to plan, and allows a planned time to make adjustments as needed rather than waiting for the end of the project to discover there are issues.

We expect high level planning at the start of the project for all except the most innovative and fast-changing work (which by nature cannot be planned much in advance). This practice is to wait to do detailed planning until just before you need it so that the plans are the most accurate based on the information you have to date. This practice avoids spending time doing detailed work that is likely to be changed later in the project. The less well-understood the work is, the more important this practice becomes.

Synchronize work including vendors

Everyone involved with the project should regularly synchronize their work, including a representative from each vendor. This synchronization can take the form of a daily standup. In some cases, that may be more frequent than needed, but this synchronization should take place at least weekly. Synchronization should be often enough to remove the risk of unnecessary delays due to unknown work blockages.

Note that this is not meant to be a long status meeting, but is rather a 15 minute or less structured ceremony where each person just states what he or she has recently completed, what work he or she is working on now, and anything preventing (blocking) the person from completing his or her work. The most important information to come from this meeting is typically the blockages. The work blockages can be addressed after the synchronization meeting.

It is very important that the vendor representatives be included. A big problem when working with vendors is lack of visibility into what they are working on. Communication can be difficult already when working with another company, so anything we can do to promote regular, meaningful communication will improve (sometimes greatly) the work we do with vendors. Our customers who have worked this way with vendor representatives report a uniformly better experience than just handing work “over the wall” and expecting everything to be fine.

Demo each increment

At least once per increment, the working solution is demonstrated to at least the project sponsor and stakeholders. By far the better choice is to have real end users also attend the demonstration. This demonstration has two purposes: it provides a real deadline when work has to be finished so that it can be demonstrated, and it provides an early opportunity for users to provide feedback on whether or not the solution meets their needs.

Note that this does not replace formal user acceptance testing, but rather is a demonstration of progress and an opportunity for feedback well before the end of a project. Instead of providing status through reports showing perhaps 85% of requirements are completed (for example), we demonstrate working solutions as a measure of progress. If the solution is not complete enough to be demonstrated, we count it as zero percent complete.

We treat solutions as zero complete or 100% percent complete and do not try to determine an increment in between. This is because with any kind of knowledge work, it is not actually possible to truly determine how much is left to do. We can only know how much has been done. We all know of cases where the solution was 90% complete but that last 10% took 90% of the resources. We prevent that in this practice by only counting a 100% solution as complete and anything else as zero.

The demonstration provides a good opportunity to review the progress of the project to get an idea of whether it is on schedule, ahead, or behind. The outcome of the demonstration is that either the work to date is acceptable as is, or defects or change requests are created to fix problems that are detected at the demonstration. The work to fix the defects or change requests can be planned into subsequent increments of the project so that these problems do not exist in the released version of the product.

Retrospective each increment

Most companies have some kind of retrospective or lessons learned at the end of a project. Unfortunately, that is too late to do any kind of good for that team. Instead, at the end of each increment, the team will review the past increment, and in a non-judgmental way discuss what went really well, what did not go so well, and what they will do in the next increment to improve as a team.

We suggest the team start the retrospective by reciting a statement such as “We acknowledge that everyone has done the best job they were able to do in the circumstances that existed during this increment” and suggest the team end the retrospective with expressions of gratitude and thanks to other members of the team or even people outside the team who helped in some special way (it is nice to let those people know in person that they did a great job if they do not happen to be at the retrospective). This helps set a non-judgmental tone to the ceremony. The most important thing is to avoid blaming and finger pointing, to make the ceremony constructive.

The outcome of the retrospective is a small set of specific action items the team will take to improve the coming increment. We recommend starting with just 1 or 2 action items so as to not overwhelm the team with improvement work while they are focused on delivering a product. Often the outcome of the retrospective are suggestions that become habits over time, as the team incorporates better and more efficient ways of working together.

“I need a name for something,” I told Kyle, as we slid into our chairs inside of The Big Bib.
Kyle had discovered this little gem his first week in San Antonio.

Shortly after Kyle started coaching he executed one of the better team building moves I’ve ever seen. He treated his entire team to a free Barbecue lunch (paid for out of his own pocket). Of course, since to Kyle any group is an audience, he used the time they were eating to talk about the fundamental differences between Waterfall and Agile.

I, of course, wanting great barbecue, crashed the lecture–hey, free barbecue. I enjoyed the food so much we scheduled the restaurant as a coaches dinner destination. The talk was good too.

I had no idea it was literally a hole in the wall. About eight of us took up over three quarters of their chairs.

“What is the problem?” Kyle asked in response to my statement about needing a name.

“You know how we radiate stories, tasks, and status on a task board with the team seated nearby?” I continued, between bites of potato salad.

“Yes.” he said, devouring a rib like a starving crocodile.

“Well, even when the team isn’t doing their daily stand-up meetings they still see those items posted.” I noted, “They are continually reminded on what is being worked on, what they need to work on, and what the progress is on the project.”

“Go on.” Kyle stated, now cutting into a brisket.

“I calculate,” I continued, knowing of course that Kyle would immediately begin checking my math, “that the team is exposed to the information on the walls literally hundreds of thousands of times during the life of a typical project.”

I could see his eyes dart for a moment. I waited for him to finish the calculation. Kyle says he calculates using clocks. I have no idea what that means. However, it seems to be fast and accurate, so I don’t ask any questions.

“OK,” he concludes a second or two later, already having completed his own estimate and finding it close enough to mine.

“Every time they look up, walk by, or see someone updating the task board they are potentially engaging in closing feedback loops.” I continue. “What if only a tiny fraction of the time they actually act on their observations.”

“They are closing hundreds of loops.” Kyle added. “Without even thinking about it.”

“It motivates them to move more quickly when they see others mark their tasks as done.” I declare excitedly. “It prompts them to ask a question about something they don’t understand.”

Geri, who as also been tracking the conversation chimes in, “It also lets a person know they need to pitch in, just by observing a task isn’t completed yet.”

“Absolutely” I continue, “And this goes on all the time!”

“So what is the question?” Kyle says, moving the conversation back to its original intent, and taking a sip of a large beverage I would swear was 10 percent tea and 90 percent cream.

“My question is, what do we want to call this type of feedback?” I said, gently pounding the table. “I want to lecture on it but it needs a name.”

Kyle sat silently for a moment,

“I think I know,” he said.

“Although the words are not perfect, they may do the trick. And there are two words that we need.”

I smile.

“Peripheral and visceral.” Kyle states, matter-of-factly, licking sauce off of his fingers, “Visceral is probably the wrong word, but it is the best I can think of for now.”

My next presentation now had a title, “Peripheral and Visceral Feedback.”

My goal, to explain the subtle psychology behind agile that most software professionals miss. To explain it in terms of feedback loops operating all of the time in the background, if we do agile correctly.

New teams always try to immediately disable all of these feedback loops. “Lets not put story cards and tasks on the wall,” they’ll say, “lets put’em in this fancy new fangled dater base. The people who sold it to me said it is really good.”

In our experience this is always a bad idea. But, it has been very difficult to explain why without sounding like a mystic. The job I assigned myself was to explain why, and to do it in terms that people could actually understand. I instantly suspected the words “Peripheral and Visceral” would help, a lot.

In the end we would write the presentation as a team and call it “Peripheral, Physical, and Visceral Feedback.”

Kyle, of course, gave it to anyone walking too close to his desk.

I think, maybe, we are moving agile from mysticism to pragmatism, with a little help from feedback loops.

Tag: feedback

How Your Waterfall Project Can Be More Agile