Agile Principles: How to Maintain a Sustainable Pace

larry apke, agile, scrum, indiegogo, agile development, agile doctor

larry apke, agile, scrum, indiegogo, agile development, agile doctor

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

When I think on this principle I cannot help but think about the potential “dark side” of agile and how it can be misunderstood and implemented incorrectly. It also reminds me of an interesting story I was told by one of my coaching colleagues recently.

Once upon a time a company hired a very talented vice president of software development. Unfortunately, when this brave soul entered employment the amount of technical debt in the code was enormous. This was a situation that needed to be fixed because this pasta code was very expensive to maintain and made it difficult to deliver software quickly and with quality.

The company’s leadership heard about agile and decided that this was the answer to all their problems so they set about sprinting. Since the concepts are so easy they felt they could forge ahead without expert agile scrum help. In their quest for agility they found that they could indeed write code faster, but without proper guidance they forgot about the concept of sustainability and did nothing more than create technical debt faster. Unfortunately for our VP, the pleas to adopt sustainable agility went unheeded and six months was all the VP could take before moving on.

The bottom line is that many companies misuse agile because they think by being agile they can cheat the iron triangle of development. What too few people realize is that you don’t choose two of three sides because it is actually an iron square where you choose three of four sides (scope, resources, schedule and quality or, as Jeff Atwood refers to it, an Iron Stool). You misuse agile when you choose everything but quality because the code becomes unmaintainable over time and agility becomes mired in the big ball of mud you have created.  I refer you to my article about refactoring in the Agile Record for the problems with unnecessarily complex and technical debt laden code.

The misguided desire to emphasize speed over quality leads to the accumulation of technical debt and is a symptom of project (and not product) centric thinking. Like I refer to in an earlier blog post, there are reasons why no one washes a rental car.  Overtime you will no longer get speed or quality and your ability to sustain agile over long periods of time is compromised.

I try to run at least a few miles everyday, but I do not sprint the entire run. If I did, I would barely make it more than about a quarter of a mile. This is why I have begun to prefer the term iteration over sprint. Sprinting goes against this principle because sprinting is, almost by definition, unsustainable. It certainly is not “constant pace  indefinitely”.

I argue that in order to maintain constant pace indefinitely there are two things an team must do and an organization must support, acceptance test driven development (ATDD) and continuous integration / delivery (CI/CD). I believe currently that BDD is the best means of accessing ATDD (and TDD) so I have taught that to my clients with spectacular success.

Without ATDD and CI/CD all teams are doing is what I call feature chasing. The question is not one of sustainability but how many and how quickly can I deliver new features. While this might be important for startups, most are not involved in such high competition that chasing features at the expense of quality and long term sustainability is ludicrous. Even those who must feature chase to remain competitive must recognize that they are creating technical debt that must be paid, and paid quickly, before servicing the “interest” on the debt is all that can be afforded.

Interestingly enough, though many people believe that employing ATDD, TDD and CI/CD slows the progress of software delivery, my experience is that, with very little training and a healthy dose of discipline, the gains far outweigh the investment. This is obvious if we look at the product and not just the project, but my experience shows that even within the misguided and arbitrary project the payoff is realized.

I have a number of teams that I have coached that delivered high quality software into production in short project time frames precisely because, and not in spite of, BDD. As Bob Martin states, “The only way to go fast is to go well,” and no one is more recognized as an expert on quality code than him.

My last point is related to the above in that one of the greatest dangers of feature chasing is not just that we tend to accumulate technical debt faster, but it (and the sprinting as fast as we can mentality) generally pressures us to not take advantage of training opportunities like learning TDD, BDD and the like. With technology changing so quickly it is critical that our people make certain to invest their time not just chasing features but building the skills necessary for sustainable development so they can maintain a constant pace indefinitely.

Larry Apke

Agile Values: Why Contracts And Software Development Don’t Mix

larry apke agile doctor

customer collaboration

“Customer collaboration over contract negotiation”

We all have customers. If we didn’t there would be no reason to do what we do. If we didn’t their would be no one to pay our invoices. And when someone agrees to pay you for work, they generally want to have some kind of agreement on the nature of the work for the money that is being paid. This agreement is usually put in writing and voila, we have a contract. This is an important part of the process and as everyone knows, contracts are valuable documents for both the customer and yourself. But as the Manifesto states, it’s important to not get caught up in negotiation fever.

While it is a good thing to have an agreement before work begins, there are a number of unfortunate aspects to even having such an agreement.

The main problem is complexity.

Software development, by its very nature, is a complex endeavor, dependent on communication and creativity to succeed. While it would be wonderful if all our customers were omniscient, they are often far from it. If we are building a house, we could easily choose things like materials and the architecture of the project and expect that the final building will resemble what was originally agreed upon. Such a thing is rarely true in something as complex as software development, but it hasn’t stopped people from trying (even today).

There is a field of study called complexity theory that applies well to software development in my opinion. In his article, ”On Understanding Software Agility—A Social Complexity Point Of View”, Joseph Pelrine writes:

Many people still regard building software as a complicated undertaking, but in fact it is a defining example of a complex or a ‘wicked’ problem. The concept of wicked problems was originally proposed by Horst Rittel and Marcus Webber. Wicked problems have incomplete, contradictory, and changing requirements; and solutions to them are often difficult to recognize as such, because of complex interdependencies. Rittel and Webber stated that while attempting to solve a wicked problem, the solution of one of its aspects may reveal or create other, even more complex problems. Rittel expounded on the nature of ill—defined design and planning problems, which he termed ‘wicked’ (that is, messy, circular, aggressive) to contrast against the relatively ‘tame’ problems of mathematics, chess, or puzzle solving.

If the nature of software development is indeed ”wicked” then trying to agree on all the requirements at the outset in contractual form is not only wasteful but counterproductive. A much better way is to form an understanding with broad brush strokes and work together to attack the problems, hence the need for collaboration over contract negotiation.

The same is true when the customer is internal. This is why Scrum has someone called the product owner who is available daily to the team to work on the software. Instead of assuming we know everything at the beginning and then have mind—numbing change reviews and contract addenda, is it not better to forge a lasting partnership? Or as Deming calls it, “a long-term relationship of loyalty and trust”.

Larry Apke

Be sure to follow me on Twitter and Quora! 

Understanding The Agile Manifesto: A Brief & Bold Guide to Agile – Podcast Now Available

Understanding The Agile Manifesto: A Brief & Bold Guide to Agile by [Apke, Larry]Those of you who may follow this blog may have noticed it has been some time since the last post. It was an eventful summer and it was good to spend some time to catch my breath. Now that the kids are heading back to school, it is time for me to renew my posting. In addition to posting the podcasts, please check back for new Agile posts.

I am pleased to announce that my first book, Understanding the Agile Manifesto: A Brief & Bold Guide to Agile is now available as a podcast. Over the next few days / weeks I will be releasing the book, chapter by chapter, on this website and through the iTunes store under my “Agile Doctor” podcast. Since the book was a series of blogs, the podcasts will also be made available under the original posts.

This project is very dear to my heart as the podcasts were recorded by my son, Vadim Kim, who assisted me over the summer as my intern. I think he did a fantastic job of recording my “little” Agile book and I am sure you will enjoy.

The most recent version of the book includes an Introduction which was never part of any individual post. I have included the podcast of the Introduction below.

 

The Five Attributes of a Good Scrum Team

One of the best experiences I can have as an Agile Coach is presenting Agile and Scrum to people with little or no experience. It grounds me and after many years of living Agile, keeps me from falling into the Curse of Knowledge cognitive bias. Team01smallBetween these presentations and my daily coaching practice, I am reminded yet again at the importance of forming a good scrum team. This, along with proper backlog compilation and maintenance, can make the difference between success and endless frustration. In my experience, proper scrum team formation is the area where companies who are unsuccessful in Agile transformations fail most often.

Over the years, with the help of my teams and the many knowledgeable colleagues that I have worked with, I have distilled everything I have learned and come up with five attributes of a good scrum team. I use it much like a mantra. Those attributes are: Small, Co-Located, Dedicated, Stable and Cross-Functional. That is not to say that a good scrum team will not have other elements or that you can’t witness improvement without all five, but like the sculptor who is only done when there is nothing left to take away, these five represent an ideal essence of a good team. None of these five things can be removed without real consequences to the success of the team.

Small

I write about size at length (and mention the five attributes) in a previous post, When it Comes to Software Development – Size Matters, so I encourage you to read it for some eye-opening account of a major study that confirms this fact. Suffice it to say that Software Development is about communication and collaboration. It is complex knowledge work. In a physical, manufacturing model, adding a person to the effort would result in the expected proportional increase in productivity. In software development, however, adding an additional person merely exponentially increases the number of communication channels whereby any expected increase in productivity is quickly overtaken by the number of communication channels created. There is a reason Agilists recommend small teams. Basically, you can save money because your cost per unit of productive (and quality) work decreases.

Co-Located

Co-Location seems to be an issue with all of the companies that I consult for. We know that the best software developers can be 10 times or more productive than the least. We know that face-to-face communication is multiple times better than any of the substitutes we employ when that mode is not available. We all laugh at the Youco-location01smallTube video “A Conference Call in Real Life”. We know that the unacknowledged creation of technical debt will cause us financial havoc for years to come. Not convinced, I encourage you to check out my blog on the High Cost of Low Cost Software Development. And yet we still continue to source our software development to low bidders around the globe. Here’s the really interesting thing. For many years I have asked a simple question – does it work? I get two answers. The majority shake their head, laugh, and provide a solid “No”. The optimists of the group say that “We have found a way to make it work.” I also see a great deal of blogs that are titled “How We Found a Way to Make Offshore Work”, but it takes little research skills to uncover the author as someone employed by a company that works offshore. Never is there an enthusiastic “Yes.” Most would bring the work back if they could, but it appears that there is a mysterious formation in many software development organizations that continues to think they are doing the right thing by piece-mealing work to various locations. I can only guess it is accounting, but I have yet to meet anyone whom I respect who actually understands software development who would recommend it.

Dedicated

I often rail against the project-centric focus of many software development organizations. For more information, I encourage you to check out my blog “Why Project Focused Mentality is Killing Software Development”dedication. The problem is that in projects we build teams around projects so that one, especially a key employee, will be spread over multiple concurrent projects. There are a number of very serious negative effects to doing this. First, context shifting means that every time a developer must change their concentrated area of focus they will lose 15 minutes minimum. If I am working on two different projects concurrently, there is not a huge loss, assuming that I might work on one in the morning and one after lunch, but many developers are bounced from thing to thing like so many pinballs with context shifting throughout the day. Hours each day are wasted and no project gets the attention it deserves. A better way is to create a small, co-located scrum team around a product backlog and bring the work of these multiple projects to the team via their backlog. The ability for individuals on the team to not fall prey to context shifting will allow them to focus on their work, even at times leading to a state called Flow in which developers (and athletes, artists, etc) are at their peak performance. For those interested, I refer you to the work of Mihaly Csikszentmihalyi.

Stable

 Project-centric thinking also results in teams that are short-lived. We put the team together for a particular period (sometimes relatively short) of time. When the project is done, the team is disbanded. This causes a number of issues. First, every team goes through a period of “storming, norming, and performing.” This is easy to see visually when you plot the velocitiScaleStability01es of multiple teams on a timeline. You can visually see the storming period as teams learn to work together as a team instead of a collection of individuals, through very low (if non-existent) velocity. In my experience, I have come to the conclusion that I expect nearly nothing from a brand new team for the first couple of iterations. It is not until they reach iterations 3, 4 or even 5 that they will reach what is their performing velocity. Therefore, every time I break up a team, I have a very high overhead associated with the team’s “storming and norming.” Another benefit of keeping the same people on teams for a longer period of time is by simply learning how to better communicate with each other over time. Since software development is about communication and collaboration, the more time we spend together, the more effective we become in communicating. I remember reading about the notion of “hyper-productive” teams from Scott Downey and Jeff Sutherland.  The interesting thing is that their “hyper-performance” did not manifest until after the team had been together for some time.

Cross Functional

 In some respects, cross functionality of people is a by-product of the fact that team size should be kept small. If there are 15 things I need to do to get my code into production, I obviously will not be able to do so if there crossfunctional01needs to be an expert for each piece. The creation of “silos” is great if your goal is to create a MDD (mortgage driven development) environment. It is bad for the flow of the product. Silos create queues. Queues, if you want to deliver something quicker, are bad. I highly recommend the book Principles of Product Development Flow by Don Reinersten for a deeper understanding of queuing theory and the damage that out-of-control queues can have. This book talks about the concept of generalizing specialists. In my experience, developers are more motivated by status and mastery of new technology and ideas than any other group of people I have worked with. Creating an environment where developers are expected to know more than a single specialized domain is not only good for moving the software through the system quicker, but it also leads to greater developer engagement. A team should be responsible for all aspects of the software product and the development cycle. This cannot be accomplished without having team members who are cross functional.

Final Thoughts

 An old African proverb states, “If you want to go fast, go alone. If you want to go far, go together.” In order to be successful in software development we need to understand teams and team dynamics. My many years of experience working with what I would estimate around a hundred development teams has led me to conclude that there are five basic attributes for an optimal development team – small, co-located, dedicated, stable and cross functional. I hope that these five attributes help you as you continue your quest for agility.

 

Maybe It’s Time to Stop Using the Word “Sprint”

sprinting

Any agilest worth their salt will tell you that culture matters, especially when you are trying to transition to an Agile methodology. Since culture is merely a reflection of an organization’s values and values come from ideas and words are how ideas are expressed, the words we use can often be critical. There is one word used Agile and Scrum that has bothered me for some time, “sprint”, and I propose eliminating the word from Scrum. In fact, I hope Ken Schwaber and Jeff Sutherland agree to incorporate this change in the next version of The Scrum Guide.

rugby scrumThe Scrum Guide states, “the heart of Scrum is a Sprint”. This is one reason why I haven’t written this blog sooner. I believe in Scrum as an excellent way for teams to embody the Agile philosophy so I am hesitant to criticize. Who am I to criticize and will my criticism be misinterpreted? It took a long time for me to have the confidence to propose something that others may view as heretical (I know how strident some can be about Agile and Scrum), but I truly believe it is the right thing to do.

I acknowledge that I might not be the first to propose such a change. In fact, I had one gig where the company decided to refer to “sprints” as “iterations”. They insisted I do the same and I found it difficult to make the change. I thought this was just another example of “scrum but” (which in this case may have been true), but even though I do so begrudgingly, I now believe this company was on to something worthwhile. I also acknowledge that my opinion might not carry the requisite weight, but I feel it is necessary to add whatever weight it might hold to this effort.

sprintingAnd why would I propose something as radical as changing the very word referred to as the heartbeat of scrum? Simply this. The word itself carries a connotation that I find at odds with the values and principles of the Agile Manifesto. Therefore, I need to make this proposal in order to preserve my integrity. As a coach I often say, “The philosophy behind Agile provides a basis for making decisions when there is doubt on which option to pursue. The Scrum framework contains ceremonies that make the philosophy come alive in our daily practice. If you cannot trace something you are doing to the original philosophy then you probably should not be doing it.” It is for this reason that I no longer can use the word “sprint” and instead choose to use the word “iteration.” Of course, until the Scrum Guide is changed to reflect the more accurate term, I often say “iteration or sprint”.

There is an Agile principle which states, “Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely”. It has become obvious to me, the word “sprint”, which, although it may have a specific meaning in scrum, generally is understood to mean “an act or short spell of running at full speed.” I jog to stay in shape. As I have stated previously it is obvious that sprinting is something that cannot be sustained over a long period of time. Therefore, I prefer the word “iterating” and hope that someday the Scrum Guide, as the official word of what constitutes Scrum, will agree with me and change the word.

Cooks, Chefs and Agile Scaling Models

Chef Photo

Chef Photo     A few years back my doctor gave me some somber news. He told me my cholesterol was too high and I must get it lower. It was then that I decided I would make changes to my less than stellar diet. At the time, we had a large family in the house – seven including my father and mother-in-law. My mother-in-law had become the default cook for the family and, while she made some very tasty dishes, it was not exactly healthy fare. I knew that if I were to reach my goal of a better diet I would have to become the new cook for the family.

Chef PhotoOriginally I thought that my mother-in-law would be upset if I banished her from the kitchen, but if she was, she did a good job of hiding it. That aside, there was only one more small problem – I had not the first clue about how to cook, let alone cook healthy. Obviously I needed help and what does someone do to find help these days? That’s right. I googled it. I found my salvation on a website that not only had thousands of recipes, but each had a rating and comments. I could print out a list of ingredients, create a shopping list, scale recipes to my large family. Most important, I could choose recipes by food type (in my case – healthy foods, low in sodium and cholesterol). Without the internet my plans to eat healthy would have died in infancy.

While I went into this quest as one who was not even a functional cook, I was able to read and follow directions well enough to create some very tasty dishes that the family enjoyed. The plan paid off. My cholesterol and blood pressure went down, as well as other family members including my mother and father-in-law. Over time I even became a functional cook. However, I never became a chef (and to this day I am still not).

Chef PhotoThe difference between a cook and a chef can be summed up in conversations I would have from time to time with my wife. She would ask, “Can you make this dish without this or that? Or maybe substitute this with this.” My response would always be the same. “Sorry honey. I wish I could help you, but I am not sure what this ingredient does and not sure what would happen if I don’t follow the directions to the letter.” I was a cook. What she was looking for was a chef; someone who had a deep culinary knowledge that would allow them to not only modify recipes, but someone with the ability to create new dishes from scratch or show creativity with existing ingredients. I could not do this. I got very good at the motions and following a recipe, but I never cultured a deeper understanding of the why behind the individuals ingredients and actions. I could not “see the big picture.”

Chef PhotoI am often reminded of the difference between a cook and a chef in my agile practice. I have used this story numerous times with developers to explain agile development practices. Like me, it seems that some developers will always be cooks. While there are some who don’t know the difference, I have even run into some that prefer to be cooks instead of chefs. Not that there is anything wrong with choosing to be a cook, but it helps when one is aware of the choice and makes a conscious decision to be one.

I was reminded of this again recently when I went to a talk on scaling agile. The presentation was supposed to last about an hour, but ran long because of the numerous questions and comments from the crowd. It wasn’t until the following day when I was explaining the presentation to a group I was coaching that I realized why the presentation was peculiarly long. The crowd at the presentation were novices with respect to this scaling model and the questions they asked were ones that we would expect cooks to ask. They wanted to know a prescriptive recipe for scaling and wondered how it could be applied to their unique circumstances. Without having the deeper knowledge of a chef, the scaling model recipe didn’t make a great deal of sense to the crowd.

Chef PhotoOf course, this phenomenon has been observed and named the Dreyfus Model of Skills Acquisition. In this model, “cooks” could best be mapped to the “Novice”, “Advanced Beginner” and maybe even the “Competent” categories while “chefs” are referred to by the name of “Proficient” and “Expert”. The Dreyfus Model helps us understand one of the biggest problems facing Agile scaling models today. The scaling models I have some experience with present the world with software development recipes. Unfortunately, the majority of individuals being presented these recipes are cooks (“novices”) and not chefs (“experts”). While one can function adequately as a cook by merely following directions, software development is much too complex. People trying to do Agile scaling are too new to scaling concepts to make successful adaptations should their “ingredients” not match exactly with the Agile scaling “recipe”.

There are a number of Agile scaling models including SAFe, LESS, Nexus, DAD. Since there appears to be quite a bit of money in creating scaling recipes, training people on the recipes and certifying that people have been given and understand the recipes, I expect to see many more in the coming years. Each scaling model claims to have many companies successful in using their model. I am dubious because I have seen overlap among the companies claimed to be successes with the different scaling models (“the usual cast of characters” a person said to me once). I would guess the companies where success is claimed are merely the ones where people paid for the training and certification. What actually happens vis-à-vis scaling models is probably far from what the proponents claim. I expect over time many of these scaling models will be spectacularly unsuccessful. It may be the luminaries of agile have created the perfect scaling “recipe”, but there are too many cooks and not enough chefs for lasting success.