Understanding The Agile Manifesto: A Brief & Bold Guide to Agile – Podcast Now Available

Understanding The Agile Manifesto: A Brief & Bold Guide to Agile by [Apke, Larry]Those of you who may follow this blog may have noticed it has been some time since the last post. It was an eventful summer and it was good to spend some time to catch my breath. Now that the kids are heading back to school, it is time for me to renew my posting. In addition to posting the podcasts, please check back for new Agile posts.

I am pleased to announce that my first book, Understanding the Agile Manifesto: A Brief & Bold Guide to Agile is now available as a podcast. Over the next few days / weeks I will be releasing the book, chapter by chapter, on this website and through the iTunes store under my “Agile Doctor” podcast. Since the book was a series of blogs, the podcasts will also be made available under the original posts.

This project is very dear to my heart as the podcasts were recorded by my son, Vadim Kim, who assisted me over the summer as my intern. I think he did a fantastic job of recording my “little” Agile book and I am sure you will enjoy.

The most recent version of the book includes an Introduction which was never part of any individual post. I have included the podcast of the Introduction below.

 

The Five Attributes of a Good Scrum Team

One of the best experiences I can have as an Agile Coach is presenting Agile and Scrum to people with little or no experience. It grounds me and after many years of living Agile, keeps me from falling into the Curse of Knowledge cognitive bias. Team01smallBetween these presentations and my daily coaching practice, I am reminded yet again at the importance of forming a good scrum team. This, along with proper backlog compilation and maintenance, can make the difference between success and endless frustration. In my experience, proper scrum team formation is the area where companies who are unsuccessful in Agile transformations fail most often.

Over the years, with the help of my teams and the many knowledgeable colleagues that I have worked with, I have distilled everything I have learned and come up with five attributes of a good scrum team. I use it much like a mantra. Those attributes are: Small, Co-Located, Dedicated, Stable and Cross-Functional. That is not to say that a good scrum team will not have other elements or that you can’t witness improvement without all five, but like the sculptor who is only done when there is nothing left to take away, these five represent an ideal essence of a good team. None of these five things can be removed without real consequences to the success of the team.

Small

I write about size at length (and mention the five attributes) in a previous post, When it Comes to Software Development – Size Matters, so I encourage you to read it for some eye-opening account of a major study that confirms this fact. Suffice it to say that Software Development is about communication and collaboration. It is complex knowledge work. In a physical, manufacturing model, adding a person to the effort would result in the expected proportional increase in productivity. In software development, however, adding an additional person merely exponentially increases the number of communication channels whereby any expected increase in productivity is quickly overtaken by the number of communication channels created. There is a reason Agilists recommend small teams. Basically, you can save money because your cost per unit of productive (and quality) work decreases.

Co-Located

Co-Location seems to be an issue with all of the companies that I consult for. We know that the best software developers can be 10 times or more productive than the least. We know that face-to-face communication is multiple times better than any of the substitutes we employ when that mode is not available. We all laugh at the Youco-location01smallTube video “A Conference Call in Real Life”. We know that the unacknowledged creation of technical debt will cause us financial havoc for years to come. Not convinced, I encourage you to check out my blog on the High Cost of Low Cost Software Development. And yet we still continue to source our software development to low bidders around the globe. Here’s the really interesting thing. For many years I have asked a simple question – does it work? I get two answers. The majority shake their head, laugh, and provide a solid “No”. The optimists of the group say that “We have found a way to make it work.” I also see a great deal of blogs that are titled “How We Found a Way to Make Offshore Work”, but it takes little research skills to uncover the author as someone employed by a company that works offshore. Never is there an enthusiastic “Yes.” Most would bring the work back if they could, but it appears that there is a mysterious formation in many software development organizations that continues to think they are doing the right thing by piece-mealing work to various locations. I can only guess it is accounting, but I have yet to meet anyone whom I respect who actually understands software development who would recommend it.

Dedicated

I often rail against the project-centric focus of many software development organizations. For more information, I encourage you to check out my blog “Why Project Focused Mentality is Killing Software Development”dedication. The problem is that in projects we build teams around projects so that one, especially a key employee, will be spread over multiple concurrent projects. There are a number of very serious negative effects to doing this. First, context shifting means that every time a developer must change their concentrated area of focus they will lose 15 minutes minimum. If I am working on two different projects concurrently, there is not a huge loss, assuming that I might work on one in the morning and one after lunch, but many developers are bounced from thing to thing like so many pinballs with context shifting throughout the day. Hours each day are wasted and no project gets the attention it deserves. A better way is to create a small, co-located scrum team around a product backlog and bring the work of these multiple projects to the team via their backlog. The ability for individuals on the team to not fall prey to context shifting will allow them to focus on their work, even at times leading to a state called Flow in which developers (and athletes, artists, etc) are at their peak performance. For those interested, I refer you to the work of Mihaly Csikszentmihalyi.

Stable

 Project-centric thinking also results in teams that are short-lived. We put the team together for a particular period (sometimes relatively short) of time. When the project is done, the team is disbanded. This causes a number of issues. First, every team goes through a period of “storming, norming, and performing.” This is easy to see visually when you plot the velocitiScaleStability01es of multiple teams on a timeline. You can visually see the storming period as teams learn to work together as a team instead of a collection of individuals, through very low (if non-existent) velocity. In my experience, I have come to the conclusion that I expect nearly nothing from a brand new team for the first couple of iterations. It is not until they reach iterations 3, 4 or even 5 that they will reach what is their performing velocity. Therefore, every time I break up a team, I have a very high overhead associated with the team’s “storming and norming.” Another benefit of keeping the same people on teams for a longer period of time is by simply learning how to better communicate with each other over time. Since software development is about communication and collaboration, the more time we spend together, the more effective we become in communicating. I remember reading about the notion of “hyper-productive” teams from Scott Downey and Jeff Sutherland.  The interesting thing is that their “hyper-performance” did not manifest until after the team had been together for some time.

Cross Functional

 In some respects, cross functionality of people is a by-product of the fact that team size should be kept small. If there are 15 things I need to do to get my code into production, I obviously will not be able to do so if there crossfunctional01needs to be an expert for each piece. The creation of “silos” is great if your goal is to create a MDD (mortgage driven development) environment. It is bad for the flow of the product. Silos create queues. Queues, if you want to deliver something quicker, are bad. I highly recommend the book Principles of Product Development Flow by Don Reinersten for a deeper understanding of queuing theory and the damage that out-of-control queues can have. This book talks about the concept of generalizing specialists. In my experience, developers are more motivated by status and mastery of new technology and ideas than any other group of people I have worked with. Creating an environment where developers are expected to know more than a single specialized domain is not only good for moving the software through the system quicker, but it also leads to greater developer engagement. A team should be responsible for all aspects of the software product and the development cycle. This cannot be accomplished without having team members who are cross functional.

Final Thoughts

 An old African proverb states, “If you want to go fast, go alone. If you want to go far, go together.” In order to be successful in software development we need to understand teams and team dynamics. My many years of experience working with what I would estimate around a hundred development teams has led me to conclude that there are five basic attributes for an optimal development team – small, co-located, dedicated, stable and cross functional. I hope that these five attributes help you as you continue your quest for agility.

 

Will Your Development Practices Shield You From Malpractice?

Gavel

GavelThere are a number of reasons why companies who develop software should use Agile. Agile is superior to phase gate approaches like Waterfall in quicker time to market, the practices associated with Agile produce a much higher quality product with less technical debt, maintenance of software (which is the bulk of the SDLC in terms of time and money) is much easier, the engagement of employees is much higher and results in less employee turnover, frequent feedback results in better risk mitigation (to name a few benefits). As I look to the future of software I believe we can begin to add one more thing –protection from software malpractice litigation.

The thought came to mind recently as I was listening to an NPR segment on autonomous automobiles. According to the story, a large segment of the American public is hesitant to trust their safety to autonomous vehicles, even though there are some estimates that cite a potential 80% reduction in traffic fatalities. As human beings we are comfortable with the notion of human error, but find it difficult to find comfort in computer, especially software, errors. We find it especially difficult to accept those that would result in loss of life, even if the average loss of life were greatly reduced. I think that the potential for litigation (and concurrent drag on profits) of such defects is quite high. Not only do I expect software defect litigation to be increased because of autonomous vehicles, the Internet of Things (IOT) and the corresponding explosion in software in all areas of our lives will also create scenarios for increased litigation.

Law BuildingIn some research for this article, I unearthed a paper by Cem Kaner that speaks directly to this issue. While I do not possess the legal mind that he does, I believe I can understand the two potential issues of Negligence and Malpractice he outlines. In regard to negligence, he states, “How do we decide whether a product was developed or tested negligently?…A critical issue to keep in mind is that the plaintiff must prove that the failure to use some ‘best practice’ actually caused the defect in the system.” As software becomes ubiquitous, the methods used to create software will begin to be questioned more often. I believe there to be overwhelming evidence that software developed using Waterfall methods tend to result in greater technical debt, and therefore, the possibility for being on the losing end of a lawsuit is greater. In other words, improper development methods that have been (and continue to be used in many places) not only result in greater technical debt, but also will expose companies to more potential lawsuits. Should I ever be called as an expert witness it would cause me no compunction to reveal poor development processes and practices. We have all heard of lawsuits that have been filed, litigated and judgments made for less.

With regards to malpractice, Kaner states, “Before calling for the professionalization of software quality advocates, please consider the problem that we need a solid basis for distinguishing unacceptable from acceptable practices.” Of course, this paper was written back in 1997 so the amount of evidence available for determining unacceptable from acceptable is growing and the evidence is strong that Waterfall development is no longer the best or accepted practice.

Judge StatueMy argument is this – most software development is complex. It is a well-established truth that complexity is best attacked through frequent feedback, the kind supplied by the agile methodologies and frameworks, as well as the commonly applied practices associated with XP like pair programming, BDD/TDD, code reviews, etc. Software that has been written using methodologies other than Agile generally result in greater number of defects and higher degree of technical debt. As these things are generally accepted by software development professionals as better ways to produce quality software, companies that do not use Agile will become exposed. Furthermore, studies indicate hours worked beyond about 30-40 per week can result in increased numbers of defects. Companies employing this already counter-productive measure have yet another reason for adhering to the agile principle of sustainable development in avoiding potential litigation.

Perhaps malpractice litigation will not affect the realm of software development as I anticipate, but that does not mean it is not appropriate. In some cases, people who with authority to make decisions regarding software development show a willful ignorance of the nature of software development. I believe their behavior is not only detrimental to the production of quality software and the satisfaction of customers and employees alike, but certainly borders on the realm of malpractice. Maybe the hint of malpractice will incentivize people making software development decisions to treat development and developers in a manner more congruent with the true nature of software development.

Maybe It’s Time to Stop Using the Word “Sprint”

sprinting

Any agilest worth their salt will tell you that culture matters, especially when you are trying to transition to an Agile methodology. Since culture is merely a reflection of an organization’s values and values come from ideas and words are how ideas are expressed, the words we use can often be critical. There is one word used Agile and Scrum that has bothered me for some time, “sprint”, and I propose eliminating the word from Scrum. In fact, I hope Ken Schwaber and Jeff Sutherland agree to incorporate this change in the next version of The Scrum Guide.

rugby scrumThe Scrum Guide states, “the heart of Scrum is a Sprint”. This is one reason why I haven’t written this blog sooner. I believe in Scrum as an excellent way for teams to embody the Agile philosophy so I am hesitant to criticize. Who am I to criticize and will my criticism be misinterpreted? It took a long time for me to have the confidence to propose something that others may view as heretical (I know how strident some can be about Agile and Scrum), but I truly believe it is the right thing to do.

I acknowledge that I might not be the first to propose such a change. In fact, I had one gig where the company decided to refer to “sprints” as “iterations”. They insisted I do the same and I found it difficult to make the change. I thought this was just another example of “scrum but” (which in this case may have been true), but even though I do so begrudgingly, I now believe this company was on to something worthwhile. I also acknowledge that my opinion might not carry the requisite weight, but I feel it is necessary to add whatever weight it might hold to this effort.

sprintingAnd why would I propose something as radical as changing the very word referred to as the heartbeat of scrum? Simply this. The word itself carries a connotation that I find at odds with the values and principles of the Agile Manifesto. Therefore, I need to make this proposal in order to preserve my integrity. As a coach I often say, “The philosophy behind Agile provides a basis for making decisions when there is doubt on which option to pursue. The Scrum framework contains ceremonies that make the philosophy come alive in our daily practice. If you cannot trace something you are doing to the original philosophy then you probably should not be doing it.” It is for this reason that I no longer can use the word “sprint” and instead choose to use the word “iteration.” Of course, until the Scrum Guide is changed to reflect the more accurate term, I often say “iteration or sprint”.

There is an Agile principle which states, “Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely”. It has become obvious to me, the word “sprint”, which, although it may have a specific meaning in scrum, generally is understood to mean “an act or short spell of running at full speed.” I jog to stay in shape. As I have stated previously it is obvious that sprinting is something that cannot be sustained over a long period of time. Therefore, I prefer the word “iterating” and hope that someday the Scrum Guide, as the official word of what constitutes Scrum, will agree with me and change the word.

Cooks, Chefs and Agile Scaling Models

Chef Photo

Chef Photo     A few years back my doctor gave me some somber news. He told me my cholesterol was too high and I must get it lower. It was then that I decided I would make changes to my less than stellar diet. At the time, we had a large family in the house – seven including my father and mother-in-law. My mother-in-law had become the default cook for the family and, while she made some very tasty dishes, it was not exactly healthy fare. I knew that if I were to reach my goal of a better diet I would have to become the new cook for the family.

Chef PhotoOriginally I thought that my mother-in-law would be upset if I banished her from the kitchen, but if she was, she did a good job of hiding it. That aside, there was only one more small problem – I had not the first clue about how to cook, let alone cook healthy. Obviously I needed help and what does someone do to find help these days? That’s right. I googled it. I found my salvation on a website that not only had thousands of recipes, but each had a rating and comments. I could print out a list of ingredients, create a shopping list, scale recipes to my large family. Most important, I could choose recipes by food type (in my case – healthy foods, low in sodium and cholesterol). Without the internet my plans to eat healthy would have died in infancy.

While I went into this quest as one who was not even a functional cook, I was able to read and follow directions well enough to create some very tasty dishes that the family enjoyed. The plan paid off. My cholesterol and blood pressure went down, as well as other family members including my mother and father-in-law. Over time I even became a functional cook. However, I never became a chef (and to this day I am still not).

Chef PhotoThe difference between a cook and a chef can be summed up in conversations I would have from time to time with my wife. She would ask, “Can you make this dish without this or that? Or maybe substitute this with this.” My response would always be the same. “Sorry honey. I wish I could help you, but I am not sure what this ingredient does and not sure what would happen if I don’t follow the directions to the letter.” I was a cook. What she was looking for was a chef; someone who had a deep culinary knowledge that would allow them to not only modify recipes, but someone with the ability to create new dishes from scratch or show creativity with existing ingredients. I could not do this. I got very good at the motions and following a recipe, but I never cultured a deeper understanding of the why behind the individuals ingredients and actions. I could not “see the big picture.”

Chef PhotoI am often reminded of the difference between a cook and a chef in my agile practice. I have used this story numerous times with developers to explain agile development practices. Like me, it seems that some developers will always be cooks. While there are some who don’t know the difference, I have even run into some that prefer to be cooks instead of chefs. Not that there is anything wrong with choosing to be a cook, but it helps when one is aware of the choice and makes a conscious decision to be one.

I was reminded of this again recently when I went to a talk on scaling agile. The presentation was supposed to last about an hour, but ran long because of the numerous questions and comments from the crowd. It wasn’t until the following day when I was explaining the presentation to a group I was coaching that I realized why the presentation was peculiarly long. The crowd at the presentation were novices with respect to this scaling model and the questions they asked were ones that we would expect cooks to ask. They wanted to know a prescriptive recipe for scaling and wondered how it could be applied to their unique circumstances. Without having the deeper knowledge of a chef, the scaling model recipe didn’t make a great deal of sense to the crowd.

Chef PhotoOf course, this phenomenon has been observed and named the Dreyfus Model of Skills Acquisition. In this model, “cooks” could best be mapped to the “Novice”, “Advanced Beginner” and maybe even the “Competent” categories while “chefs” are referred to by the name of “Proficient” and “Expert”. The Dreyfus Model helps us understand one of the biggest problems facing Agile scaling models today. The scaling models I have some experience with present the world with software development recipes. Unfortunately, the majority of individuals being presented these recipes are cooks (“novices”) and not chefs (“experts”). While one can function adequately as a cook by merely following directions, software development is much too complex. People trying to do Agile scaling are too new to scaling concepts to make successful adaptations should their “ingredients” not match exactly with the Agile scaling “recipe”.

There are a number of Agile scaling models including SAFe, LESS, Nexus, DAD. Since there appears to be quite a bit of money in creating scaling recipes, training people on the recipes and certifying that people have been given and understand the recipes, I expect to see many more in the coming years. Each scaling model claims to have many companies successful in using their model. I am dubious because I have seen overlap among the companies claimed to be successes with the different scaling models (“the usual cast of characters” a person said to me once). I would guess the companies where success is claimed are merely the ones where people paid for the training and certification. What actually happens vis-à-vis scaling models is probably far from what the proponents claim. I expect over time many of these scaling models will be spectacularly unsuccessful. It may be the luminaries of agile have created the perfect scaling “recipe”, but there are too many cooks and not enough chefs for lasting success.

Hit Rock Bottom? Maybe Now You’re Ready for Agile

Despair

Despair One of the things I enjoy most about my work is meeting other Agile practitioners and coaches and sharing war stories. Recently I was at an Agile Coffee with people who had various experience with Agile, ranging from complete neophytes to folks with many years of experience. The topic chosen was “What are the preconditions for a large company to be successful implementing Agile?” – a very pointed, yet valid, question to be sure. All of the answers were ones that one might expect; support from upper management, learning culture, etc. except for one. The lone unobvious answer was provided by someone whose opinion I respect. I was somewhat surprised at the time and it has reverberated in my mind ever since, “one precondition for a company to be successful in an Agile transformation is they have failed miserably and have hit rock bottom.”

The term “rock bottom” is most commonly used by folks in NA and AA (Narcotics Anonymous and Alcoholics Anonymous). I found the following definition of rock bottom on a website for alcohol addiction:

… often used to describe a point … when they are finally willing to seek help. Things are now so bad for them that it is impossible to deny their problem anymore. Hitting rock bottom may result due to a particular event, or it can be a slow decline over time. This is a subjective term because some … will be willing to suffer a lot more … than others.

I first came in contact with Anonymous groups through my work in healthcare (nearly a lifetime ago when I was a licensed Health Care Administrator) and have more than one commented on how Agile transformations remind me of some of the twelve steps – with the first step being to admit one has a problem.

So when my friend mentioned that a good indicator for agile transformation success was a company had hit rock bottom I knew exactly what he was referring to. In this particular case he used the examples of the FBI Sentinel Project and Healthcare.gov website debacle. In both cases, it wasn’t until each was a total disaster that Agile was actually tried with any seriousness and rigor and in both cases the results were amazing.

The total spend of these [two] failed [Waterfall] attempts to replace the ACS system was $597m and wasted 10 years. The Agile project, which is now delivering a solution, will only cost $114m for a three-year long project.

DespairIn another recent talk I went to on Agile a gentleman explained how Agile helped with an ad agency. He was only brought in AFTER the agency blew millions on a website that never made it to production. After losing millions I would guess this agency hit “rock bottom”. The good news was that Agile was able to help them to change and allowed them to produce high quality software with a much better time to market (than never).

In my agile coaching practice I have had more than one conversation with enlightened management that has acknowledged our Agile transformation wasn’t working as good as hoped and, barring a complete meltdown, most folks were happy to continue along a non-optimal path. In fact, relative success is a particularly sticky barrier to change as people often confuse the ability to get something done with the ability to get something done optimally. Bill Gates has often remarked, “Success is a lousy teacher. It seduces smart people into thinking they can’t lose.” Sometimes what we mistakenly call “failure” is a much better teacher. Sometimes the lesson is more important than the perception of relative success or failure.

Just this morning I was listening to my local NPR station as they were having a fund drive. Like many others, I couldn’t wait for the drive to be over so I could no longer had to feel the shame and guilt of being a “free rider”. Fortunately, it was the last day of the pledge drive and my mind was only half listening to the chatter of the announcers when they introduced Shankar Vedantam who does segments on the Hidden Brain (which I enjoy immensely). His topic was why people donate (or don’t donate) to pledge drives. He talked about humans’ tendency to pay more attention to emergencies and crisis than what is important. Being the last day of the pledge he hoped what was merely important (donating) would now also be given the status of an emergency because time was running out.

DespairMy mind went immediately back to the concept of “rock bottom” and why this is one good indicator of where Agile can be successful. Sometimes it is only when we hit rock bottom that the importance of being Agile begins to align with the immediate crisis of having to be Agile. It is then, and unfortunately for some, only then when Agile gets the attention and focus it deserves. It is my hope companies realize becoming an Agile organization is essential to their long term survival (at least in organizations that rely heavily on software development), seek the help they need to transition and actually carry out the transition before they hit rock bottom. In the meantime, if you have hit rock bottom (are you listening Yahoo!?), please seek out Agile help.