Leadership - Agile Doctor

8 April, 201612 April, 2016

“We Can’t Find Good People” Myth and the Rise of Trump and Sanders

Working for a staffing company and living the life of a consultant for many years, I believe it safe to say I have learned a great deal about how employment is done in our country, and particularly, how it is done in IT. I don’t think it takes a genius to see it is broken.

I have been doing some recent work with NOVA (with SIS support I conducted a one month deep-dive into Agile and Scrum with NOVA participants) and was introduced to ProMatch by Jennifer Cheyer of NOVA. I have also been working with Fred Fowler and his Scrum training class that recently completed a public/private partnership with Silicon Valley Polytechnic Institute (SVPI), Coding Dojo (a coding “bootcamp”), NOVA/ProMatch,, the Taylor Family Foundation (a non-profit) and SIS, Inc. It is an amazing story of what can be accomplished when people work to align motivations. I encourage you to check out their story.

Since some of the work was done with ProMatch, I was curious to see what they did and how I could help them. I went to one of the ProMatch meetings yesterday in Sunnyvale, California and it was then I was reminded of just how sorely broken our employment system is. According to its website “ProMatch is a collaboration between EDD (Employment Development Department of the State of California) and the NOVA Job Center” and “is a powerful networking program for unemployed Silicon Valley professionals … over 200 active members and a waiting time of approximately one month to join.”

I hope that you take a moment to let that sink in. Companies complaining that they cannot find good people, so much so that they have to go outside to United States through programs like H1B (or worse yet, ship work completely offshore), while there are over 200 people actively looking for work. There are so many good people looking for work that there is a waiting list to enroll in a program that helps them to find work. Hate to veer to political, but if anyone is still mystified by the Trump and Sanders “phenomenon” they should look no further than a ProMatch meeting in Silicon Valley!

The number is more staggering when you actually show up in person. The meeting is held in the Sunnyvale Council Chambers and the place is overflowing with people. The demographics are all over the board and seem to be a homogeneous representation of society at large with one exception – the age of the participants skews much older. Each new member spends a short 15 seconds to introduce themselves and what they are looking for. Not only do the individuals have diversity of race and gender, they also have a great variety of skills, high tech included. Words cannot express the amount of talent and experience in that one room.

And yet there is one thing that I found conspicuously absent – employers. Though there may have been more that I was unaware of, it appeared that the only two people representing real jobs were a former SIS, Inc.. colleague and myself. I spoke briefly with Robert Withers, a full time ProMatch employee, and he was as perplexed as I was that more employers and staffing agencies were not there. He wonders, as I do, why they do not see what he and I do when we look out at the over one hundred well-qualified applicants in the council chambers that morning.

This is why I know the “I can’t find good talent” argument is a myth. Right there in downtown Sunnyvale was a treasure trove of immense talent just waiting to be employed. The problem is that we, as a society, no longer seem to know how to invest in our people or our infrastructure. Companies no longer train their employees, but use them up and spit them out so that when they come to ProMatch their skills might be slightly outdated. They look for the “perfect” fit and don’t invest in making a close fit better. We look for cheaper through younger and overseas workers, oblivious to the fact that knowledge management is won by the best, brightest and most experienced.

The American public screams for change which is why we have seen the rise of establishment “outsiders” like Trump and Sanders. The ProMatch meeting I witnessed yesterday was the personification of the change that they American public seeks. The American public desires a world where employees are valued for their experience, where they are invested in instead of harvested. America dreams of a day when instead of a waiting list to be part of ProMatch, a meeting has been cancelled due to lack of attendees. The public/private partnership work that I am part of and alluded to at the beginning of this post is merely one small dent. What is needed is the participation of more people, more companies. We need to make this issue viral. Share it, comment on it, like it so that maybe next time I go to ProMatch (and I will keep coming back) the room will be literally crowded with representatives from all these companies crying and complaining “we can’t find any good people.”

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Android | RSS

14 March, 201628 June, 2016

Will Your Development Practices Shield You From Malpractice?

There are a number of reasons why companies who develop software should use Agile. Agile is superior to phase gate approaches like Waterfall in quicker time to market, the practices associated with Agile produce a much higher quality product with less technical debt, maintenance of software (which is the bulk of the SDLC in terms of time and money) is much easier, the engagement of employees is much higher and results in less employee turnover, frequent feedback results in better risk mitigation (to name a few benefits). As I look to the future of software I believe we can begin to add one more thing –protection from software malpractice litigation.

The thought came to mind recently as I was listening to an NPR segment on autonomous automobiles. According to the story, a large segment of the American public is hesitant to trust their safety to autonomous vehicles, even though there are some estimates that cite a potential 80% reduction in traffic fatalities. As human beings we are comfortable with the notion of human error, but find it difficult to find comfort in computer, especially software, errors. We find it especially difficult to accept those that would result in loss of life, even if the average loss of life were greatly reduced. I think that the potential for litigation (and concurrent drag on profits) of such defects is quite high. Not only do I expect software defect litigation to be increased because of autonomous vehicles, the Internet of Things (IOT) and the corresponding explosion in software in all areas of our lives will also create scenarios for increased litigation.

In some research for this article, I unearthed a paper by Cem Kaner that speaks directly to this issue. While I do not possess the legal mind that he does, I believe I can understand the two potential issues of Negligence and Malpractice he outlines. In regard to negligence, he states, “How do we decide whether a product was developed or tested negligently?…A critical issue to keep in mind is that the plaintiff must prove that the failure to use some ‘best practice’ actually caused the defect in the system.” As software becomes ubiquitous, the methods used to create software will begin to be questioned more often. I believe there to be overwhelming evidence that software developed using Waterfall methods tend to result in greater technical debt, and therefore, the possibility for being on the losing end of a lawsuit is greater. In other words, improper development methods that have been (and continue to be used in many places) not only result in greater technical debt, but also will expose companies to more potential lawsuits. Should I ever be called as an expert witness it would cause me no compunction to reveal poor development processes and practices. We have all heard of lawsuits that have been filed, litigated and judgments made for less.

With regards to malpractice, Kaner states, “Before calling for the professionalization of software quality advocates, please consider the problem that we need a solid basis for distinguishing unacceptable from acceptable practices.” Of course, this paper was written back in 1997 so the amount of evidence available for determining unacceptable from acceptable is growing and the evidence is strong that Waterfall development is no longer the best or accepted practice.

My argument is this – most software development is complex. It is a well-established truth that complexity is best attacked through frequent feedback, the kind supplied by the agile methodologies and frameworks, as well as the commonly applied practices associated with XP like pair programming, BDD/TDD, code reviews, etc. Software that has been written using methodologies other than Agile generally result in greater number of defects and higher degree of technical debt. As these things are generally accepted by software development professionals as better ways to produce quality software, companies that do not use Agile will become exposed. Furthermore, studies indicate hours worked beyond about 30-40 per week can result in increased numbers of defects. Companies employing this already counter-productive measure have yet another reason for adhering to the agile principle of sustainable development in avoiding potential litigation.

Perhaps malpractice litigation will not affect the realm of software development as I anticipate, but that does not mean it is not appropriate. In some cases, people who with authority to make decisions regarding software development show a willful ignorance of the nature of software development. I believe their behavior is not only detrimental to the production of quality software and the satisfaction of customers and employees alike, but certainly borders on the realm of malpractice. Maybe the hint of malpractice will incentivize people making software development decisions to treat development and developers in a manner more congruent with the true nature of software development.

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Android | RSS

23 February, 201628 March, 2016

Yahoo! – Some Thoughts on Regaining Former Glory

I try my best not to call out particular companies in my writing though from time to time I have done so. I do so when only absolutely necessary to make a point that I would like to make, basically because it’s not great business to have people (or entire companies) upset with you. Also, though I work for enlightened people who have given me freedom to express my personal ideas through presentations, blogs podcasts, etc., I don’t want to strain that relationship. As always, the thoughts expressed here are my own and not necessarily endorsed by my employer.

Caveats aside, I wanted to address something that has been weighing heavy on my mind. I have found that expressing myself through writing has a tendency to clear my mind so you can view this blog as a form of self-therapy. My topic this week is the fall of a company I (still) have a great deal of affection for, one that I continue to support. To me this company “was” the internet and their rise and past prominence represented what was best of Silicon Valley. More troubling is their subsequent lack of direction and downfall represents what can be thought of as the worst of Silicon Valley. I am not the first (and will not be the last) to weigh in on the tragedy that goes by the name of Yahoo!

I have often stated that when a company has hit “rock bottom” (as in my previous blog) it is time to acknowledge some core truths about their business and, only then, can the healing process begin. There is much to heal at Yahoo! I speak not simply about financial concerns. Although these are paramount, in many respects financial difficulties are merely the symptoms of much larger problems, for instance, confidence in Yahoo! as a brand. While many, myself included, continue to use Yahoo!, it has, for some time, lacked the cache’ that brands like Google, Facebook, Apple (and even Microsoft) have. Not only does it not seem to have a particular direction, if it had some direction, that direction would not be seen as “cool”.

This begs the question as to what direction should Yahoo! take. This is where I believe Yahoo! has squandered some opportunity because I believe that the only mission worthy of enticing talent back to a failing enterprise must be a bold one, a moon shot if you will. To date, Yahoo! appears to have taken no real bold moves (with the exception of over spending on Katie Couric). Their competition, the ones “eating their lunch” have – Alphabet (formerly Google) has multiple moon shots, including self driving cars, Elon Musk is in Fremont forging a new future of electric cars and rockets, Apple is rumored to be developing their own car, etc. Technology IS talent and talent will go where it is inspired. I thought of this the other day when I attended an event at AOL. Tell someone that you attended an event at AOL and the first thing out of their mouth is “you mean they are still in business?” Yahoo! will be the same in very short order unless they can change perception. To me that will begin when they find their own “moon shot.” It will not happen anymore with only incremental change.

I remember reading somewhere that the problem with Silicon Valley is there is a tendency to solve the problems of no longer living with your mother. There are a ton of startups not taking on the larger battles society is facing (and many like global warming that society is currently losing). I think there are two lessons for Yahoo in this; one, they must begin acting more like a startup, and two, they must be a startup that is actually solving problems whose solutions will be meaningful and impactful to society at large.

While there are a number of societal problems Yahoo could choose to tackle, like global warming or clean and safe drinking water, let me suggest one that I feel would be a good fit – education. In fact, I have a great name for it, Yahoo University (or as my wife affectionately calls it – Yahoo U). Why education? Yahoo is now primarily a content provider so a great deal of infrastructure is in place to begin solving the problems of the current educational system. I think that smart acquisition or partnerships with a number of companies operating in this space (think Khan Academy and the various coding bootcamps) could give Yahoo University an advantage. It would certainly have name recognition (which is one of its few convertible assets outside of Alibaba).

Attempts to solve the problems of education for the way we work today have been made by many parties, but no satisfactory solutions have been made. At issue is creating education that is actually useful to the general populace in positioning them for gainful employment in our quickly shifting, knowledge-based economy. Traditional classroom education has suffered because it still dictates “what” to learn instead of “how” to learn. For example, what good is it to teach someone Objective C when it is no longer being used or desired in the real world? What happens when the world has moved on to Swift? The real issue is to teach someone how to teach himself or herself to learn Objective C so that when they need to learn Swift they have the skills to teach it to themself. In other words, we must move away from content for content’s sake and teach the mental agility necessary to survive. Technology has existed for sometime to solve this particular issue yet we have done little to use technology for this goal. Instead we have used the technology to shovel content. It’s as if we teach people to drive a Ford and when given a Chevy they cannot drive it.

While I could rant more on this topic, I feel it is important to also address how one would go about making real money from such an endeavor and how Yahoo can reinvigorate its sagging stock prices. It would be most obvious that Yahoo University could borrow the model from existing online educators such as the University of Phoenix, but this would be a mistake. Anyone paying attention to this space would know that the fortunes of private sector online educators is bleak. I, myself, worked for the Apollo Group, the parent of University of Phoenix and personally know some of the problems that have crippled them over time – things like over-reliance on government-backed student loans, high tuition costs, students with high debt and poor prospects of repaying debt, etc. Online education was a brief success and made some wildly prosperous – just not necessarily the students.

I propose a more radical solution for Yahoo University – free education. But how can you make money providing education for free? You can, if the education you provide actually has a value to the world at large. I propose that students entering courses that have a good prospect of being placed in real jobs be given a free education with the stipulation that Yahoo University be reimbursed when they student is placed in a job related to their education. This aligns the motivation of the educator in doing what is necessary to provide marketable skills and the motivation to actually ensure that the student get a job! Being in the staffing industry I know that a properly educated and placed individual can provide enough compensation on the back end to justify the expense on the front end. In talking with my friends in software development, this is exactly what is happening in countries outside the United States with H1-B candidates taking the jobs that could be filled with Yahoo University graduates.

As to the issue of stock value, I think that taking this course of action will make it much easier to increase shareholder value. If I were the CEO of a company on the course described above, I would take my argument not only to Wall Street but to Main Street. The incredible societal value of providing free and meaningful education and job placement would inspire investor outside of Wall Street. While I am the last to wrap myself in the flag, there is certainly not only something not only highly practical to this plan, but something downright patriotic in solving American problems with American citizens. I don’t find it hard to imagine that people would want to purchase shares of a company that had such a unique and worthy vision. Of course, it would not hurt that as Main Street bought shares that Wall Street would join.

I do not know Marissa Mayer and it may be that where Yahoo! is now has nothing to do with her leadership. She may be the most competent individual on planet earth (we all know she was successful at Google), but what matters is perception, and right or wrong, the perceptions of Marissa Mayer are predominantly negative, both with the public at large and within the employees (and ex-employees) of Yahoo! I cannot see how Yahoo! could make the necessary transition without removing the current CEO.

I guess you can file this blog under the title of a modest proposal. I wish Yahoo all the best and I am sure that there are many others who agree and would like to see this once proud company bounce back. If you, dear reader, agree, I would suggest that you take the time to share this post with others, like this post, or feel free to leave your comments. Perhaps the folks at Yahoo will take notice. Perhaps if they did they might find something valuable as they face the tough times ahead. At the very least one can hope.

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Android | RSS

11 January, 201612 January, 2016

Project Manager/Scrum Master: A Cry for Agile Help

Over my time as an Agile Coach I cannot even begin to count the number of times I have been approached by recruiters for a position as a Project Manager/Scrum Master. My coaching colleagues and myself often joke about this particular job title saying, “we like the title Project Manager/Scrum Master because it helps us know which jobs and companies to avoid.” Why would we avoid such positions? For better or worse, by using the title Project Manager/Scrum Master we are able to quickly infer a great deal about the company and their experience with Agile – most of it not positive.

The first thing we notice about a company using this title is that they usually have little or no understanding of Agile and Scrum. The duties of a Project Manager and Scrum Master are quite different. Not only do the two roles perform different functions, but they represent a fundamental different in the way they view the world. Project managers in Waterfall try to control projects. Scrum Masters work with Agile Scrum teams to facilitate. In Waterfall, Project Managers have responsibility. Scrum Masters, on the other hand, do not have direct responsibility as the responsibility for success shifts to the development team and Product Owner. Since the Product Owner decides the sequence of the work, a Product Owner is the closest thing to a Project Manager in Scrum. A simple Google search of “project manager versus scrum master” can provide more information on this for those who are curious.

This particular problem manifests when a company desires the potential benefits of Scrum without really understanding Scrum. Without a good understanding, people attempt to map their existing roles with those of Scrum. Let me make one thing perfectly clear. The role of Scrum Master is unique to Scrum and any attempt to map it to existing roles will only result in confusion, frustration and less than optimal outcomes. As coaches, if given the choice of coaching a complete novice as scrum master or “retrofitting” a project manager as a scrum master, we chose the former. It is not that project managers cannot become good scrum masters because many have, but in order to properly train one as a scrum master there is a great deal of work in “unlearning” much of what has been previously trained. With novices the time spent “unlearning” is non-existent.

This is why I believe that companies recruiting for Project Managers/Scrum Masters are actually making a very public plea for help. If a company truly wants the benefits of Agile, it is essential that they actually take the time to truly understand that becoming Agile through the use of the Scrum framework is a serious commitment. People must gain a better understanding of software development and how knowledge workers differ, change their fundamental thinking around projects and products, pursue organizational change and realign people around properly configured scrum teams, work with recruiters who understand the difference between project managers and scrum masters and work as partners to recommend better solutions to underlying organizational needs.

For those seeking Project Managers/Scrum Masters, I hear your cries for help. As an Agile Practice Director I would love to help you to better understand your real Agile needs, to help you reorganize your work and people to take advantage of Scrum (other Agile frameworks, methodologies and development techniques), to optimize your organization so that you can deliver high quality software in a shorter time. I am available anytime to provide the help you need. If not me, please reach out to another Agile professional so we can rid the world of Project Manager/Scrum Master for good.

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Android | RSS

21 December, 201521 December, 2015

On Better Hiring (or Do Coding Interviews Work? Part 2)

Last week I wrote about coding interviews and questioned whether they are the best method to predict future job success. There were strong opinions on both sides of the argument. Someone expressed the opinion that I was taking a quote from Laszlow Bock, senior vice president of people operations at Google, out of context, so I would like to begin by giving him the first word on what Google has discovered about interviews. Interestingly, he refers to cognitive biases, specifically confirmation bias (as I did in my post) as a reason traditional interviews are not good indicators of job performance.

In April 2015 he wrote an excellent article for Wired Magazine and I encourage everyone to take a look so that they can form their own conclusions. I will quote at length in order to make sure that Mr. Bock’s ideas are not taken out of context. In the article he stated:

In 1998, Frank Schmidt and John Hunter published a meta-analysis of 85 years of research on how well assessments predict performance. They looked at 19 different assessment techniques and found that typical, unstructured job interviews were pretty bad at predicting how someone would perform once hired.

Unstructured interviews have an r2 of 0.14, meaning that they can explain only 14 percent of an employee’s performance. This is somewhat ahead of reference checks (explaining 7 percent of performance), ahead of the number of years of work experience (3 percent).

The best predictor of how someone will perform in a job is a work sample test (29 percent). This entails giving candidates a sample piece of work, similar to that which they would do in the job, and assessing their performance at it. Even this can’t predict performance perfectly, since actual performance also depends on other skills, such as how well you collaborate with others, adapt to uncertainty, and learn.

I believe that the work sample test he refers to is analogous to the coding interview. If not, I will error on the side of caution since equating the two would mean that coding interviews are the most effective. Even so, the very best interview technique is only twice as effective as an unstructured interview in predicting job performance. I hope some statisticians can weigh in on the topic, but an r2 value of 0.29 does not seem to be very indicative of future job success. I vaguely remember from college that in physical sciences we look for much higher r2 values before we accept a hypothesis as proven.

Bock goes on to claim that using a combination of interview techniques in a “structured interview” is even more predictive (though he does not present any r2 values for the combination of techniques). And, believe it or not, I agree with him. He has presented some real scientific information that a structured interview process is better than an unstructured one, but I still wonder if we have committed a type 3 error (solving the wrong problem precisely).

Bock and I both agree that “the goal of our interview process is to predict how candidates will perform once they join the team,” but he also admits that doing things as he recommends is “hard to develop” and “a lot of work.” Even Bock states:

Full disclosure: I’m the Senior Vice President of People Operations at Google, and some of these (unstructured) interview questions have been and I’m sure continue to be used at the company. Sorry about that.

And this is where I would begin to question whether this process is one that everyone should follow. Of course, for highly desirable and highly capitalized companies like Google, Facebook, etc. the benefits likely outweigh the costs, but for the majority of companies that interview software developers, this may be a luxury they cannot afford. This overall hiring style is time consuming. There is not doubt that developers are willing to jump through hoops to work at Google or Facebook, but such is not the case for most companies where time is not a friend. If a majority of companies use this process they may find a much higher cost in potential candidates lost than the marginal effect of hiring better. This is the reality that many managers and companies face.

This begs the question of what should those who are not Google or Facebook, those without unlimited resources and legions of potential candidates, do to improve their hiring of software development professionals? The first thing is to be aware of the true costs and benefits associated with pursuing one path over another. Eschew convention and ask the hard questions like, “Given limited time and financial resources, is this the best use of either (or both)?” It appears that Bock agrees there is a high cost to more effective interviews, but for most companies is the higher cost justified? Is there a better way to allocate time and money? I believe there are things that should be considered and evaluated.

One thing I have suggested in the past is the concept of apprenticeship and creating a pipeline of talent from within a company. This is an essential aspect of good management and would reduce the need for interviewing new candidates. In my experience is that in software development in particular we often look for perfect fits and do not do a good job of creating a pipeline of candidates. This puts us “behind the eight ball” and increases the risk of hiring poorly, providing high pressure to make the right hire.

If we are truly looking for the best predictor of how well someone will preform on the job, then we could certainly achieve a much higher r2 value by actually putting someone on the job for a limited period (or at least give them an experience that would be as much like the job as possible). In response to my original blog, Duncan Campbell had perhaps the most insightful comment when he wrote:

The best way to find out if someone is good for a job is for them to do the job… which is why our “coding interview” is a real problem done in the candidate’s own time on their own PC followed by a code review.

I agree with Duncan, but it may even be cost effective to go further. Instead of spending many hours via interview, make the quick decision to bring the candidate into the company on a trial basis, perhaps two weeks at pay. We all seem to have projects that are less critical that would be a good proving ground for potential long-term employees. If the candidate doesn’t work out after two weeks, then part company. It may prove more cost effective for managers to actually do the work of managing existing people than spending a huge chunk of time in a laborious interview process.

Another possibility is the concept of contract to hire or using staffing agencies to do the heavy lifting. Full disclosure: I work for a company that has staffing as one of our offerings. Nevertheless, having another company take the time to screen candidates and having the employment-related risk owned by a third party does have its place in this discussion.

My original article, titled “Do Coding Interviews Work?”, was purposely open-ended. I have tried my best to present information to help people answer that particular question. The title was not “are there better ways to interview software developers?” for a reason. I am truly questioning any interviewing technique because of the high cost and the low correlation – even when interviews are conducted perfectly (or as least as perfectly as science has instructed us). I do not think that we will get rid of interviewing altogether, but it is important to know that for a great majority of companies there may be alternatives that, given all the costs and benefits, may be more effective ways of answering the larger question of how best to hire. I gave a few suggestions. I look forward to others.

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Android | RSS

1 December, 20159 December, 2015

Agile – It’s All About Making Better Decisions

I’ve been spending a lot of time recently doing research, reading and presenting on human cognitive biases. To the initiated, cognitive biases are defined as

“…a systematic pattern of deviation from norm or rationality in judgment, whereby inferences about other people and situations may be drawn in an illogical fashion. Individuals create their own ‘subjective social reality’ from their perception of the input.” (Wikipedia Definition)

In other words, cognitive biases exist when there is a gap between our perception of reality and objective reality. For example, there is the “confirmation bias” which is our human tendency to seek out or interpret information that confirms one’s existing opinions.

While the term “cognitive bias” is relatively new (it was coined in 1972 by Amos Tversky and Daniel Kahneman), researchers have already uncovered literally over a hundred cognitive biases, some which are relatively tame like the “google effect” (or digital amnesia), where there is a tendency to forget information that can be easily researched, to ones that can lead to more disastrous consequences like the Sunk Cost Fallacy where people justify increased investment in a decision based on prior investment instead of looking only at future efficacy. The Sunk Cost Effect, along with the Overconfidence Effect and Receny Effect, played a role in the May 1996 mountain climbing tragedy, made famous in the movie Everest, that resulted in the death of five experienced climbers.

A great number of cognitive biases have been found through the work of behavioral economics researchers like Dan Ariely who wrote the wonderful books Predictably Irrational and The Upside of Irrationality. Underlying all of classic economics is the concept of homo economicus, or economic man who behaves in rational ways to maximize individual returns and acts in his own self-interest. Unfortunately, this is not the case and humans often act irrationally (and predictably so) because of their inherent cognitive biases. Humans all have biases for loss aversion and would choose to avoid loss over a larger corresponding potential gain and thus act as “homo irrationalis” as discovered by behavioral economics instead of “homo economicus” as predicted by classic economics.

It is our cognitive biases that cause us to make irrational decisions. Since behavioral economists found many of these cognitive biases, it was not a great leap to see how cognitive biases would be a paramount concern for the economics of software development. In my coaching practice, a great deal of my time and effort is used in helping organizations make better decisions about software development. Many times the optimal decisions are counter intuitive to people’s inherent biases so my job (and my passion) is helping companies see the world of software development differently so that, when it comes down to making a decision, they have all the knowledge necessary to make the optimal economic decision.

One of the most prevalent biases in software development is to see the world in a mechanistic / Tayloristic manner. Taylor’s viewpoint was fine for the old world of physical work, but does not hold up in the complex knowledge work being done by software development professionals today. Unfortunately, most of the people making software development decisions are predominantly influenced by this old, less optimal way of viewing the world, and, as a result, make sub-optimal decisions. For example, in the mechanistic worldview, adding more people to an effort results in a corresponding increase in output. If there is an existing team of seven people and we add seven more then we would (if we hold this mechanistic bias) expect the work to be approximately twice as fast. However, like the behavioral economists that found the real world to be counter intuitive to homo economicus, actual studies have found that the need for increased communication of knowledge work nearly outpaces any incremental increase in individual productivity (see Top Performing Projects Use Small Teams). I have always said that if you want to double productivity of a fourteen person team all that is necessary is to create two teams of seven.

The mechanistic bias can also be seen in many of the ways that the Agile philosophy is implemented. For example, the scrum framework is often trained as a series of ceremonies and actions with little or no understanding of the reason such mechanistic actions are successful. “Scrum Masters” are “certified” with only two days of training and a simple test. The training deals with ideal situations, but when the scrum master actually has to implement scrum, he or she is woefully unprepared. In the real world compromises and decisions must be made. Without understanding the underlying “why” of agile and the basic nature of software development, the decisions and compromises that are made are not optimal. In my experience, this is why project managers are tougher to train than people with no project management experience. When faced with ambiguous information and the need to make optimal decisions, project managers tend to fall back on existing mechanistic knowledge and the decisions made range from mildly irritating to completely disastrous. As I have often pointed out, to say that one was successful with waterfall reeks of confirmation bias because it begs the question of whether or not one would have been more successful using another methodology or framework like Lean or Scrum.

In addition to the mechanistic bias, software development suffers from another bias, the project-centric bias, which is the tendency to see all work done in terms of projects. Unfortunately, the project-centric bias is so ingrained in companies that there needs to be some radical changes to the way we view software development across all areas, including accounting. Viewing work as a project when we are actually working on software products results in a whole raft of poor software economic decisions like concentrating on features more than quality and security. Remember that no one washes a rental car.

As I think back on my coaching work in agile, the blogs I have written, the many discussions I have had and the presentations I have made, I think that all of these boil down into one very simple thing – my work is all about helping people understand the true nature of the software development business process and, thereby helping them to make better decisions. Understanding our cognitive biases, therefore, is extremely important for my clients and myself because, in the end, Agile is all about making better decisions.

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Android | RSS