Agile Adoption: Improving Software Quality

By Gemba Systems

Faster, better, cheaper. That's what we must do to survive. The Time to Market Refcard addresses faster, the Reduce Cost Refcard addresses cheaper, and this Refcard addresses better. This is about improving the quality of your software; that means reducing bugs and improving design.

The vast majority of software projects suffer from a steady degradation of design quality and it becomes more and more difficult to maintain the software with the same level of quality. As the software ages it calcifies and becomes harder and harder to maintain. In some cases it becomes too expensive to maintain and so it the software is put to rest and rewritten. In others, the software is released with a steadily increasing number of defects. Both of these common situations are deeply unsatisfying, but there is another way.

Many of the practices from the Agile world stop the degradation of software quality and turn the trend around. It is not unheard of for teams to have maintained a zero-defect status for months and years. Design and architecture have become malleable; they now emerge and transform over time. In fact, Gartner now recommends an emergent approach to enterprise architecture (http://www.gartner.com/it/page.jsp?id=1124112).

Figure 1 Practices that help improve the quality that your software development team(s) builds.

Figure 1: Improve Quality Practices

You will be able to use this refcard to get a 50,000 ft view of what will be involved to incrementally improve the quality of your software.

The Agile community has been fertile ground for quality improvements in software development.

There are four major strategies that can help you improve the quality of your software:

Reduce Defects

Reducing defects is the first thing that comes to mind when examining the quality of software. A low defect count is often synonymous with high quality software. Defects are also the most visible sign of quality problems.

Improve Design

Design is the model that a development team builds and maintains. High quality design makes for an application that is easy to understand and change as new requirements are discovered. Traditionally, the team has one shot to get the design right and then it degrades over time as it is patched over time. Agile practices, however, give an alternative; using practices like test driven development and refactoring teams are now able to continuously improve the design of their system.

Theory Building

One way to look at software development is 'theory building'. That is, programs are theories models of the world mapped onto software in the head of the individuals of the development team. Great teams have a shared understanding of how the software system represents the world. Therefore they know where to modify the code when a requirement change occurs, they know exactly where to go hunting for a bug that has been found, and they communicate well with each other about the world and the software.

Conversely, a team that does not have a shared 'theory' makes communication mistakes all the time. The customer may say something that the business analyst misunderstands because she has a different worldview. She may, in turn, have a different understanding than the developers, so the software ends up addressing a different problem or, after several trials, errors and frustrations, the right problem but very awkwardly. Software where the theory of the team does not match, or even worse, the theory is now lost because the original software team is long-gone, degrades in quality as design changes are made that don't fit with the theory, or even just as bad, cut & paste work is done because the theory is not understood. Building a shared theory of the world-to-software-mapping is a human process that is best done face-to-face by trial and error with ample time.

Build Less

It has been shown that we build many more features than are actually used. In fact, we can see in Figure 2 that most functionality we do build is never used. So, one very effective way to improve the quality of our software is to build less of it. It makes it easier to understand, gives us more time to focus on the important parts that are actually used, and almost always has fewer defects.

Figure 2: Functionality Usage, only about 20% of functionality we build is used often or always. More than 60% of all functionality built in software is rarely or never used! One way to improve the quality of software is to write less code which makes it easier to understand and maintain. There are several Agile practices that help you get to that point.

The four strategies above: maintain the theory of the code, build less, building less and improving the design are not independent.

Figure 3: Strategy Dependencies

Maintaining the theory of the code makes it easier to modify the design because of a greater understanding of the existing design and also directly affects the number of defects and the difficulty in fixing those defects once found. Improving the design also makes it easier address defects by being inherently easier to change. And building less code makes it easier to understand and communicate the theory of the code and is directly related to the number of defects in the system.

Test Driven Requirements
	Test Driven Requirements call for the customer to provide requirements in an unambiguous format, usually an acceptance test, at the beginning of the iteration.
	Test driven requirements drive the architecture of the system much like test driven development drives the design. They also help developers only build what is needed and maintain the theory of the code as up-todate executable requirements.
	Test driven requirements needs a customer who is willing and able to participate more fully as part of the development team. Your team will also be willing to make difficult changes to the code to accommodate for testing. Finally, you are willing to pay the steep price of the learning curve for this practice (which is well worth it).

The remaining practices also help improve the quality software development. Because of the limited size of the Refcard, we will only summarize them below.

To successfully adopt Agile practices let's start by answering the question "which ones first?" Once we have a general idea of how to choose the first practices there are other considerations. Then, once you've chosen the first practices that best fit your environment, you and your team(s) will need to be aware of the mindset you'll need to get the most out of the practices you choose.

Choosing a Practice to Adopt

Choosing a practice comes down to finding the highest value practice that will fit into your context. Figure 4 contains practices that help improve the quality that your software development team(s) builds. Figure 4 will also guide you in determining which practices are most effective in increasing the quality of your software and will also give you an understanding of the dependencies. The other parts in this section discuss other ideas that can help you refine your choices. Armed with this information:

Figure 4: Steps for Choosing and Implementing Practices

For improving quality, the set of practices that will give you the most value are those nearest the top of Figure 1. Four of the practices are independent: done state, automated developer tests, automated acceptance tests, and pair programming. Consider adopting pair programming as a support practice to the other three practices, then take them on one or two at a time. Next on your list (or maybe even concurrent) to consider should be done state and automated developer tests, and then finally automated acceptance test (probably the most difficult of this set to adopt correctly).

The Mindset

Be Disciplined, Confront Issues, Respond Positively to Pain

The practices involved in improving the quality to market are some of the most difficult to do from the body of Agile practices. Things will get harder before they get easier. The first rule is to expect the difficulty, be patient, and don't stop the practices just because they uncover significant problems; be disciplined in your practice. Once you start a practice give it a chance because you will slow down and confront frustrations before speeding up. For example, pair programming is frequently seen as a waste of resources and uncomfortable to many developers who are used to (and enjoy) working alone. Consider giving it a chance by agreeing as a team to practice pair programming for a couple of months before deciding whether it is worth adopting permanently.

Figure 5: Learning Curve

Figure 5 The J-curve is what to expect when adopting new practices. First things will be hard and you will be less productive; stay with it and it will improve.

Confront issues when they come up instead of stopping a practice because it is 'too painful'. Deal with pain differently than you are used to; instead of discontinuing something painful, examine it and find the source. Often Agile practices will uncover problems that have always been there but have not been felt. Feeling the pain is a chance to correct a problem and improve towards your goal of increased quality. A good example of this happens when teams start adopting done states for the first time. There is no partial credit, either you are 100% done or 0% done. A team that adopts this for the first time frequently works on multiple features at a time and at the end of the iteration they have not fully completed any of the features. Therefore they are 0% done with all of their tasks. This is discouraging and painful and a common response is to stop doing the practice instead of examining the pain and looking for alternatives to correct the problems in the next iteration.

Get Good at Small Steps

Small steps are going to save your life with these practices because many are completely new ways of doing things that may slow you down and frustrate you as you are learning them. Take one practice, do it well, and do it regularly. You might consider pair programming along with any and all of the practices to make it easier and keep you on-track. How do you know you are doing a practice well? You get the value that you originally hoped to get - i.e. the quality of your software noticeably increases. You also have confronted pains and learned from them. If a practice is completely easy and comfortable from the get-go, or has not noticeably improved the quality of your work then you probably are not done yet.

Be Prepared to "Suspend Your Disbelief"

Much of what you will be doing will not make immediate sense. It will feel that you are doing things that are more trouble than they are worth. For example writing your tests first, before writing your code in the automated developer tests practice is non-intuitive. What can you possibly gain by doing things backward? Those who have successfully adopted this practice have "suspended their disbelief" and done it anyway. After experientially learning the practice they then made their judgments about its utility and usually kept doing it because they saw the value.

Know What You Don't Know

The Dreyfus Model of Skill Acquisition, is a useful way to look at how we learn skills such as learning Agile practices necessary to improve quality. It is not the only model of learning, but it is consistent, has been effective, and works well for our purposes. This model states that there are levels that one goes through as they learn a skill and that your level for different skills can and will be different. Depending on the level you are at, you have different needs and abilities. An understanding of this model is not crucial to learning a skill; after all, we've been learning long before this model existed. However, being aware of this model can help us and our team(s) learn effectively.

So let's take a closer look at the different skill levels in the Dreyfus Model:

Figure 6 The Drefyus Model for skill acquisition. One starts as a novice and through experience and learning advances towards expertise.

How can the Dreyfus Model help in an organization that is adopting agile methods? First, we must realize that this model is per skill, so we are not competent in everything. Secondly, if Agile is new to us, which it probably is, then we are novices or advanced beginners; we need to search for rules and not break them until we have enough experience under our belts. Moreover, since everything really does depend on context, and we are not qualified to deal with context as novices and advanced beginners, we had better get access to some people who are experts or at least proficient to help guide us in choosing the right agile practices for our particular context. Finally, we'd better find it in ourselves to be humble and know what we don't know to keep from derailing the possible benefits of this new method. And we need to be patient with ourselves and with our colleagues. Learning new skills will take time, and that is OK.

This Refcard is a quick introduction to Agile practices that can help you improve the quality of your software by reducing defects, improving design, sharing the theory of the code and building less. It includes an introduction of how to choose the practices for your organizational context. It is only a starting point. If you choose to embark on an Agile adoption initiative, your next step is to educate yourself and get as much help as you can afford. Books and user groups are a beginning. If you can, find an expert to join your team(s). Remember, if you are new to Agile, then you are a novice or advanced beginner and are not capable of making an informed decision about tailoring practices to your context.