The Book about Cost Estimation in Agile Software Development

Agile software development and a binding agreed delivery date are not mutually exclusive at all. On the contrary, the principles of the Agile Manifesto are well-proven measures to mitigate the typical risks of development projects with binding regulations.

Crucial for the achievement of project goals is a reliable plan which can be created without significant expense and readjusted flexibly in case of changed requirements. Under these conditions, indirect cost estimations on basis of functional size measurements are a proven practice. They require the size of the functional requirements as an input value, determined by precise rules, as well as an empirical value of the own productivity, measured after earlier sprints.

To the Shop

For the determination of functional size, methodological knowledge and expertise is required regarding the selected method, for example Function Point, COSMIC or Data Interaction Point method, along with a sufficient specification of requirements, for example by described use cases and elementary processes. Under these conditions, the resulting value is a valid key figure of the functional requirements’ size. It is independent from the individual performing the estimation. Repeated estimations of the same requirements always result in the same value.

An empirical value of the own productivity, that is the functional size which can be typically implemented by the team with given personnel costs, is required in addi-tion. It can be measured and updated within the scope of regular retrospectives of completed increments, sprints or releases. Thereby, the implemented functional size is set in relation to the arising personnel costs. The functional size is determined by counting – depending on the method – elementary processes or data elements. With an appropriate mapping to structural characteristics of the system, this counting process can be automated. A such counting program, once implemented, can quickly measure the total size of the system and, by comparing the current size with the previous measurement, return the growth of the functional size resulting from the last development process. 

Regularly measured and compared values of productivity and quality - in the simplest case the number of production defects – aid in recognizing any need for action and allow to verify the effectivity of implemented improvement measures.

This book illustrates, how size metrics can be utilised profitably in software development processes oriented towards agile values. It points out differences and restrictions, how the accuracy of cost estimations can be increased with each sprint and examines the feasibility of automated measurements. It is available as eBook, paperback and hardcover at most bookshops, for example at Amazon.com.

When is Software Development agile?

The following aspects characterise agile software development :

• Parts of a system are developed at different times. The system is always being enhanced by already accomplished parts. This is called incremental development.

• Defect analyses and measurements are used as learning opportunity by team members and for improvements of the organization.

• All involved parties / persons work close together and cooperate directly, e.g. by a high degree of teamwork and a continuous cooperation of the customer or product owner. 

Actually models for agile development exist since the 90s. They follow processes and, due to the mostly short iteration cycles (sprints), they heavily rely on measurements of progress, code quality, test coverage, etc. and a fast feedback of it to the developers.

The Effectiveness of Improvement Measures

In my post Measures for optimizing Productivity I pointed out some key areas with a high leverage for productivity improvements. In business often the cost-to-benefit ratio is another important criteria to find the right measures.

Diagram 1

Thereby the breadth or scope of a measure plays an important role: How large is the percentage of the total development process being affected by the measure? Example: An action which doubles the productivity but only in a part of a project which accounts for 50% of the total effort (blue area of diagram 1) increases the productivity of the overall project only by 33%. Explanation: Productivity and effort are reciprocal, i.e. increasing the productivity results in a reduction of the effort and vice versa (see: Measuring Productivity in Software Development). The mentioned action reduces the effort in a sub-project with 50% of the total effort by 50%. This results in an effort reduction of 25% for the overall project, i.e. 75% (3/4) of the effort remains. Due to the reciprocity the productivity is 4/3 of the original value, which is a raise of 33%.

Diagram 2

The productivity improvement being effective after multiple actions have been performed depends of how these actions influence each other. As long as they are independent the resulting productivity can be calculated by setting each of the according effort reductions in relationship to their percentage of the total effort. Thus leads to the remaining effort and hence to the improvement of the productivity. Example: Diagram 2 shows how two different measures increase the productivity each by 100% in completely different areas of a project (blue and green), each area accounting for 25% of the total project effort. The resulting effort is in total 3/4 of the original, i.e. the productivity is 4/3 and has been increased by 33%.

Diagram 3

For improvement measures affecting the same area and influencing each other their effort reductions can be multiplied. Example: Diagram 3 shows how two actions increase the productivity each by 100% (halves the effort) in the same area (blue) accounting for 25% of the total project effort. The resulting effort in the affected area can be calculated by 50% * 50% = 25% (a quarter) of the original effort, i.e. the total effort of the overall project is now 13/16 of the original and therefore the productivity 16/13. The productivity has been increased by about 23%.


The following lessons can be drawn from these examples:

• Improvement measures are more effective the higher the effect on the entire process is (percentage of the total effort).
• Any productivity improvement resulting by multiple measures depends on how these measures influence each other. As a general rule: The more independent they are, the more effective they are in total. Unfortunately this is difficult to predict. Usually only a range can be determined for the expected improvement.

Measures for optimizing Productivity

From experience the productivity of software development is affected by the following key areas - in different intensity and without any claim to comprehensiveness:

• Knowledge and experiences of the involved staff

• Their motivation, especially the willingness to increase productivity

• Knowledge management, i.e. the ability to make the required knowledge available for staff at the right time

• Risk management, i.e. the ability to anticipate negative influences on productivity and to take countermeasures prematurely

• Constructive quality assurance, e.g. by predefined processes, methods, guidelines which give process security, or by standards and reusable components, which help reducing development and testing efforts

• Automation, e.g. code generation in the area of implementation and test automation in the area of analytical qa. Automated testing reduces the test effort, at the same time it increases the reliability of test results and by early and continuous tests it enables quality improvements (see: Are Quality Improvements always reducing the Productivity?).

In these areas it is worthwhile to look out for improvement measures.

Also Measurements can help creating Quality

Additionally to root cause analyses of defects (see: Defects can help creating quality) steady measurements and benchmarks are an important basis for continuous improvements. The following three metrics can already cover the most important aspects:

• Productivity (Cost Efficiency): The performance of the development process until the achievement of a predefined (outer and inner) quality ready for delivery (see: Measuring Productivity in Software Development).

• Defect Rate (Defect Density): The quotient from the number of production defects per time unit and the size of the system. It must be defined which types of defects are relevant for the defect rate.

• Maintenance Effort/Costs: The quotient from the effort for maintenance tasks per time unit and the size of the system. Ideally differentiated according to task types as defect analysis, defect correction, advisory services, etc.

For each of these aspects target values resp. thresholds (baselines) are important, where exceeding indicates a need for clarification or, in the next step, a need for immediate action. Finding these thresholds will be facilitated through benchmarks of different systems, projects or shops, where measurements will be performed by using exactly the same methods.

The easiest way is using the mean value or, if extreme values shall not be considered, the median as a threshold. In cases where a measured value is worse than the mean value or the median of all values an analysis or assessment helps finding the root causes. Mostly root causes lead to improvement measures. Their effectiveness becomes transparent by the course of subsequent measurements.

Defects can help creating Quality

Even if software defects are not considered as ‘normal’ (see: Is Software with Zero-Defect Quality possible?), every defect is a chance for improvement.

For each defect a root cause analysis should be performed. This does not mean superficial causes like ‘programming error’. A proven method for the root cause analysis is ‘5 Why’, where you ask multiple (not necessarily five) times for the reason. Only when you reached the beginning of the cause-and-effect chain that there is no practical answer to the ‘why?’ question, the root cause most likely has been found.

At this point it is important that the root cause of the defect matches with a defect category. The list of defect categories should be standardized company-wide. Technically these categories can be implemented as a pick-list in the ticketing or defect tracking system. The standardization is the key for evaluations like categories which are defect hotspots, which is important to see where improvement measures are most effective.

With knowledge of the root causes improvement measures should be found which prevent the recurrence of such defects. This can be improvements of the constructive quality assurance, e.g. development methods or tools, possibly changes of the team, or regarding other areas. Of course there may be some defects which can not be prevented in the future by any measures.

A continuous improvement process on basis of defect analyses will only work if anybody is responsible for the communication of the found measures, plans their implementation, keeps track of it and checks the effectiveness after implementation.

This process, which similarly is in use in the car industry since the 70s, helps preventing defects in the future, which is a better investment than in analyses, tracking and corrections of defects. In other words: It creates quality.

Using Sonar for steady Analyses of the Code Quality

The platform Sonar allows static code analyses of different projects and presents the results in a standardised form. Sonar applies some own rules enhanced by the rules of integrated tools as e.g. FindBugs, Checkstyle or PMD (for the Java world).

The experiences with Sonar used in a huge portfolio of applications are good. For Java there are more than 800 rules available, which are well described, can be activated and deactivated and can be configured regarding the severity of violations (e.g. Blocker, Critical, etc.) or parameters. Experience has shown that it is better not to use the standard configuration but to discus an own configuration with the most experienced developers and to rate less rather than more rules as blockers. Further it is important that the process model respectively the quality gates clearly define that e.g. it is forbidden to use a release as long as there are blockers, which at least are potential production defects.

Sonar Dashboard showing the results of a Code Analysis (Sample)

Additionally to these rules Sonar measures the comment density, the rate of commented or duplicated code, the cyclomatic complexity, the lack of cohesion in methods (LCOM4) and the class complexity (response for a class, RFC), which are proven indicators for the code quality. Based on an own coding styleguide, which should clearly define threshold values for these metrics, a Sonar profile can be used to set these thresholds for an indication of warnings and errors for each metric. This makes sure that each application being analyzed will be measured and rated by the same standard.

Best practise is to integrate the Sonar code analysis in the build processes of the different applications. Then, after each new build, i.e. each night or continuously - according to the configuration, a new analysis result is available.

The Sonar web client presents the analysis results in dashboards and even in case of a huge portfolio of applications clearly structured. There are tools for e.g. the analysis of time courses or hotspots. The authorized user can drill-down for each violation into the code and, if required, declare it as false-positive.

The experience is that only with a tool such as Sonar the definition of a coding styleguide is really worthwhile, because the compliance can be checked without extensive time and in short intervals.

Information about Sonar can be found at http://www.sonarqube.org/features/.

Is Software with Zero-Defect Quality possible?

The expression of a zero-defect quality has been coined by Philip B. Cosby in the early 1960s. TQM (Total Quality Management) as well as Six Sigma consider a zero-defect production as desirable objective.

In software development zero-defect quality would mean that – depending on the definition, what a defect is – all explicit (described in a requirements specification) and all implicit requirements (e.g. typical expectations of non-functional characteristics as robustness, usability, etc.) are fulfilled. Due to the size and the complexity of software, but often also as a debt of a fuzzy or incomplete requirements specification, it is not feasible with reasonable expenses to verify that software is free of defects.

However zero-defect quality should be understood as an ideal goal. The issue is the common understanding of quality by all persons involved in software development. Software defects must not be considered as ‘normal’, i.e. a defect rate greater than zero is not acceptable. This should not lead to punishing developers who are responsible for defects, but better to a general root cause analysis for each defect to find out by which measures similar defects can be avoided in the future. This is the key for a continuous improvement of all areas related to software development, which definitely helps to approach to a zero-defect quality – even if it is not possible to reach it.