Mastering Opportunities and Risks in IT Projects (Book Excerpt)

Identifying, anticipating and controlling opportunities and risks: A model for effective management in IT development and operation.

Management systems show the management objectives (in relation to a specific purpose), proven methods for achieving the objectives and the associated control and monitoring mechanisms. The purpose can be the management of a company, an IT project or just compliance with a quality, environmental or information security standard. This book uses a model to describe how the fundamentally necessary core process of risk management works within such a management system. Its main feature is the cyclical repetition of the identification and evaluation of opportunities and risks, which leads to necessary control measures influencing them and to the subsequent implementation of measures to improve the effectiveness and efficiency of the management system.

Each cycle begins with the definition of key goals for the purpose intended and a review of the decisions already made. This is followed by the identification of influencing factors that threaten or even favor the achievement of goals - and their assessment. As the basis for focusing on key goals and acute threats, this step is an important prerequisite for economic risk management. 

In order to find influencing factors that favor or threaten the achievement of the selected goals, the combination of a methodical cause-and-effect analysis with own empirical values is recommended. The risk portfolio comprises the set of all influencing factors. After an assessment of their probability of occurrence as well as of the amount and type of potential damage that may occur if a goal is not achieved, this enables the focus to be placed on a subset of the threats, which in the model described is referred to as the risk profile.

Model for managing opportunities and risks

The risk profile reassessed in the respective cycle forms the basis for management decisions on the treatment of risks. This treatment can, for example, consist of taking out insurance (risk transfer), outsourcing risky areas from the IT project under consideration to other parts of the organization (restructuring) or implementing and following up measures to actively reduce risk. If, in individual cases, the risk owners do not consider it necessary to reduce the risks by means of a transfer, restructuring or measures, this must be documented as acceptance of the residual risks.

In the final step of each cycle, the measures that can be taken to improve the effectiveness of risk management and its cost/benefit ratio should be examined. This includes a review of all components of the model such as processes, methods, metrics and scales for evaluation, documentation, etc.

Considerable potential for improving the management model lies in making knowledge about general or organization-specific threats to the goal types explicit - and in permanently adapting this explicit knowledge to changes in the threat situation, for example. One way of implementing this explicit knowledge is the creation and maintenance of sets of rules that submit questionnaires to the people responsible for risk analysis and from whose responses corresponding entries in the risk profile result. However, these suggested values must be validated and possibly corrected on the basis of own human judgment and intuition. A further improvement is foreseeable as soon as machine learning systems can be used for risk analysis purposes.

It is possible to carry out risk management both effectively and efficiently: With good methodology, threats can be noticeably mitigated. Thereby risk management becomes effective. The avoided damage results in a benefit that can be calculated and placed in relation to the required effort. This results in a measure of efficiency. As with any other management process, both effectiveness and calculability of efficiency are an essential basis for optimization.

The book is available as e-book, paperback and hardcover (directly in the Tredition book shop incl. reading sample - or in bookstores).  

Book Series "Increasing Productivity in Software Development"

The PASS Group has been using key figures as the basis for its IT management in more than 10 different application landscapes with more than 500 customers and over 250,000 users for several years and is now publishing its experiences in the form of a series of books. In some areas, the regularly collected key figures demonstrate a 100-fold increase in delivery productivity compared to just a few years ago and they show in which areas further improvements are possible.


Why measure productivity?

IT has already changed almost all areas of life through fundamental innovations. Our future will be dominated by virtualization and smart helpers, i.e. things equipped with intelligence. Software is the stuff of innovation. Software development is a key competence.

Productivity and quality are critical success factors for companies developing software. Only with suitable measuring methods, regular measurements and rapid feedback of the measured values to the team and its management can the effort of planned developments be reliably determined and productivity and quality be continuously improved.

Part 1: "Productivity and Performance Measurement - Measurability and Methods"

Book Series "Increasing Productivity in Software Development"

The first book in the series first explains a practicable definition of productivity for software development and thus shows the conditions under which productivity can be measured at all. It describes the ISO/IEC 14143 standard as an important common basis for all modern function-oriented measurement methods. Some methods are explained in detail and compared with their advantages and disadvantages on the basis of practical examples - tried and tested as well as new ones. The book deals with the automatability of measurements, but also points out their limits, the exceeding of which leads to invalidity measurements due to a strong orientation towards design features instead of application cases.

Another topic is the influence of complexity on the scope to be measured, the different consideration of complexity through different measurement methods and the limits of measurability through function-oriented metrics, for example in systems with above-average algorithmic complexity.

Part 2: "Management Model, Cost Estimation and KPI Improvement"

The second book describes a model based on three key performance indicators (KPIs): Productivity (measured using the measurement methods described in the first book), cost and quality. It explains its cyclical survey, analytical evaluation and indicators, for example by comparing productivity and quality over time or as a result of methodological analyses of the causes of errors, which lead to improvement measures in important areas of influence in software development, so-called key performance areas (KPAs). In order to be able to assess the benefits of the measures in advance, it provides empirical values as well as a method for calculating their effectiveness. The model described is a navigation tool that always shows management in which direction and at what speed it moves with regard to its three KPIs.

The books are available as e-book, paperback and hardcover (in the PASS book shop or at Amazon.com).

Direct and indirect Cost Estimation Methods (3)

Indirect Estimations by measuring the Functional Size

A different type of an indirect estimation is determining the functional size (instead of Story Points) and relating it to an empirical value describing which size can be implemented with specific personnel costs (often called productivity or efficiency). The basics for measuring the functional size are defined in the industry standard ISO/IEC 14143. They require the refinement of user stories to use cases.

Sample Use Case Diagram

A use case stands for a system behavior perceptible by an actor from outside of the system boundaries. Actors can be human system users as well as other systems or machines (hardware or software). An example is the use case diagram of a (highly simplified) Internet Booking Engine for flights shown in the accompanying picture. Within the system boundaries (shown as a rectangle with the label "Internet Booking Engine”) use cases are represented by ellipses. They are related to (used by) actors, which in the example below are a traveler and a Computer Reservation System (CRS). Their relation is indicated by lines.

Each use case stands for actions which can be described by base functional components (BFCs) respectively elementary processes. The use case “search flight” could, for example, consist of the following (simplified) elementary processes:

1. Traveler calls the dialog Flight Search.
2. Traveler enters the departure date.
3. Traveler enters the first letters of the destination (name or code).
4. System looks up for matching airports in the database and displays a list of found items showing names and codes.
5. Traveler selects a list entry and clicks on the button “Search”.
6. System sends a message of type “Flight Search Request” including the departure date and airport code to the CRS.
7. System receives a message of type “Flight Search Response” from the CRS and reads the fields departure time, arrival time, airline, flight number, class, price and currency of all flight records included.
8. System displays a table showing this information about all received flight records.

With the knowledge of these BFCs or elementary processes many measuring methods can be used to determine the functional size and, with the aid of an empirical value of the own productivity, the expected development costs.

Alternatives:

    Expert Estimations
    Indirect Estimations with Story Points

Book recommendation: "Cost Estimation in Agile Software Development"

Agile Principles are Risk Mitigation Methods

Many risks of commercial software development can be mitigated by the principles of agile development:

Risk: The requirements of the customer are not covered completely. His expectations for example regarding usability have not been fulfilled. 
Measures: The product is developed incrementally, with the cycles being as short as possible. The result of each cycle is operable software, which can be checked by the customer regarding the fulfillment of his expectations. The customer, in particular his business and technical professionals, collaborate closely and intensively with the development team.


Risk: Objectives and requirements of the customer change, before the development is completed, for example as a reaction to unexpected market changes.
Measures: One principle of the Agile Manifesto is: “Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.” Thus, agile software development is generally characterized by the willingness to consider new or changed requirements. However, this does not preclude a consensus between the contract parties regarding the commercial effects of such changes. 

Risks: Wrong assessment of the development progress, delays due to communication problems, loss of know-how by staff turnover.
Measures: In agile development processes these risks are mitigated by a high level of teamwork. In short daily team meetings problems can be addressed, the progress can be estimated together, pending tasks can be assigned or even tasks in progress can be re-assigned if it seems reasonable. The avoidance of too rigid role and task assignments makes the team robust toward the loss of individuals. 

Risk: Additional expenses due to quality deficiencies which have been discovered too late. 
Measures: One characteristic of agile development are automated processes for the daily build of an executable system, where an initial code analysis is performed, followed by functionality checks of particular components using unit tests and, if possible, the execution of scripted test cases. 


Risk: Invalid planning of the manpower required to fulfil a development contract within the agreed time.
Measure: This risk can be countered by applying a cost estimation method with accuracy improving with every retrospective. Such methods are described in detail in my book "Cost Estimation in Agile Software Development".

The Book about Cost Estimation in Agile Software Development

Agile software development and a binding agreed delivery date are not mutually exclusive at all. On the contrary, the principles of the Agile Manifesto are well-proven measures to mitigate the typical risks of development projects with binding regulations.

Crucial for the achievement of project goals is a reliable plan which can be created without significant expense and readjusted flexibly in case of changed requirements. Under these conditions, indirect cost estimations on basis of functional size measurements are a proven practice. They require the size of the functional requirements as an input value, determined by precise rules, as well as an empirical value of the own productivity, measured after earlier sprints.

To the Shop

For the determination of functional size, methodological knowledge and expertise is required regarding the selected method, for example Function Point, COSMIC or Data Interaction Point method, along with a sufficient specification of requirements, for example by described use cases and elementary processes. Under these conditions, the resulting value is a valid key figure of the functional requirements’ size. It is independent from the individual performing the estimation. Repeated estimations of the same requirements always result in the same value.

An empirical value of the own productivity, that is the functional size which can be typically implemented by the team with given personnel costs, is required in addi-tion. It can be measured and updated within the scope of regular retrospectives of completed increments, sprints or releases. Thereby, the implemented functional size is set in relation to the arising personnel costs. The functional size is determined by counting – depending on the method – elementary processes or data elements. With an appropriate mapping to structural characteristics of the system, this counting process can be automated. A such counting program, once implemented, can quickly measure the total size of the system and, by comparing the current size with the previous measurement, return the growth of the functional size resulting from the last development process. 

Regularly measured and compared values of productivity and quality - in the simplest case the number of production defects – aid in recognizing any need for action and allow to verify the effectivity of implemented improvement measures.

This book illustrates, how size metrics can be utilised profitably in software development processes oriented towards agile values. It points out differences and restrictions, how the accuracy of cost estimations can be increased with each sprint and examines the feasibility of automated measurements. It is available as eBook, paperback and hardcover at most bookshops, for example at Amazon.com.

The Impact of Complexity

In software development complexity is defined as the effort for understanding a program or algorithm. Consequently the effort for the new or further development of software is not only reliant on the number of user functions, data structures, data elements, etc. (depending of the measuring method) but also of the complexity. However it is necessary to distinguish between different types of complexity.

The Complexity of an Implementation represents the effort for understanding the code and design of an application. It is important for estimating the costs for the maintenance and further development of existing systems. There are multiple metrics measuring different aspects of the complexity inherent in the code: The McCabe metric measures the cyclomatic complexity, the metric of Halstead the textual-lexical complexity, the RfC metric the class complexity, etc. For the cost estimation of a planned new development project this type of complexity has no significance because it accrues from the development process. Furthermore different implementations of the same use case can have different levels of complexity.

The complexity of an implementation has no significance for functional measurement, but the interactional Complexity has. It represents the effort for understanding and implementing the steps of the use case scenarios. The Function Point Analysis derives it from structural parameters as e.g. the number of data element types and record element types related to user functions. The Data Interaction Point Method considers the complexity related to use cases according to the usage of the counted data elements, e.g. for input/output, persistence or read-only access, etc. The COSMIC Method does not consider complexity at all, which is a disadvantage.

All functional size metrics which are conformal to the standard ISO/IEC 14143 ignore the algorithmic Complexity of the business logic lying in processes and functions of the system. They all are based on the assumption that algorithmic complexity has secondary importance towards the functional size. Actually at most systems the algorithmic complexity is limited to input validations, data inquiries, output processing, less complex logical operations and calculations, etc. Usually the number of such functions correlates with the number of interactions with the system's actors. Hence the error due to ignoring the algorithmic complexity is generally negligible. However this does not apply for systems with primarily algorithmic processing, e.g. a route planning program: Its input is a start location and a destination, the output a list of route segments. Every functional size metric would consider only the user interactions and measure a small functional size. Since the development of the undoubtedly highly complex planning algorithms requires high expenditures, the calculation of productivity on basis of the small functional size would indicate a very low productivity compared with other systems - which is incorrect. Conclusion: Functional size measurement is not suitable for systems with an above-average algorithmic complexity.

Economical Measurements (The Limits of Automation)

The functional size of an application should be measured regularly in order to determine the growth and hence the development productivity of each release or increment. Only by comparing cyclic measurements, improving or worsening effects can be identified. Furthermore the organisation gets empirical productivity values which are the key for cost estimations of new development projects (see: Measuring Productivity in Software Development). 

Experience shows that cyclic measurements will be neglected if the accociated effort is too high. Measuring techniques based on counting rules allow a high degree of automation by mapping the objects to be counted to design features of an existing application and implementing the according scripts or programs for counting. The investment in implementing these script or programs quickly pays for itself when repeated measurements do not require anymore manual effort. Possible targets for automated counting are:

• Dialog models represented by XML or XHTML files as being used by many GUI frameworks

• XML schemas defining interface or message structures

• Meta data of a DBMS including information about tables and attributes

When applying automated measurement processes which have a stronger focus on design features than on the use cases of a system there is a risk of inaccurateness if counting objects have been used multiple times by the application design. An example of this is the comparison of two applications A and B, which are different implementations of the same use cases:

• At application A similiar dialogs have been implemented separately and partially redundant.

• At application B a generic dialog has been implemented which automatically adapts according to the use case respectively invocation. 

An automated size measurement based on design features will result in a lower value for application B, which is wrong because both applications are implementations of the same use cases. In the worst case this will promote bad development practice (copy & paste) and poor design, if developers try to increase the measured productivity by creating redundant counting objects. Actually this is fatal in several ways, because the reuse of components and services is one of the most important keys to sustainably increase productivity and improve maintainability.

Conclusion: Automated measuring techniques based on design features are necessary to make size measurements practicable at all. However a side effect is the limited comparability of applications with different degrees of reutilization.

The COSMIC Method

COSMIC is a size metric (see Methods for measuring the Size of Software) defined by the standard ISO/IEC 19761. The method is conformal to the standard for Functional Size Measurement (ISO/IEC 14143) and thereby it considers the use cases of a system (similar to the Function Point Analysis).

Objects to be counted by the COSMIC method

COSMIC uses an own definition of actors, the so-called "Functional Users", which can be - additionally to human users and external systems - any hardware or software components sending data to the software to be measured across the system boundaries and thereby triggering a functional process, or receiving and processing data from a functional process of the software. Depending on the direction these data movements are called "entries" or "exits". Additionally, the method counts accesses to data elements which are stored in the database ("read" or "write" operations) and related to the functional processes.

An advantage of the COSMIC method is that it counts data elements directly instead of larger data structures or elementary processes as the Function Point Analysis does (see also Is the Function Point Method still up-to-date?). Another advantage is that regarding the database it considers data movements performed by the functional processes, i.e. read and write operations, and not the static size of the database. These characteristics along with the broad definition of "Functional Users" makes COSMIC suitable for measuring the functional size of real-time systems.

A disadvantage can be that all entries, exits, reads and writes have the same weight and therefore differences in the complexity of their pre- or post-processing can not be considered.      

Software Development with Methods of the industrial Production

Methods of industrial production may be a response to 

• increasing complexity of requirements,
• shorter development cycles and
• growing cost pressure.

This has nothing to do with the series production of the same product. It more concerns the following aspects:

• Standardization of system and application architecture 
• Reusability of components
• Automation  of development and quality assurance processes
• Measurability of productivity and quality

While today a lot of standards, concepts for reusability and tools for automation are available and established in software development, many companies neglect measurements or use estimations instead. However regular and accurate measurements of productivity and quality are an important basis for the reliable determination of the effort for upcoming development projects (see also Measuring Productivity in Software Development).

The Impact of Technical Debt on Productivity

Some measures for raising the productivity are not very recommendable:

• Incomplete implementation of requirements
• Postponed corrections of known defects
• Disregard of commentation and technical design
• Insufficient or missing documentation
• Insufficient test coverage
• Build and deploy processes too complex and not automatable

A development project increasing the productivity by such measures takes on technical debt which must be paid back including interest after this project is completed - while the further development. Interest payments are:

• increasing effort for further development, which results in decreasing productivity,
• above-average effort for defect analyses and corrections.

The impact of technical debt may look similar to the diagram below. After completion of a development project and production start of an application (Rel. 1.0 in this example) the productivity measured for the further development (Releases 1.1, 1.2, etc.) is decreasing continuously (see: Measuring Productivity in Software Development), provided that the input considers the total expenses including these for defect analyses and corrections. 

Course of Defect Density and Productivity in case of technical Debt (Sample)

Steady code analyses are suitable to detect symptoms of technical debt almost early, already while the initial development project (see: Using Sonar for steady Analyses of the Code Quality). 

My post Measures for optimizing Productivity shows better measures for increasing the productivity.

The PASS Data Interaction Point Method (DIP Method)

The Data Interaction Point method of the PASS Consulting Group is a function-oriented size metric (see: Methods for measuring the Size of Software). It counts data elements which are involved in interactions of the considered system with the actors of its use cases, which can be human users as well as other systems. Complexity is considered by different weights of the counts, e.g. when input elements of a dialog have a higher weight than display elements.

Interactions to be counted by the
Data Interaction Point Method

The basic principles, i.e. the orientation on use cases and the counting of data objects from the external view of the actors are the same on which the Function Point Analysis (FPA) and the COSMIC method are based on. However, a major difference to the FPA is the level of detail for counting. FPA counts e.g. data structures, where the score depends of their number of data element types and record element types defined by open-ended interval scales with only three steps. The DIP and the COSMIC method directly count data elements. While COSMIC does not distinguish score values the DIP method derives them by the usage of data elements, e.g. whether they are for read-only or for read-write access, and thus oriented towards the interactional complexity.

The DIP method is better suited for benchmarking size-related KPIs, e.g. the productivity (see: KPIs for Controlling in Software Maintenance and Development).

The Effectiveness of Improvement Measures

In my post Measures for optimizing Productivity I pointed out some key areas with a high leverage for productivity improvements. In business often the cost-to-benefit ratio is another important criteria to find the right measures.

Diagram 1

Thereby the breadth or scope of a measure plays an important role: How large is the percentage of the total development process being affected by the measure? Example: An action which doubles the productivity but only in a part of a project which accounts for 50% of the total effort (blue area of diagram 1) increases the productivity of the overall project only by 33%. Explanation: Productivity and effort are reciprocal, i.e. increasing the productivity results in a reduction of the effort and vice versa (see: Measuring Productivity in Software Development). The mentioned action reduces the effort in a sub-project with 50% of the total effort by 50%. This results in an effort reduction of 25% for the overall project, i.e. 75% (3/4) of the effort remains. Due to the reciprocity the productivity is 4/3 of the original value, which is a raise of 33%.

Diagram 2

The productivity improvement being effective after multiple actions have been performed depends of how these actions influence each other. As long as they are independent the resulting productivity can be calculated by setting each of the according effort reductions in relationship to their percentage of the total effort. Thus leads to the remaining effort and hence to the improvement of the productivity. Example: Diagram 2 shows how two different measures increase the productivity each by 100% in completely different areas of a project (blue and green), each area accounting for 25% of the total project effort. The resulting effort is in total 3/4 of the original, i.e. the productivity is 4/3 and has been increased by 33%.

Diagram 3

For improvement measures affecting the same area and influencing each other their effort reductions can be multiplied. Example: Diagram 3 shows how two actions increase the productivity each by 100% (halves the effort) in the same area (blue) accounting for 25% of the total project effort. The resulting effort in the affected area can be calculated by 50% * 50% = 25% (a quarter) of the original effort, i.e. the total effort of the overall project is now 13/16 of the original and therefore the productivity 16/13. The productivity has been increased by about 23%.


The following lessons can be drawn from these examples:

• Improvement measures are more effective the higher the effect on the entire process is (percentage of the total effort).
• Any productivity improvement resulting by multiple measures depends on how these measures influence each other. As a general rule: The more independent they are, the more effective they are in total. Unfortunately this is difficult to predict. Usually only a range can be determined for the expected improvement.

Measures for optimizing Productivity

From experience the productivity of software development is affected by the following key areas - in different intensity and without any claim to comprehensiveness:

• Knowledge and experiences of the involved staff

• Their motivation, especially the willingness to increase productivity

• Knowledge management, i.e. the ability to make the required knowledge available for staff at the right time

• Risk management, i.e. the ability to anticipate negative influences on productivity and to take countermeasures prematurely

• Constructive quality assurance, e.g. by predefined processes, methods, guidelines which give process security, or by standards and reusable components, which help reducing development and testing efforts

• Automation, e.g. code generation in the area of implementation and test automation in the area of analytical qa. Automated testing reduces the test effort, at the same time it increases the reliability of test results and by early and continuous tests it enables quality improvements (see: Are Quality Improvements always reducing the Productivity?).

In these areas it is worthwhile to look out for improvement measures.

Also Measurements can help creating Quality

Additionally to root cause analyses of defects (see: Defects can help creating quality) steady measurements and benchmarks are an important basis for continuous improvements. The following three metrics can already cover the most important aspects:

• Productivity (Cost Efficiency): The performance of the development process until the achievement of a predefined (outer and inner) quality ready for delivery (see: Measuring Productivity in Software Development).

• Defect Rate (Defect Density): The quotient from the number of production defects per time unit and the size of the system. It must be defined which types of defects are relevant for the defect rate.

• Maintenance Effort/Costs: The quotient from the effort for maintenance tasks per time unit and the size of the system. Ideally differentiated according to task types as defect analysis, defect correction, advisory services, etc.

For each of these aspects target values resp. thresholds (baselines) are important, where exceeding indicates a need for clarification or, in the next step, a need for immediate action. Finding these thresholds will be facilitated through benchmarks of different systems, projects or shops, where measurements will be performed by using exactly the same methods.

The easiest way is using the mean value or, if extreme values shall not be considered, the median as a threshold. In cases where a measured value is worse than the mean value or the median of all values an analysis or assessment helps finding the root causes. Mostly root causes lead to improvement measures. Their effectiveness becomes transparent by the course of subsequent measurements.

Measuring the Productivity of IT Services

From an enterprise perspective the productivity of IT services is determined by the interplay of the following factors:

• Efficiency: Regards the economic efficiency concerning a cost/benefit ratio. When looking at a software development process the benefit corresponds with the output of created software and the costs are the development effort (input).
• Capacity: Depends on the provider’s flexibility in addressing changes in demand and concurrently utilise the capacity of his resources (employees) optimally.
• Effectiveness: The ratio between an achieved and the targeted result. Depends on external factors as the provider’s market position and image, the opinion on the service quality by the customer, the quality of customer interactions, etc.

According to this model productivity measurement requires the consideration of more areas than only the delivery: marketing, sales, staff planning, etc. The challenge is to find feasible metrics e.g. for rating the opinion of the service quality by the customer.

Factors influencing the productivity of IT services

For controlling of the service productivity it is important to understand how these factors are influencing each other. E.g. standardisation can improve the efficiency, but it can be perceived as negative by customers. Furthermore, the weights must be identified, i.e. how strong each of these factors affects the total productivity. This depends on the type of services, customers, industries, the business and payment model of the provider, the type of contracts, market position, etc. For more information I recommend the PAC Research Report for the project “ProdIT” supported by the German Federal Ministry of Education and Research (http://www.berlecon.de/studien/downloads/121212_ProdIT_Master_Final.pdf).

For measuring the productivity of software development processes usually it is sufficient to focus on the efficiency (see: Measuring Productivity in Software Development). This simplification is possible if influences by the disposition and utilization of employees are negligible and if the process finishes with a defined quality. Otherwise, if the productivity measurement is being used for supporting decisions beyond the delivery, e.g. regarding the market position of a standard software or the cooperation with a (major) customer, external factors as achieving the customer’s objectives, the flexible consideration of new or changed requirements, the quality and reliability of customer provisions, transparency of intermediate results, etc. must be considered.

Is the Function Point Method still up-to-date?

As shown in my post Methods for measuring the Size of Software, the comparability between KPIs of IT systems of different sizes requires a size measurement method. As well as the effort estimation of a planned development task. 

Since 1979 the best-known method for the analysis of the functional size (as described in the standard ISO/IEC 14143 for Functional Size Measurement) is the Function Point Analysis (FPA). The current standard of FPA is ISO/IEC 20926:2009. Now the method does not need estimations, because it only counts, and all weights are determined. Thus different measurements by different persons always result in the same value, sufficient methodical know-how provided. 

Objects to be counted by the Function Point Analysis

The Function Point Method considers the use cases of a system and counts related user functions / transactions, where data is being sent across the system boundaries (EI = External Input, EO = External Output, EQ = External Inquiry), and structures of internal and external data files (ILF = Internal Logical File, EIF = External Interface File) related to it. A Function Point Analysis according to the original method requires a good understanding of the system’s functionality. Therefore the effort can be high. Today there are approaches for an automated counting (e.g.: http://www.omg.org/spec/AFP/1.0/Beta1/PDF/). These approaches require a good understanding of the code structures and naming conventions related to the user functions and transactions, otherwise structural characteristics will be counted instead of functional. The benefit of such approaches is that counting can be repeated easily after changes of the application have been made and need to be measured.

Matrix for the rating of a structure
in an Internal Logical File

A disadvantage of the method is that weights are defined by open-ended interval scales with three steps. As a result, e.g. all files, where the number of data element types (DETs) exceeds 50 and with more than one record element type (RETs) have the same rating – irrespective of the real number of elements.

Most companies, which are measuring the functional size at all, apply the Function Point Method. Unfortunately often by using the approximation method Rapid, which do not consider the complexity at all and replaces the interval scales by average values. Measurements show that counting user functions and data structures without considering the number of related data elements resp. their complexity leads to a restricted comparability of the measured systems. There are other methods complying with ISO/IEC 14143 where this has been improved.

Using Sonar for steady Analyses of the Code Quality

The platform Sonar allows static code analyses of different projects and presents the results in a standardised form. Sonar applies some own rules enhanced by the rules of integrated tools as e.g. FindBugs, Checkstyle or PMD (for the Java world).

The experiences with Sonar used in a huge portfolio of applications are good. For Java there are more than 800 rules available, which are well described, can be activated and deactivated and can be configured regarding the severity of violations (e.g. Blocker, Critical, etc.) or parameters. Experience has shown that it is better not to use the standard configuration but to discus an own configuration with the most experienced developers and to rate less rather than more rules as blockers. Further it is important that the process model respectively the quality gates clearly define that e.g. it is forbidden to use a release as long as there are blockers, which at least are potential production defects.

Sonar Dashboard showing the results of a Code Analysis (Sample)

Additionally to these rules Sonar measures the comment density, the rate of commented or duplicated code, the cyclomatic complexity, the lack of cohesion in methods (LCOM4) and the class complexity (response for a class, RFC), which are proven indicators for the code quality. Based on an own coding styleguide, which should clearly define threshold values for these metrics, a Sonar profile can be used to set these thresholds for an indication of warnings and errors for each metric. This makes sure that each application being analyzed will be measured and rated by the same standard.

Best practise is to integrate the Sonar code analysis in the build processes of the different applications. Then, after each new build, i.e. each night or continuously - according to the configuration, a new analysis result is available.

The Sonar web client presents the analysis results in dashboards and even in case of a huge portfolio of applications clearly structured. There are tools for e.g. the analysis of time courses or hotspots. The authorized user can drill-down for each violation into the code and, if required, declare it as false-positive.

The experience is that only with a tool such as Sonar the definition of a coding styleguide is really worthwhile, because the compliance can be checked without extensive time and in short intervals.

Information about Sonar can be found at http://www.sonarqube.org/features/.

KPIs for Controlling in Software Maintenance and Development

The following indicators allow controlling of the essential key performance areas of software maintenance and development:

• Productivity (Cost Efficiency): The quotient from the size of produced or changed software and the effort required (see: Measuring Productivity in Software Development).
• Defect Rate (Defect Density): The quotient from the number of production defects per time unit and the size of the system. It must be defined which types of defects are relevant for the defect rate.
• Effort/Costs for correcting Defects: The quotient from the effort for correcting defects (only types which are relevant for the defect rate) per time unit and the size of the system.

These measurements are completely independent of any process model or methodology, i.e. they are useful for heavyweighted processes as well as for agile development. Even agile development requires KPIs in-time for justification.

The benefits of KPIs in software maintenance and development are:

• Performance and quality become measurable and comparable.
• KPIs allow the determination of quantitative success criteria and a precise verification of their achievement.
• Trends become transparent. KPIs serve as indicators for a need for action.
• Performance and quality become predictable (e.g. effort estimations on basis of experiences and empirical data regarding the productivity of similar projects).

Measuring Productivity in Software Development

In industrial production productivity can be measured as the efficiency of a production process by the relationship of output to input. Under the following conditions this relationship can be used to measure the productivity in software development:

• Every measurement considers the same process which finishes with a defined quality, e.g. by passing a quality gate prior to the handover to production. 
• The input can be determined accurately and reliably by the work performance (effort) of all related workers, e.g. in man days.
• The output can be determined by a size metric.

Hence productivity can be calculated by the quotient of size (the output) and effort (the input). A critical success factor is quantifying the output by a size metric which is free from constructive or technological aspects and only counts objects, i.e. no subjective ratings by an estimator (see: Methods for measuring the Size of Software).

In most cases the benefit of productivity measurements is first applied after a longer period of time, when sufficient experience and empirical data are available:

• Productivity influences of changes in staffing, development methods or tools, different aspects of the constructive quality assurance, etc. can be recognized and allow specific optimizations. Without measurements the effectiveness of improvements is a matter of chance.
• The effort for a planned development project can be estimated on basis of experiences and empirical data for similar projects, if the size of the project can be estimated accurately (effort = size divided by productivity).