For more than 50 years, Auerbach Publications has been printing cutting-edge books on all topics IT.

Read archived articles or become a new subscriber to IT Today, a free newsletter.

This free newsetter offers strategies and insight to managers and hackers alike. Become a new subscriber today.


Partners




Contact

Interested in submitting an article? Want to comment about an article?

Contact John Wyzalek editor of IT Performance Improvement.

 

Achieving the Right Balance between Process Maturity and Performance

Louis A. Poulin

Organizations have come to recognize that in order to survive and to grow, they need to demonstrate to their clients that they are among the best. If not, they run the chance of becoming the next outsourcing statistic. Likewise, organizations in developing countries in Asia, South America and Eastern Europe that provide outsourcing services also need to demonstrate to their clients in North America and Europe that they are among the best in the world, in order to benefit from outsourcing opportunities.

This research summarizes observations made in the course of 58 assessments based on the CMMI© and on its predecessor, the CMM©. It stresses, among other things, the challenges faced by these organizations in optimizing ways to improve the quality of their products and services using the CMMI as a starting point.

Introduction

In May 2005, the results of a 10-year research initiative were published [1] and consolidated the observations made as part of 40 assessments GRafP Technologies conducted in North America, South America and Europe over a period ranging from 1993 to 2003. In particular, results showed that development initiatives in the Information Technology (IT) field had a 33% probability of experiencing serious schedule or budget problems, or problems related to products or services that did not adequately satisfy their requirements. This 33% value corroborated the finding made by the Standish Group International in 1995 [2] to the effect that 31% of IT projects were canceled before completion in the United States. From 2003 and on, we tried to gain a better understanding of the factors that contributed the most to the success, and sometimes failure, of organizations, particularly when they implemented development models, such as the CMMI [3], Control Objectives for Information and related Technology (COBIT) [4], Six-Sigma [5], and Information Technology Infrastructure Library (ITIL) [6], or development standards such as ISO 9001, ISO 12207 and ISO 27001 (also known as ISO 17799 [7]). Implementation of such models and standards can be very taxing, particularly for small and medium enterprises. Even though they are sometimes required in order to demonstrate to potential clients the capability of an organization, the work we had performed with enterprises over several years suggested that there was a better way than traditional audits to demonstrate that capability. In particular, our work focused on understanding how processes suggested by these models and standards can be optimized taking into account the context in which an organization operates, and the market forces and other constraints to which it is subjected.

On Process Engineering and Optimization

Process Engineering and Optimization consists of devising the most efficient and effective ways of integrating material and human resources, methods, procedures and tools in order to achieve a given objective.

An optimal process is one that allows an organization to maximize the outcome offered by opportunities it has of meeting its business objectives, and to minimize the number and severity of problems it has to deal with in the course of pursuing these objectives. Process engineering and optimization is meaningless without having this definition in mind.

This is closely related to risk, since an optimal process can also be described as one that allows an organization to prevent undesirable situations from happening, and desirable situations from not happening.

One of the models used to optimize product development processes is the CMMI for Development. The CMMI has five maturity levels, which are explianed in Table 1.

Table 1. The Five Levels of the CMMI

Optimal Processes

An optimal process, as defined above, is a theoretical concept. In order to be applied, the term “Process Capacity” also needs to be defined.

In theory, it is possible to envision a model describing an optimal process specifically designed for a given business area, for instance, software development. This process will then be associated with the ability of undertaking the development of a given range and a given scope of software products, in a way that will maximize the growth opportunities these products offer, while minimizing the number and severity of problems encountered during development and commercialization.

However, few organizations, if any, may have the capacity of implementing such a model. For instance, it may be found that an organization masters only 25% of the process suggested by this model. Which 25% can make an enormous difference. In order to achieve this ratio of 25%, referred to in this case as “Process Capacity”, the organization may have implemented the process suggested by the model such that each practice making up this process is implemented in a way that satisfies 25% of its intent. Alternatively, some practices may be fully satisfied (i.e. 100%), some others may only be partially satisfied (e.g. 50%), and others yet may not be implemented at all (e.g. 0%), such that the resulting average is 25%. Therefore, there must be at least one combination of practices resulting in a 25% implementation ratio that is better than all the other ones, and a least one that is worse than all the others. If this were not the case, implementation of any subset of practices leading to a 25% implementation ratio would all result in the same performance, which makes no sense. A higher percentage indicates that the organization has the ability to undertake the development of a larger range and broader scope of products in the business area for which the model was initially developed.

Data collected with respect to the CMMI and other models suggest practices are not all equal, given the context in which an organization operates. Some practices have high risk-mitigation potential, in the sense that they are better at preventing undesirable events from happening, and desirable events from not happening in this given context.

Given that a process typically consists of a large number of practices, the optimal set is one of a very large number of possible combinations. For example, if a process is made up of 100 practices that can be applied to take advantage of 50 opportunities and to prevent 50 problems from occurring, and if the importance of each practice, the potential of each opportunity, and the consequence of each problem are characterized using a scale of 5 degrees of significance (e.g., very important, somewhat important), a total of 250,000 combinations must each be examined to determine the optimal process. Furthermore, if the degree of each practice implementation and the likelihood of each opportunity and each problem are quantified with three bits (i.e., 8 levels, for example six increments of 20%, an unknown status, and a not applicable status), the number of combinations increases to 16,000,000.

Generalization of Information Theory Principles

Our involvement in process engineering and optimization has led to the generalization of a theorem, which was established at the Massachusetts Institute of Technology by Claude E. Shannon in 1948, and dealt with the transmission of information in noisy channels [8]. This theorem, although less well known, represents for the field of communication what Einstein’s relativity theory represents for the field of physics.

The generalization of Shannon’s theorem applied to product or service development leads to the following conclusion: For any organization, there is at least one process that will allow that organization to reduce to an arbitrarily low value the risk of not being able to fulfill its business objectives, as long as this process does not exceed the capacity of the organization to apply it.

In other words, there is at least one set of key practices that will allow an organization to fulfill its objectives, in light of the constraints that characterize it, such as the business context in which the organization operates, the types of products it develops or services it provides, its available resources, and its culture. This is summarized by the two graphs presented in Figure 1 and Figure 2.

Figure 1. Organization with a High Process Capacity

Figure 2. Organization with a Low Process Capacity

It is assumed in these graphs that the organization can do well as long as its likelihood of not being able to fulfill its business objectives is less than or equal to 20%. This is defined as the margin of efficient operation.

Figure 2 shows that in the case of an organization characterized by a low process capacity, however simple the process may be, the likelihood that the organization will not fulfill its business objectives will approach zero if that process enables the organization to anticipate and manage its risk well.

The difficulty, and this directly stems from Shannon’s theorem, is that the optimal set of key practices for a given organization is unknown a priori. To make matters worse, it is reasonable to assume that the optimal process is not static but is organization-dependent and time-dependent, and will have to be modified as the context in which the organization operates evolves. Any significant deviations in the way the process matches the opportunities to exploit or the problems to prevent, which may have been caused by the loss of a few key personnel or a change in market conditions, result in the likelihood that the organization will not fulfill its business objectives suddenly increasing to an unacceptable value.

There is obviously a cost in minimizing the likelihood that an organization’s business objectives will not be fulfilled. In a way similar to encoding and decoding schemes that are devised to correct errors in information transmitted over a noisy communication channel, without directly contributing to the quantity of transmitted information, processes that do not directly contribute to fulfilling business objectives must be devised to prevent undesirable situations from happening, and desirable situations from not happening.

Example

Assume that Figure 1 represents Organization X, the industry leader in a given business area, and Figure 2 represents Organization Y, a small enterprise in the same business area.

The degree to which good business practices are applied, in the case of Organization X, is very high, which explains the high concavity of the curve. For Organization Y, this takes the form of a convex curve indicating the low capacity of its process. From its past performance, it can also be assumed that the degree to which Organization X is capable of anticipating and managing undesirable situations that are liable to happen, and desirable situations that liable not to happen, is also very high. Therefore, the projection of this value on the graph displayed in the right part of Figure 1 translates into a very low likelihood that Organization X will not fulfill its business objectives. Based on this graph, one can conclude that Organization X has achieved a level of success that Organization Y can hardly conceive.

Is Organization Y therefore condemned, at best, to mediocre achievements as a result of the low capacity of its process?

Not necessarily. In fact, the generalization of Shannon’s theorem described in the preceding section leads us to believe that it is illusory for Organization Y to think of being able to emulate Organization X. Organization Y simply does not have the capacity to do so. However, the graph shown in Figure 2 suggests that this is not a hopeless issue. There is a margin within which Organization Y can achieve success. However, in order to do so, Organization Y must be fully aware of the opportunities it can realistically exploit and the challenges it faces, which in turn, makes it recognize situations liable to jeopardize fulfillment of its business objectives, and devise a process that will compensate its limitations.

Two choices are available to Organization Y to reduce the likelihood of not being able to fulfill its business objectives. Organization Y can, in one case, seek a higher degree of applying good business practices that make up the model corresponding to its particular business area, and thus reduce the convexity of graph of Figure 2. Alternatively, Organization Y can choose to eliminate practices that bring little or nothing, and instead focus on those that help identify situations to which it is vulnerable, in light of its limitations, while implementing corrective actions as appropriate. This second choice leaves the process capacity of Organization Y unchanged, which remains relatively low. In this second case, the curve convexity of the graph in Figure 2 remains the same, but there is an increase in the capability of Organization Y to anticipate and manage undesirable situations that are liable to happen, and desirable situations that are liable not to happen. The most promising approach will likely be a combination of both approaches. In theory, self-assessments with a model compatible with the business context Organization Y is pursuing, either on a continuous or a periodic basis, will help the organization achieve a satisfactory performance that takes into account the context in which the organization operates, the types of products it develops or services it provides, its available resources, and its culture, in other words, the constraints and limitations with which it must deal on a daily basis. This assumes that the reference model on which Organization Y has based its process is a “good” model, without necessarily implying that it is the optimal model. As suggested in the present document, other models can also be used for the same purpose.

The margin within which Organization Y can expect to fulfill its business objectives essentially differentiates it from Organization X. From the graphs presented in Figure 1, Organization X has a much better capacity to overcome the obstacles it faces. In the case of Organization Y, certain events are susceptible to make it cross the aforementioned margin threshold, such that the likelihood of not fulfilling its business objectives will sharply increase to a value that effectively eliminates its chances of achieving them.

Conclusion

Information technology has now reached a point where it is a critical component of a country’s infrastructure. Any country that does not invest in this infrastructure increases the vulnerability of its industry that is involved in manufacturing products or providing services drawing on technological development and maintenance. This is compounded by the fact that the United States, China and India, among others, have been aggressively pursuing an improvement of their business and industrial capability in this area since the mid 1980’s, and a country neglecting to do the same could find itself in the unenviable position where an exceedingly large part of its technological industrial complex is controlled by foreign-owned companies.

It then becomes that much more important that a country’s industry find innovative ways to optimize its technological development processes in order to improve its performance and become more competitive, since it often takes years before an organization having embarked in improvement initiatives is able to see the return on its investment. Delays in undertaking such initiatives will simply make competitive advantages more difficult to achieve, by increasing the number of parties with whom it is necessary to compete or by widening the gap with industry leaders. ♦

References

[1] Louis A. Poulin. Reducing Risk with Software Process Improvement Auerbach Publications, Boca Raton FL. 2005.
[2] The Standish Group International. Chaos—Application Project and Failure January 1995.
[3] Mary Beth Chrissis, Mike Konrad, and Sandy Shrum. CMMI: Guidelines for Process Integration and Product Improvement Addison-Wesley, Boston MA. 2006.
[4] IT Governance Institute. Control Objectives For Information And Related Technology 2007.
[5] Thomas Pyzdek. The Six Sigma Handbook McGraw-Hill, New York. 2000.
[6] OGC—IT Infrastructure Library (ITIL)
[7] BS 7799 Part 1. (2000). ISO/IEC 17799: 2000 Part 1 Code of practice for information security management.
[8] Claude Shannon. “A mathematical theory of communication,” Bell Syst. Techn. J., vol. 27, pp. 379-423 and 623-656, 1948.

© CMMI, CMM, and Capability Maturity Model are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

Read more IT Performance Improvement

Related Book

Reducing Risk with Software Process Improvement

Louis A. Poulin

The author describes the observations that he made over a period of ten years in IT projects and organizations. He focuses on the areas of software development and maintenance, highlighting the most frequently encountered problems that occur due to poor processes.

About the Author

Louis A. Poulin is involved in assessing the capability of software development organizations, and in developing risk assessment methodologies and risk management applications. He holds a Bachelor degree in Engineering Physics, a certificate in Naval Engineering and a Master’s degree in Electrical Engineering. Prior to his active involvement in software engineering, Mr. Poulin served in the Canadian Navy as a Combat Systems Engineering Officer. He is a Senior Member of the Institute of Electrical and Electronics Engineers and a Fellow of the Engineering Institute of Canada.