Needed – A Capability Maturity Model for Engineering Education

Pankaj Jalote,

Professor, Computer Science and Engineering

I. I. T. Kanpur

Jalote [AT] iitk.ac.in

There is a lot of similarity between the software situation that prevailed in the world a few years ago and the IT/engineering/MBA education situation currently prevailing in the country. Some year ago, the high demand for software had led to the creation of thousands of software companies. These organizations often took the customers for a ride – not delivering what was promised, overcharging, poor quality, etc. The buyer had no visibility into their capability for delivering the service and had to buy it almost on faith, and many buyers felt cheated. To take control of this situation, the department of defense in the US, which was the major customer for software, set up the software engineering institute (SEI). The charter of SEI was to evolve the capability maturity model (CMM) which can be used to provide suitable visibility into the capability of a supplier for providing the software service.

The current situation in technical and management education in India is similar. Due to the high demand, fueled largely by the boom in knowledge-based industries like the IT sector, many private colleges have come up for education in IT, engineering, and management – currently 90% or more of the colleges are private. Many of these colleges and institutes do not have the capability of providing the training they claim to provide, but the customer (the students and parents) have no way to judging their capability, and “buy” the education at high prices. And in the end, they frequently feel cheated and that did not get value for their money as the education provided added few skills, and was of little help in procuring proper employment. What is worse in this scam is that not only do people loose money, the youth also loose precious years of their lives.

Clearly, to service the huge demand, participation of the private sector is essential. Like in software, what is currently needed is a capability maturity model for education which can be used to evaluate the capability of the education providers, and provide proper visibility to the customers about the different aspects of their capability. Such a system can also be helpful to the colleges to do focused improvement to enhance their capability.

Clearly, the CMM for education will look very different in details than CMM for software. However, there are some general properties of such a framework. First, the overall capability of a service provider can be represented by a few levels. It is now recognized that a two-level accreditation systems, like the ISO-certified or ISI-mark are suitable for some sectors like the manufacturing sector, but are not well suited for service sector. For a service sector, it is much better to have a framework that is multi-level, such that capability of delivering quality has a finer gradation. Then for each level, a few key aspects of providing the service (called the Key Process Areas or KPAs in the CMM for software) at a certain level are specified. During an assessment, each of these areas are evaluated and rated with weaknesses and strengths identified. And if all the areas of a level are fully satisfied, then the organization is supposed to be at that level. For a higher level, all areas of that level and the levels below must be satisfied.

For the education sector in India, levels can be defined like in the SPICE model (which is the ISO answer to CMM) which has levels 0 to level 5. Key areas for each level can be defined. These could be areas like infrastructure, number and quality of faculty, education programs, on-campus recruitment, off-campus recruitment, perception of employers about the institute, involvement in R&D, impact on society, etc. For lower levels, for example, only things like infrastructure, faculty, and programs need to be considered. For higher levels, continuing education or growth of faculty, involvement in research and development, projects undertaken, publications, etc. can be the areas that should be evaluated. For each of these areas, a rating, along with strengths and weaknesses should be specified. Not only will such a framework provide a finer evaluation of quality, an appraisal will provide the much needed visibility to the consumer about the capability of the institute in the key dimensions, as well as provide the institutes with a clear path for improvement.

How is this CMM for software to be implemented? In this again we can learn from the International Standards Organization and the SEI. These organizations do not themselves do the assessment – that will be a mistake and will make the organization the bottleneck. Instead, these organizations also developed standards and procedures for evaluation, trained people and organizations for performing the evaluation, and then let these organizations actually do the assessment. Generally, the central bodies record the results of the assessments performed by the authorized organizations. For CMM for education, something similar can be followed, by picking up the best practices from the ISO and the CMM frameworks, and improving upon them. The name of the certifying body can also be reported, which acts as a pressure on the certifying body to follow the assessment procedures in letter and spirit. The assessment can be followed by regular surveillance audits, as is done in ISO, to make sure that practices continue to be followed, and improvements are taking place.

What is the role of the Government, if this approach is to be followed? The role of the government will be to create a center that will develop a capability maturity model for engineering education, a capability maturity model for MBA programs, a capability maturity model for IT-training, etc. This center will then develop the key areas for evaluation, the criteria and procedure for evaluation, training program for the assessors, etc. This is a major task, but can be achieved within a year or so. In addition to this, the center will have the ongoing task of collecting the feedback on the framework and improving it with time. Once the assessments start, the center will have to approve the assessments performed by authorized assessors, and maintain the central repository of the results of these assessments. By keeping the results on a well designed web-site which is available to the students, complete transparency to the customers will be available not only about the overall capability, but also on capability in different areas. Such results are also invaluable in taking an overall perspective of the sector and how it is evolving (e.g. for the software sector, we are able to say that India has so many companies at CMM level 5, so many at level 4, etc, and how the overall levels have changed with time.)

Finally, how do we ensure that colleges participate in this? An easy way is for the government to require that assessments to be done if they are to be recognized in some way (or their university affiliation will be withdrawn.) Of course, sustaining force can only be created by the consumer demanding to know the capability of the service providers. If software experience is anything to go by, consumers quickly realize the value of such a system and start demanding it and the suppliers, in response to consumer demands, quickly adopt these frameworks.