Thinking Beyond Metrics in Higher Education
While the entire world is rethinking the role of universities to deal with complexities of the 21st century aggravated by the COVID-19 pandemic, our higher education system has gone directionless. Some of its original architects prefer status quo and emphasize the continuation of an outdated model developed decades ago. Recent debacle of unceremonious removal of the HEC chairman is also a consequence to challenge this status quo. There has been a tug of war between status quo forces and progressive ones during the last three years. Former are in majority and have been primary beneficiaries of the system developed twenty years back. Later tried to introduce reforms not only in ad hoc fashion but also without broader consultation among the stakeholders.
This crisis demands comprehensive analysis by government rather fixing the issue on ad hoc basis. It’s time to rethink higher education system role and contribution to the society in a new paradigm. The HEC system, currently in vogue, was introduced two decades ago. The time has come to introduce changes as per lessons learnt during these 20 years. Continuity of the system has created perverse incentives with beneficiaries enjoying perks, privileges, and monetary incentives. This lobby has had a presence over higher education horizon for several decades. It is part and parcel of most of the bodies working in higher education system. They are members of higher education governing body, selection committee for the Vice Chancellors, national science academies and other bodies. Daring to challenge them will obviously have a cost. If there is no cost of being daring, each of us can be very daring.
Proponents of the existing system strongly believe that we must not just measure everything but reward as well. There was an overemphasis on Pakistan higher education system ranking on certain global and self-perceived national metrics. We spent all our energy on improving ranking instead of linking research with the demands of society. Nor did we focus on developing problem-solving and analytical skills. This ranking business does more harm than good most times. For example, surgeons will not operate on risky patients in order to improve their ranking in many countries. This means they leave some patients to die for sure, but on metric criteria these surgeons’ ranking is very high. Similarly, many airlines have slightly inflated flight times to avoid being penalized for having delays.
Being a student of metrics myself, I am in a position to say the following. The proponents of metrics-based system were ignorant of unintended consequences of using them for rewards or sanctions. The issue has never been with measurement but with excessive measurement and inappropriate measurement- not metric but metric fixation. It is unfortunate we apply certain yardsticks to all academic disciplines as a one size fit all measure. We apply these yardsticks, that are suitable only to certain disciplines to measure faculty productivity, across the board. We showed performance as more numbers: PhDs, research papers, universities and research projects to get funding from the government. Similarly, we use metrics to gauge productivity of faculty members, among others. Architects of the system ignored the fact that productivity is for robots. Humans excel at wasting time, experimenting, playing, creating, and exploring.
“If what is actually measured is a reasonable proxy for what is intended to be measured, and if it is combined with judgement, then measurement can help practitioners to assess their own performance, both for individuals and for organizations. But problem arises when such measures become the criteria used to reward and punish- when metric become the basis of pay-for-performance/ratings.”Excerpts from “Tyranny of Metrics”, by Muller.
Showing measured performance is deceptively attractive and distracts one from finding a proper solution to metric madness. Measurement becomes unproductive when we apply all energies to quantify the unquantifiable. We set research papers as a criterion for promotions, performance-based increments. For national awards, for short-listing of candidates for the Vice Chancellors and many others. We linked university funding to greater number of PhDs and M.Phil students. Judged universities by number of publications, which provided incentives to have a greater number of publications, rather than better. We did not focus on problem solving or writing a book – something that may take three to four years. Instead, we gave incentives for speed and output. This has resulted in a decline of a meaningful research culture. Faculty has temptation to have quantity first and good research second.
This has resulted in publishing in areas where it is easy to do so with a higher impact factor associated. We ignore whether it has any link to the demand of society, industry and government or not. Social scientists started publishing either on technical issues by minor tweaking of papers published in international journals. Or in local journals paying no attention to social issues of the society. The fields of business and law, despite being of a more practical nature, follow suit to survive. Natural and biological sciences, which have many international journals to publish in, shifted focus to quantity over quality. They set high-impact factor, determined by a commercial agency, as the sole criterion for a better-quality paper with little recognition among the peers of the subject.
With focus on more years of education instead of quality of education, what has happened over time is that jobs that once required 14 or 16 years of education now require M.Phil and sometimes PhDs. That is not because jobs have become more demanding or require a high skill but because employees can afford to choose among the many applicants who are M.Phil/PhD, while excluding the rest. The result is both to depress many graduates who lack higher degree, and place high degree personals where they don’t make any substantial contribution.
Is it not surprising that percentage of students with MPhil and PhD degrees in Pakistan has gone up, while the rate of its economic growth has declined? Correlation is obviously not causation. We associate high rewards with performance metric designed according to the criteria on which architects of the system were the best themselves. Anyone who dares to challenge this metric-based reward system or differs with this dominant thinking gets sidelined or thrown out of the system.
When performance evaluation and accountability was made a part of HEC’s mandate, focus was more on metrics. Over time this led to establishing Quality Enhancement Cells (QECs), and Offices of Research, Innovation and Commercialization (ORIC) in universities. More the metric, more the administrative positions. HEC and universities have now large number of QEC and ORIC staff, which have become a permanent burden on the budget. Instead of spending more on development, emphasis has shifted to have more administrative positions in the name of having better accountability.
Universities have a key role to play in building a prosperous, dynamic, tolerant, talent attractive and technology intensive society. Pakistan has a huge youth bulge and any negligence by society in not equipping this youth with requisite skills will lead to a demographic disaster. This requires the government to carry out comprehensive analysis of the situation and reform the higher education system with professionals who have no conflict of interests.