Counting the uncountable — I

A highlight of the ICMs is the Fields Medals in Mathematics that are announced and awarded at these meetings. This medal in Mathematics is often compared to the Nobel Prize

Dr Khalid Saifullah

December 06, 2014

Comment

   

The vast and splendid hall in the state-of-the-art building of the Hyderabad International Convention Centre was filled with delegates from all over the world, gathered there to witness the spectacular inaugural ceremony of the International Congress of Mathematicians (ICM). They were waiting for the President of India, Shrimati Pratibha Devisingh Patil, to come and declare the proceedings open. The host on the dais was introducing the audience to the charms and wonders India offers to its visitors. “Where in the world would you see one-third of the people driving on the left side of the road, one-third on the right and one-third in the middle of the road?” she asked and the hall of about 3,000 participants roared with laughter.

ICMs are among the oldest scientific meetings, starting around 120 years ago, and are held every four years. This forum was established to review the current trends in research in Mathematics and to discuss future developments in the field. It was at this meeting in 1900 in Paris that David Hilbert, the German mathematician, presented his famous 23 problems meant to set the future of research in Mathematics, many of which are still unsolved. It was around this time that another famous problem, the Poincare Conjecture, after the name of the French mathematician Henri Poincare, was posed. This problem remained unsolved for about 100 years. Another highlight of the ICMs is the Fields Medals in Mathematics that are announced and awarded at these meetings. This medal in Mathematics is often compared to the Nobel Prize in other areas (there is no Nobel Prize in Mathematics). In the Madrid ICM of 2006, Grigori Perelman of Russia was awarded the Fields Medal for solving the 100-year-old Poincare Conjecture. It is interesting to note that at the time of this award he did not have any publications, nor has he anything published today!

An important session at the 2010 Hyderabad ICM was devoted to the discussion on the use of metrics to evaluate research, particularly on the uncritical use of the Impact Factor (IF) for the same purpose. The IF is the average number of citations made in a given year to a journal’s papers from the preceding two years. Their practical application arises from the need of assessing research by simple and objective methods. The panel included Professor Douglas Arnold from the University of Minnesota, the president of the Society of Industrial and Applied Mathematics (SIAM) also. He had highlighted the issues and instances of fraudulently increasing the IF of journals and other blatant misuses of this measure in his articles ‘Nefarious Numbers’ and ‘Integrity under Attack: The State of Scholarly Publishing’. He analysed the cases of IF manipulation for a few journals in detail. He described, for example, how the IF of the International Journal of Nonlinear Sciences and Numerical Simulation (IJNSNS) rose to 8.9, more than double the next highest journal in applied Mathematics. He told the audience that this journal was in his area of research but that he never knew about it. Explaining the reason for this large IF he said that most of the citations came from IJNSNS itself or special issues of other journals edited by someone on the IJNSNS board. Only in the year 2008, the journal’s editor-in-chief himself cited the journal 243 times within the last two years (the crucial window for calculating the IF). Apart from him two other editors cited this journal 114 and 58 times. With IJNSNS, 72 percent of their citations were in the two years that count for the IF and only 28 percent in all the other years. For normal journals it is usually the opposite. With this glaring performance, the editor-in-chief was among the world’s most highly cited mathematicians and was named the “rising star” and the “hottest researcher of the year”. He repeated this performance the next year as well and was considered worthy of many awards and honours. As the grand hall burst into laughter, I started sinking into my seat; it seemed as if they were discussing the situation in Pakistan!

Professor Arnold concluded by mentioning Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure.” He further elaborated by mentioning “an example used in economics, that if a nail factory in a centralised economy is judged on the number of nails produced, pretty soon they will figure out they should make lots and lots of tiny nails. If it is judged on the weight of the output, they will start making very big nails.”

Professor Malcolm MacCallum of Queen Mary University, London was another panelist. He shared his experience of the Research Assessment Exercise that was carried out in the UK in which he played a prominent role. He told the audience that the assessment was done not by any bibliometric data but by actually reading the research papers! His conclusion was: “I do not believe one can judge a paper by where it appears.” This seemed to be the bottom line of the discussion to which everyone agreed. Let me repeat: a paper cannot be judged by where it appears. If someone wants to evaluate research he will have to read the papers, or get them read by somebody. 

(To be continued)

The writer is a faculty member at Quaid-e-Azam University, Islamabad. He may be contacted at ksaifullah@fas.harvard.edu

Reference : Daily Times (dailytimes.com.pk)

Read More

Counting the uncountable — II

During their PhD, students here do not get training in how to think, investigate and analyse, but only how to write papers in such a way as to get them published

Dr Khalid Saifullah

December 08, 2014

 comment 

   

Coming to the situation in Pakistan, there was a time when researchers here did not publish internationally. It was a good step forward when they started doing this. Following this, scientists used to be evaluated by the list of publications they included in their CVs. However, with the advent of Impact Factor (IF), things began to change. When this measure was applied to the list of publications of many so-called scientists and heads of scientific institutions, they were left with nothing. Hundreds of publications vanished altogether. But what we must realise is that this was only one step forward and not an end in itself.

Here we should pause for a moment and ask ourselves: what is the aim and purpose of research in the Sciences and Mathematics? Is it to accumulate IFs like a video game where one strives to score high and beat the existing highest score, or is it to come up with ideas that can add to the existing body of knowledge? But this is exactly what the institutions that have taken upon themselves to judge and evaluate research and rank researchers are not doing. In Pakistan, the Higher Education Commission (HEC) and the Pakistan Council for Science and Technology (PCST) are ranking scientists who have been ‘conferred’ awards (whether they deserve them or not) merely on the basis of some numbers; these include all civil and other awards. Is there any respectable research award in the world that is given without actually reading the research? How can one judge research from the colour of the title page of the journal where it is published, or by some number attached to that journal? As mentioned above, the Fields Medal was awarded to Grigori Perelman when his research had not been published in any journal!

The worst effect these policies have is on the section of our society that cannot afford it at all: our youth. Research for a PhD degree is supposed to be cutting-edge and on the frontiers of the field it is awarded in. Most of the research done in Pakistan is by PhD candidates, which means these policies make their research work questionable. As a result of the current policies, what they learn during their whole research-training period is how to achieve certain numbers and then how to increase them. And, since they start hiring new PhD students as soon as they complete their own PhD, they impart exactly what they had learnt during their own ‘training’ and it goes on like a chain reaction. The recent explosion of business houses for online journals is basically to fulfil the increasing demand for publishing. Doing PhD in our universities currently means only one thing: a few research papers where your name appears among the list of authors, even though that might be only by mistake (and in many instances that is indeed the case!). Thus, during their PhD, students here do not get training in how to think, investigate and analyse, but only how to write papers in such a way as to get them published. It is like being trained to be an orator when you have nothing to say.

The message is loud and clear: your research performance is not judged by what you do and what you write; it is judged by some numbers associated with the journal where your work is published. So your job now is simply to look for ways by which you can increase those numbers. Those who are attached to our universities know that the message is very well taken by our youth, and they are doing the needful. This is what happens when we blindly follow artificial ranking systems for scientists and institutions. It is the result of striving to fulfil the ranking criteria for institutions; a couple of years ago researchers affiliated with King Abdulaziz University in Jeddah outnumbered highly cited researchers at the University of Cambridge by 50 percent in the field of astronomy! Following these measures as targets and conferring awards to people on the basis of these criteria in Pakistan without actually judging and reading their research work is producing examples that are no less amusing.

The current policies are creating an environment that is not different from the Nobel Prize winner economist George Akerlof’s market for lemons where, as a result of the asymmetry of information between the seller and the buyer, the bad drives out the good in the market. In our case, the HEC and PCST know nothing about the product scientists are handing over to them for evaluation, as a consequence of which the good is constantly being discouraged. These standards are set so deep in our minds that people not conforming to them, however good they may be, are looked upon as aliens and a threat to our system. These methods for evaluating and rewarding scientific investigation pose obstacles in the way of many capable researchers. They are, in fact, preventing many good scientists abroad from coming back to Pakistan.

A pertinent question that can be asked here is: who should then judge and evaluate research work? Should researchers not be encouraged by giving out awards and recognition? The answer is: if you cannot judge do not give awards, and do not rank and categorise. “When we decide what to measure, we signal what counts,” Drew Faust, President Harvard University recently wrote in the New York Times. Thus, crediting the wrong people and presenting them as role models, particularly to our youth, is far more harmful than not crediting at all. But the actual situation is not that bad. It is true that the community in Pakistan is very small; there are not enough people who can do justice to evaluations and assessments. But we should not forget that Mathematics is essentially an international activity. It is not (and should not be) done in isolation. Once we decide to really promote Mathematics and encourage real mathematicians by awarding them then it is not something that cannot be done. We need to ask ourselves if this is something in which we are pioneers. If not, then we should see how other people have been doing this for ages. There are international bodies to promote Mathematics research in developing countries and emerging nations. For example, the Commission for Developing Countries formed by the International Mathematical Union is assisting many countries right now. They can be approached for help and guidance.

I forgot to mention earlier that at the conclusion of the opening ceremony of the Hyderabad ICM, when the delegates were being taken by buses to their accommodations, they actually witnessed what they had been told earlier: people were driving on the left, right and in the middle of the road. I thought the situation was not very different in Pakistan!

(Concluded)

The writer is a faculty member at Quaid-e-Azam University, Islamabad and can be contacted at ksaifullah@fas.harvard.edu

Reference : daily times (dailytimes.com.pk)

Read More