International Journal of Information Science and Management (IJISM)

Challenges of Research Performance Evaluation in Iran: A Qualitative Study

Document Type : Original Article

Authors

Nadia Saniee 1
Shahram Sedghi 2
Abdolreza Noroozi chakoli 3
Salime Goharinezhad 4
Leila Nemati anaraki 5

1 Department of Basic Sciences, Asadabad School of Medical Sciences, Asadabad, Iran.

2 Health Management and Economics Research Center, Iran University of Medical Sciences, Tehran, Iran; Librarianship and Medical Information Science, Iran University of Medical Sciences, Tehran, Iran

3 Department of Information Science and Knowledge Studies, Shahed University, Tehran, Iran

4 Preventive Medicine and Public Health Research Center, Psychological Health Research Institute, Iran University of Medical Sciences, Tehran, Iran

5 Department of Medical Library and Information Science, Iran University of Medical Sciences, Tehran, Iran

https://doi.org/10.22034/ijism.2024.1990489.1056
Abstract
 
Research performance evaluation indicators can be used in research and financial policy-making, as well as the promotion of researchers and universities in a country. Therefore, this study aims to investigate the challenges of research performance evaluation in Iran. Participants in this qualitative study were selected using a purposive sampling method. 11 Out of 21 experts from the two areas of the Ministry of Science, Research, and Technology and the Ministry of Health and Medical Education were interviewed face-to-face, by telephone, and WhatsApp social media based on an interview guide. Interviews were analyzed with MaxQDA software version 10. Most of the participants were male (54%), had a Ph.D. degree in library and information science (54%), and had at least four years of research management and scientometric experience (36%). The analysis of the interviews included 143 codes in the form of six main categories and 50 sub-categories. The most critical challenges included a lack of normalization in research evaluation, lack of suitable scientometric training, lack of importance of scientometrics in research management, the necessity to change the scientometricians role, absence of scientometrician in research policy-making, the focus of universities ranking systems on research indicators, the emphasis of universities ranking systems on quantity, the over-focus on quantitative indicators, the incompleteness of scientometric indicators, the lack of proper application of scientometric indicators, the role of scientometricians in improving the research quality, and the need for balanced attention to the quantity and quality of research. Overall, the research performance evaluation in Iran is facing the challenges of globalization of research performance measurement, providing sufficient research infrastructures, the position of scientometrics in research policy, university ranking systems, application of scientometric indicators, and inefficiency of the research management.
 

Keywords

Research Performance Evaluation
Challenge
Scientometrics
Research Management
Research Policy-Making

Subjects

Abdollahi, M. (2010). [Editorial] Perspectives of science in Iran. Iranian Journal of Pharmaceutical Research, 9(3), 207-208. https://doi.org/10.22037/ijpr.2010.858
Alamdari, A., & Afshoun, E. (2003). The viewpoint of faculty members on research barriers at Yasuj Universities. Armaghane-Danesh, 8(29), 234-7. [in Persian]
Atash Deligani, F., Asadi, S. & Noormohammadi, H. (2017). Identification and classification of the aims, audiences, and challenges of scientometrics departments in Iranian medical universities. Journal of Scientometrics, 3(6), 1-14. https://doi.org/10.22070/rsci.2017.503 [in Persian]
Badrizadeh, A., Gholami, Y., Birjandi, M., Beiranvand, G. & Mahooti, F. (2009). Barriers to research from the viewpoint of Lorestan University of Medical Sciences faculty members. Yafteh, 11(3), 93-100. Retrieved from http://yafte.lums.ac.ir/article-1-169-en.pdf [in Persian]
Braun, V. & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 57–71). American Psychological Association. https://doi.org/10.1037/13620-004
Butler, D. (2006). Islam and science: The data gap. Nature, 444(7115), 26-28. https://doi.org/10.1038/444026a
Cadez, S., Dimovski, V. & Zaman Groff, M. (2017). Research, teaching and performance evaluation in academia: The salience of quality. Studies in Higher Education, 42(8), 1455-1473. https://doi.org/10.1080/03075079.2015.1104659
Çakır, M. P., Acartürk, C., Alaşehir, O. & Çilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103(3), 813-848. https://doi.org/10.1007/s11192-015-1586-6
Eskrootchi, R. & Sanee, N. (2018). Comparison of medical research performance by thermodynamic and citation analysis methods. Scientometrics, 117(3), 2159-2168. https://doi.org/10.1007/s11192-018-2930-4
Farhoodi, R. & Bastehnegar, M. (2004). Providing a model for evaluating the performance of research units with a final license from the Ministry of Science, Research and Technology. Modirsaz, 7(15), 33-53. [in Persian]
Feng, G. C. (2020). Research performance evaluation in China: A big data analysis. SAGE Open, 10(1). https://doi.org/10.1177/2158244019901257
Foroughi, Z., Tahmasebi Limooni, S. & Ghiasi, M. (2020). A review of the status of the existing scientific index and selection of the most appropriate index for evaluating scientific outputs in medical sciences. Clinical Excellence, 9(4), 23-33. Retrieved from https://ce.mazums.ac.ir/article-1-498-en.pdf [in Persian]
Franceschini, F. & Maisano, D. (2011). Criticism on the hg-index. Scientometrics, 86(2), 339-346. https://doi.org/10.1007/s11192-010-0261-1
Hamidah, I., Sriyono, S. & Hudha, M. N. (2020). A bibliometric analysis of COVID-19 research using VOSviewer. Indonesian Journal of Science and Technology, 5(2), 34-41. Retrieved from file:///C:/Users/Reza/Downloads/24522-52993-7-PB.pdf
Hashemi Daran, H. (2012). Factors affecting research performance of faculty members in Islamic Azad University Roudehen branch. The Journal of Modern Thoughts in Education, 7(2), 103-112. 20.1001.1.20081138.1391.7.2.8.0 [in Persian]
Jahanshahi, H., Hassanpour, H. & Ahmadi Ghavami, M. (2012). Ranking the practical factors in the successful implementation of knowledge management. Police Organizational Development Journal, 9(41), 43-60. [in Persian]
Johnes, J. (2018). University rankings: What do they show? Scientometrics, 115(1), 585-606. https://doi.org/10.1007/s11192-018-2666-1
Malekzadeh, R., Mokri, A. & Azarmina, P. (2001). Medical science and research in Iran. Archives of Iranian Medicine, 4(1), 27-39. Retrieved from https://www.researchgate.net/publication/288142441_Medical_science_and_research_in_Iran?enrichId=rgreq-02e5b459c19022d496b6795265020370-XXX&enrichSource=Y292ZXJQYWdlOzI4ODE0MjQ0MTtBUzozMTY3OTI3MzAyNTk0NTdAMTQ1MjU0MDY4MDg1NA%3D%3D&el=1_x_2&_esc=publicationCoverPdf
Moin, M., Mahmoudi, M. & Rezaei, N. (2005). Scientific output of Iran at the threshold of the 21st century. Scientometrics, 62(2), 239-248. https://doi.org/10.1007/s11192-005-0017-5
Moradi, M., Dustar, M., Ghaderifar, E. & Zanjani, B. (2013). Identifying and prioritizing the barriers to research: The case study of research centers of Ministry of Science, Research, and Technology. Journal of Science and Technology Policy, 6(1), 35-47. 20.1001.1.20080840.1392.6.1.5.6 [in Persian]
Noroozi Chakoli, A., Ghazavi, R. & Taheri, B. (2016). Valuation of research evaluation indicators in different scientific fields in Iran. Journal of Science & Technology Policy, 7(4), 31-41. 20.1001.1.20080840.1394.8.4.4.5 [in Persian]
Pooryazdian, M. & Karami, M. (2019). Evaluation of university ranking systems to evaluate the university research performance with an organizational excellence approach. Paper presented at The First National Conference on Humanities and Development, Shiraz, Iran.
Sabzevari, S., Mohammad Alizadeh, S. & Aziz Zadeh Foroozi, M. (2000). Views of faculty members of Kerman University of obstacles in conducting research activities. Journal of Shaheed Sadoughi University of Medical Sciences, 8(2), 18-27. [in Persian]
Sanee, N., Mohammadi, M. & Ghazi Mirsaeid, J. (2016). Evaluation of scientific status of endocrine, diabetes, and metabolism research centers in Iran using exergy method. Webology, 13(2), 32-41. Retrieved from https://www.webology.org/data-cms/articles/20200515045631pma150.pdf
Sani'ee N., Nemati-Anaraki L., Sedghi S., Noroozi Chakoli A., Goharinezhad S. (2022). Comparative study of research performance and innovation-industry indicators in national and international university ranking systems. Journal of Medical Library and Information Science, 3, 1-11. https://doi.org/10.22037/jmlis.v3i.38107
Sani'ee, N., Nemati-Anaraki, L., Sedghi, S., Noroozi Chakoli, A. & Goharinezhad, S. (2022). The Effective Trends and Driving Forces in The Future of Research Performance Evaluation: A Qualitative Study. Medical journal of the Islamic Republic of Iran, 36, 55. https://doi.org/10.47176/mjiri.36.55
Shafiee, F. S., Naderi Roshanavand, A., Abili, K. & Sobhani, H. (2018). Analyzing and explaining faculty members' research performance with mixed approach and multilevel modeling, case study: University of Tehran. Educational Planning Studies, 7(13), 98-120. https://doi.org/10.22080/eps.2018.2024 [in Persian]
Shah, S. H. H., Lei, S., Ali, M., Doronin, D. & Hussain, S. T. (2019). Presumption: bibliometric analysis using HistCite and VOSviewer. Kybernetes, 49(3),1020-1045. https://doi.org/10.1108/K-12-2018-0696
Waltman, L. & van Eck, N. J. (2019). Field normalization of scientometric indicators. In Springer Handbook of Science and Technology Indicators (pp. 281-300): Springer.

  • Receive Date 26 February 2023
  • Revise Date 21 December 2024
  • Accept Date 21 December 2024

Home

Guide for Authors

Submit Manuscript

Reviewers

Contact Us

Copyright Ownership