International Journal of Information Science and Management (IJISM)

ISC World University Ranking: Its correlation with Leiden, Nature Index, Times Higher Education (THE) and Quacquarelli Symonds (QS)

Document Type : Articles

Authors

Maryam Yaghtin 1
Mansoureh Serati Shirazi 2

1 Institute for Science Citation (ISC), Research Department of Scientometric, Shiraz, Iran

2 institute for Science Citation (ISC), Research Department of university rankings, Shiraz, Iran

https://doi.org/10.22034/ijism.2023.1977954.0
Abstract
The present study intended to examine ISC World University Ranking (ISC WUR) as an emerging ranking system. It draws on a descriptive-correlational method. First, the internal correlation held between the criteria of the ISC WUR was examined to clarify the representation of the total university score from each of its functional criteria in the ISC WUR system. Afterwards, the study assessed the extent to which the ISC WUR results diverge or converge at the general level and examined the similarities and differences between ISC WUR criteria and indicators results and those of Leiden, Nature Index, THE, and QS as complicated university rankings. The research sample included the universities ranked via the five global ranking systems in 2020. All ISC WUR’s indicators revealed significant and positive correlations with the total ranking scores of the universities. There was a significantly positive correlation between the ISC WUR and those of Leiden, Nature Index, THE, and QS.
Moreover, the indicators similarly present in ISC WUR and other ranking systems were significantly and positively correlated. In conclusion, the overall performance of each university in ISC WUR reflects all the functional criteria of that university, especially the criteria of research and international activities. Besides, although the ICS WUR and the other systems have differences and similarities in methodology, the results obtained from these systems are similar to a certain extent. Nevertheless, these similarities are not strong enough to claim that the ISC WUR presents the same results as other ranking systems.

Keywords

University Ranking
ISC World University Ranking (ISC WUR)
Times Higher Education (THE)
QS
Leiden
Nature Index
  1. Aguillo, I., Bar-Ilan, J., Levene, M. & Ortega, J. (2010). Comparing university rankings. Scientometrics, 85(1), 243-256. https://doi.org/10.1007/s11192-010-0190-z
  2. Alma, B., Coşkun, E. & Övendireli, E. (2016). University ranking systems and proposal of a theoretical framework for ranking of Turkish universities: A case of management departments. Procedia-Social and Behavioral Sciences, 235, 128-138. https://doi.org/10.1016/j.sbspro.2016.11.008
  3. Anowar, F., Helal, M. A., Afroj, S., Sultana, S., Sarker, F. & Mamun, K. A. (2015). A critical review on world university ranking in terms of top four ranking systems. In: Elleithy, K., Sobh, T. (Eds) New Trends in Networking, Computing, E-learning, Systems Sciences, and Engineering(pp. 559-566). Springer, Cham. https://doi.org/10.1007/978-3-319-06764-3_72
  4. Baty, P. (2011). THE Global Rankings: Change for the better. Retrieved from https://www.timeshighereducation.com/world-university-rankings/2011-12/world-ranking/analysis/the-global-rankings
  5. Baty, P. (2014). Times higher education world university rankings, 2004-2012. Ethics in science and environmental politics, 13(2), 125-130. http://dx.doi.org/10.3354/esep00145
  6. Billaut, J. C., Bouyssou, D. & Vincke, P. (2010). Should you believe in the Shanghai ranking? An MCDM view. Scientometrics, 84(1), 237-263. https://doi.org/10.1007/s11192-009-0115-x
  7. Bookstein, F., Seidler, H., Fieder, M. & Winckler, G. (2010). Too much noise in Times Higher Education rankings. Scientometrics, 85(1), 295-299. https://doi.org/10.1007/s11192-010-0189-5
  8. Bornmann, L. & Glänzel, W. (2018). Which differences can be expected when two universities in the Leiden Ranking are compared? Some benchmarks for institutional research evaluations. Scientometrics, 115(2), 1101-1105. https://doi.org/10.1007/s11192-018-2700-3
  9. Bornmann, L. & Haunschild, R. (2017). An empirical look at the nature index. Journal of the Association for Information Science and Technology, 68(3), 653-659. https://doi.org/10.1002/asi.23682
  10. Çakır, M. P., Acartürk, C., Alaşehir, O. & Çilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103(3), 813-848. https://doi.org/10.1007/s11192-015-1586-6
  11. Chen, K. H. & Liao, P. Y. (2012). A comparative study on world university rankings: a bibliometric survey. Scientometrics, 92(1), 89-103. https://doi.org/10.1007/s11192-012-0724-7
  12. Chen, W., Li, Q., & Hildebrand, A. J. (2019). Comparing University Rankings: Statistical Analysis of Four Global University Ranking Systems. Nebraska Conference for Undergraduate Women in Mathematics, Lincoln, NE, Jan 25-27.
  13. Chen, W., Zhu, Z. & Jia, T. (2021). The rank boost by inconsistency in university rankings: Evidence from 14 rankings of Chinese universities. Quantitative Science Studies, 2(1), 335-349. https://doi.org/10.1162/qss_a_00101
  14. Cheng, S. K. (2011). World university rankings: Take with a large pinch of salt. European Journal of Higher Education, 1(4), 369-381. https://doi.org/10.1080/21568235.2012.662837
  15. CWTS (2019). Leiden Ranking 2019, Methodology. Retrieved from https://www.leidenranking.com/Content/CWTS%20Leiden%20Ranking%202019.pdf
  16. Daraio, C., Bonaccorsi, A. & Simar, L. (2015). Rankings and university performance: A conditional multidimensional approach. European Journal of Operational Research, 244(3), 918-930. https://doi.org/10.1016/j.ejor.2015.02.005
  17. Dobrota, M. & Dobrota, M. (2016). ARWU ranking uncertainty and sensitivity: What if the award factor was excluded? Journal of the Association for Information Science and Technology, 67(2), 480-482. https://doi.org/10.1002/asi.23527
  18. Doğan, G. & Al, U. (2019). Is it possible to rank universities using fewer indicators? A study on five international university rankings. Aslib journal of information management, 71(1), 18-37. https://doi.org/10.1108/AJIM-05-2018-0118
  19. Florian, R. V. (2007). Irreproducibility of the results of the Shanghai academic ranking of world universities. Scientometrics, 72(1), 25-32. https://doi.org/10.1007/s11192-007-1712-1
  20. Frenken, K., Heimeriks, G. J. & Hoekman, J. (2017). What drives university research performance? An analysis using the CWTS Leiden Ranking data. Journal of informetrics, 11(3), 859-872. https://doi.org/10.1016/j.joi.2017.06.006
  21. Haunschild, R. & Bornmann, L. (2015). Discussion about the new Nature Index. Scientometrics, 102(2), 1829-1830. https://doi.org/10.1007/s11192-014-1505-2
  22. Holmes, R. (2006). THES university rankings: Are they really world class? Asian Journal of University Education (AJUE), 2(1). Retrieved from https://education.uitm.edu.my/ajue/wp-content/uploads/2019/04/Chapter-1-The-THES-University-Rankings-Are-They-Really-World-Class.pdf
  23. Huang, M. H. (2011). A comparison of three major academic rankings for world universities: From a research evaluation perspective. Journal of Library & Information Studies, 9(1), 1-25. http://doi.org/10.6182/jlis.2011.9(1).001
  24. Huang, M. H. (2012). Opening the black box of QS World University Rankings. Research Evaluation, 21(1), 71-78. https://doi.org/10.1093/reseval/rvr003
  25. Hussein, K., Buhari, S. M., Tsaramirsis, G. & Basheri, M. (2017). A new methodology for ranking international universities. Indian Journal of Science and Technology, 10(36). https://doi.org/10.17485/ijst/2017/v10i36/104094
  26. Ioannidis, J. P., Patsopoulos, N. A., Kavvoura, F. K., Tatsioni, A., Evangelou, E., Kouri, I., Contopoulos-Ioannidis, D. G. & Liberopoulos, G. (2007). International ranking systems for universities and institutions: A critical appraisal. BMC medicine, 5(1), 30. https://doi.org/10.1186/1741-7015-5-30
  27. ISC (2019a). The results of the ISC global ranking in 2019 have been published. News. Retrieved from https://b2n.ir/g29525 [in Persian].  
  28. ISC (2019b). Our methodology. Retrieved from https://wur.isc.ac/Home/Methodology
  29. Ismail, E. M. (2010, December). Ranking of universities. In 2nd International Conference on Assessing Quality in Higher Education (pp. 6-8).
  30. Jajo, N. K. & Harrison, J. (2014). World university ranking systems: An alternative approach using partial least squares path modelling. Journal of Higher Education Policy and Management, 36(5), 471-482. https://doi.org/10.1080/1360080X.2014.936090
  31. Jöns, H. & Hoyler, M. (2013). Global geographies of higher education: The perspective of world university rankings. Geoforum, 46, 45-59. https://doi.org/10.1016/j.geoforum.2012.12.014
  32. Khosrowjerdi, M. & Kashani, Z. S. (2013). Asian top universities in six world university ranking systems. Webology, 10(2). Retrieved from https://www.webology.org/data-cms/articles/20200515052750pma114.pdf
  33. Liu, N. C. & Cheng, Y. (2005). The academic ranking of world universities. Higher education in Europe, 30(2), 127-136. https://doi.org/10.1080/03797720500260116
  34. Luke, G. & Walsh, A. (2010).World University Ranking. Retrieved from https://documents.manchester.ac.uk/display.aspx?DocID=8422
  35. Lukman, R., Krajnc, D. & Glavič, P. (2010). University ranking using research, educational and environmental indicators. Journal of cleaner production, 18(7), 619-628. https://doi.org/10.1016/j.jclepro.2009.09.015
  36. Moed, H. F. (2017). A critical comparative analysis of five world university rankings. Scientometrics, 110(2), 967-990. https://doi.org/10.1007/s11192-016-2212-y
  37. Moshfeghi, N. & Nadi, M. A. (2018). The comparative study of ranking system of islamic countries universities and national ranking of universities in iran using the most famous ranking systems in the world. Czech-Polish Historical & Pedagogical Journal, 10(2), 77-106. https://doi.org/10.5817/cphpj-2018-016
  38. Moshtagh, M. & Sotudeh, H. (2021). Correlation between universities’ Altmetric Attention Scores and their performance scores in Nature Index, Leiden, Times Higher Education and Quacquarelli Symonds ranking systems. Journal of Information Science, https://doi.org/10.1177/01655515211030868
  39. Nature Index (2018). Revisions to the Nature Index: A revised list of journals improves the balance of disciplines between the natural sciences. Retrieved from https://www.natureindex.com/news-blog/revisions-to-the-nature-index
  40. Olcay, G. A. & Bulu, M. (2017). Is measuring the knowledge creation of universities possible?: A review of university rankings. Technological Forecasting and Social Change, 123, 153-160. https://doi.org/10.1016/j.techfore.2016.03.029
  41. Rauhvargers, A. (2013). Global university rankings and their impact: Report II. Brussels: European University Association. Retrieved from https://eua.eu/downloads/publications/global%20university%20rankings%20and%20their%20impact%20-%20report%20ii.pdf
  42. Shehatta, I. & Mahmood, K. (2016). Correlation among top 100 universities in the major six global rankings: policy implications. Scientometrics, 109(2), 1231-1254. https://doi.org/10.1007/s11192-016-2065-4
  43. Soh, K. C. (2011). Don't read university rankings like reading football league tables: Taking a close look at the indicators. Higher Education Review, 44(1), 15-29. Retrieved from file:///C:/Users/Reza/Downloads/DontReadUniversityRankingsLikeReadingFootballLeaguesTableHER44-1.pdf
  44. Soh, K. C. (2015). Multicolinearity and indicator redundancy problem in world university rankings: An example using THEWUR 2013-2014 data. Higher Education Quarterly, 69(2), 158-174. http://dx.doi.org/10.1111/hequ.12058
  45. Soh, K. (2017). The seven deadly sins of world university ranking: A summary from several papers. Journal of Higher Education Policy and Management, 39(1), 104-115. https://doi.org/10.1080/1360080X.2016.1254431
  46. Sohail, S. S., Siddiqui, J., Shakil, M. T., Ubaid, S., Ahmed, J. & Alam, M. A. (2020). Sustainable Approach for University Ranking System. ICIDSSD. http://dx.doi.org/10.4108/eai.27-2-2020.2303115
  47. Tofallis, C. (2012). A different approach to university rankings. Higher Education, 63(1), 1-18. https://doi.org/10.1007/s10734-011-9417-z
  48. Van Raan, A. F. (2005b). Challenges in ranking of universities. In Invited paper for the First International Conference on World Class Universities, Shanghai Jaio Tong University, Shanghai(pp. 133-143).
  49. Van Raan, A. F. (2005a). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133-143. https://doi.org/10.1007/s11192-005-0008-6
  50. Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S. & van Raan, A. F. (2011). Towards a new crown indicator: Some theoretical considerations. Journal of informetrics, 5(1), 37-47. https://doi.org/10.1016/j.joi.2010.08.001
  51. Waltman, L., Calero‐Medina, C., Kosten, J., Noyons, E. C., Tijssen, R. J., van Eck, N. J., van Leeuwen, T. N., van Raan, A.F. J., Visser, M. S. & Wouters, P. (2012). The Leiden Ranking 2011/2012: Data collection, indicators, and interpretation. Journal of the American society for information science and technology, 63(12), 2419-2432. https://doi.org/10.1002/asi.22708
  52. Waltman, L. & Van Eck, N.J. (2015). Field-normalized citation impact indicators and the choice of an appropriate counting method. Journal of Informetrics, 9(4), 872-894. https://doi.org/10.1016/j.joi.2015.08.001

 

 

International Journal of Information Science and Management (IJISM)
Volume 21, Issue 2 - Serial Number 2
Spring 2023
Pages 271-288

  • Receive Date 24 September 2022
  • Revise Date 29 March 2023
  • Accept Date 29 March 2023

Home

Guide for Authors

Submit Manuscript

Reviewers

Contact Us

Copyright Ownership