International Journal of Information Science and Management (IJISM)

Advancing Multi-Task Learning Techniques for Citation Context Classification: Sentiment, Role Source, and Function

Articles in Press

Document Type : Original Article

Authors

Yaniasih Yaniasih 1
Indra Budi 2

1 PhD. in Computer Science, Research Center for Data and Information Science, National Research and Innovation Agency, Jakarta, Indonesia.

2 Professor, Faculty of Computer Science, Universitas Indonesia, Depok, Indonesia.

10.22034/ijism.2026.2029231.1494
Abstract
The pattern of the in-text citation is significant in determining its value. This study focuses on advancing multi-task learning (MTL) techniques for citation-context classification: sentiment, role source, and function. Specifically, the research explores the integration of shared-trunk, cross-stitched, and shared-private networks with deep learning (DL) models, namely a convolutional neural network (CNN), a long short-term memory (LSTM), and a bidirectional LSTM. The dataset comprises 8566 sentences that include in-text citations of journal articles originating from Indonesia. To address the issue of unbalanced data, distinct loss weights were assigned to each output. Additionally, Bayesian optimization was employed to determine the hyperparameter sizes. The findings of the optimization process confirm that the learning rate is the most influential hyperparameter. The architectural complexity of the three MTL types varies, but this does not affect time efficiency, which is heavily influenced by the DL model employed. By achieving an F1 score of 0.87 for role source, 0.90 for function, and 0.81 for sentiment classification, the shared-private CNN model surpasses the current state-of-the-art model. The proposed model may improve citation analysis, making it beneficial for developing citation recommendation systems and conducting science evaluations.

Keywords

multi-task learning
deep learning
citation classification
hyperparameter optimization
imbalanced dataset
in-text citation
Subjects
Akiba, T., Sano, S., Yanase, T., Ohta, T. & Koyama, M. (2019, July). Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 2623-2631). https://doi.org/10.1145/3292500.3330701
Baig, Y. M., Oesterling, A. X., Xin, R., Yu, H., Ghosal, A., Semenova, L. & Rudin, C. (2021, June). Multitask learning for citation purpose classification. In Proceedings of the Second Workshop on Scholarly Document Processing (pp. 134-139). Association for Computational Linguistics. Retrieved from https://aclanthology.org/2021.sdp-1.18.pdf
Budi, I. & Yaniasih, Y. (2023). Understanding the meanings of citations using sentiment, role, and citation function classifications. Scientometrics, 128(1), 735-759. https://doi.org/10.1007/s11192-022-04567-4
Chen, J., Qiu, X., Liu, P. & Huang, X. (2018). Meta multi-task learning for sequence modeling. AAAI'18/IAAI'18/EAAI'18: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence (pp. 5070 – 5077). New Orleans, Louisiana, USA: AAAI Press.
Cohan, A., Ammar, W., Van Zuylen, M. & Cady, F. (2019, June). Structural scaffolds for citation intent classification in scientific publications. In Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: human language technologies, volume 1 (long and short papers) (pp. 3586-3596). Minneapolis, Minnesota. Association for Computational Linguistic https://doi.org/10.18653/v1/N19-1361
Crawshaw, M. (2020). Multi-task learning with deep neural networks: A survey. https://arxiv.org/abs/2009.09796
Ghosh, S., Das, D. & Chakraborty, T. (2018). Determining sentiment in citation text and analyzing its impact on the proposed ranking index. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9624 LNCS (pp. 292–306). https://doi.org/10.1007/978-3-319-75487-1_23
Chubin, D. E. & Moitra, S. D. (1975). Content analysis of references: adjunct or alternative to citation counting? Social Studies of Science, 5(4), 423-441. https://doi.org/10.1177/030631277500500403
Iqbal, S., Hassan, S. U., Aljohani, N. R., Alelyani, S., Nawaz, R. & Bornmann, L. (2021). A decade of in-text citation analysis based on natural language processing and machine learning techniques: an overview of empirical studies. Scientometrics, 126(8), 6551-6599. https://doi.org/10.1007/s11192-021-04055-1  
Justus, D., Brennan, J., Bonner, S. & McGough, A. S. (2018, December). Predicting the computational cost of deep learning models. In 2018 IEEE international conference on big data (Big Data) (pp. 3873-3882). IEEE. https://doi.org/10.1109/BigData.2018.8622396 
Lever, J., Krzywinski, M. & Altman, N. (2016). Classification evaluation. Nature Methods, 13(8), 603–605. https://doi.org/10.1038/nmeth.3945
Liu, P., Qiu, X. & Huang, X. J. (2017, July). Adversarial multi-task learning for text classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (volume 1: long papers) (pp. 1-10). https://doi.org/10.18653/v1/P17-1001
Misra, I., Shrivastava, A., Gupta, A. & Hebert, M. (2016). Cross-stitch networks for multi-task learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3994-4003). https://doi.org/10.1109/CVPR.2016.433
Moravcsik, M. J. & Murugesan, P. (1975). Some Results on the Function and Quality of Citations. Social Studies of Science, 5(1), 86–92. https://doi.org/10.1177/030631277500500106 
Nazir, S., Asif, M., Ahmad, S., Aljuaid, H., Iftikhar, R., Nawaz, Z., & Ghadi, Y. Y. (2022). Important citation identification by exploding the sentiment analysis and section-wise in-text citation weights. IEEE Access, 10, 87990-88000. https://doi.org/10.1109/ACCESS.2022.3199420
Nicholson, J. M., Mordaunt, M., Lopez, P., Uppala, A., Rosati, D., Rodrigues, N. P., Grabitz, P. & Rife, S. C. (2021). Scite: A smart citation index that displays the context of citations and classifies their intent using deep learning. Quantitative Science Studies, 2(3), 882–898. https://doi.org/10.1162/qss_a_00146
Ruder, S. (2017). An overview of multi-task learning in deep neural networks. https://doi.org/10.48550/arXiv.1706.05098
Sewak, M., Sahay, S. K. & Rathore, H. (2020, November). Assessment of the relative importance of different hyper-parameters of LSTM for an IDS. In 2020 IEEE REGION 10 CONFERENCE (TENCON) (pp. 414-419). IEEE. https://dx.doi.org/10.1109/TENCON50793.2020.9293731
Sharma, A., Van Rijn, J. N., Hutter, F. & Müller, A. (2019, October). Hyperparameter importance for image classification by residual neural networks. In International conference on discovery science (pp. 112-126). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-33778-0_10
Su, X., Prasad, A., Kan, M. & Sugiyama, K. (2019). Neural Multi-Task Learning for Citation Function and Provenance. JCDL '19: Proceedings of the 18th Joint Conference on Digital Libraries (pp. 394-395). https://doi.org/10.1109/JCDL.2019.00122
Swales, J. (1986). Citation analysis and discourse analysis. Applied Linguistics, 7(1), 39-56. https://doi.org/10.1093/applin/7.1.39
Tovedal, S. (2020). On the Effectiveness of Multi-Task Learning: An evaluation of multi-task learning techniques in deep learning models. Master’s thesis. Umeå University, Sweden. Retrieved from http://www.diva-portal.org/smash/record.jsf?pid=diva2:1442398
Wang, X., Lyu, J., Dong, L. & Xu, K. (2019). Multitask learning for biomedical named entity recognition with cross-sharing structure. BMC Bioinformatics, 20(1), 427. https://doi.org/10.1186/s12859-019-3000-5
Xu, H., Martin, E. & Mahidadia, A. (2014). Contents and time sensitive document ranking of scientific literature. Journal of Informetrics, 8(3), 546–561. https://doi.org/10.1016/j.joi.2014.04.006
Yaniasih, Y. & Budi, I. (2021). Systematic Design and Evaluation of a Citation Function Classification Scheme in Indonesian Journals. Publications, 9(27), 27. https://doi.org/10.3390/publications9030027
Yousif, A., Niu, Z., Chambua, J. & Khan, Z. Y. (2019). Multi-task learning model based on recurrent convolutional neural networks for citation sentiment and purpose classification. Neurocomputing, 335(28), 195-205. https://doi.org/10.1016/j.neucom.2019.01.021
Zhang, Y. & Yang, Q. (2018). An overview of multi-task learning. National Science Review, 5(1), 30-43. https://doi.org/10.1093/nsr/nwx105
Zhang, Y., Zhao, R., Wang, Y., Chen, H. & Mahmood, A. (2022). Towards employing native information in citation function classification. Scientometrics, 127(11), 6557-6577. https://doi.org/10.1007/s11192-021-04242-0
Zhao, D. & Strotmann, A. (2020). Deep and narrow impact: Introducing location filtered citation counting. Scientometrics, 122(1), 503-517. https://doi.org/10.1007/s11192-019-03280-z
Zhao, H., Luo, Z., Feng, C. & Ye, Y. (2019, July). A context-based framework for resource citation classification in scientific literatures. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1041-1044). https://doi.org/10.1145/3331184.3331348
Zhao, H., Luo, Z., Feng, C., Zheng, A. & Liu, X. (2019, November). A context-based framework for modeling the role and function of on-line resource citations in scientific literature. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 5206-5215). https://doi.org/10.18653/v1/d19-1524
 
International Journal of Information Science and Management (IJISM)

Articles in Press, Accepted Manuscript
Available Online from 08 August 2026

  • Receive Date 21 May 2024
  • Accept Date 08 August 2026

Home

Guide for Authors

Submit Manuscript

Reviewers

Contact Us

Copyright Ownership