Automatic Scoring Using Term Frequency Inverse Document Frequency Document Frequency and Cosine Similarity
(1) Department of Informatics Engineering, Institut Teknologi Sumatera, Indonesia
(2) Department of Informatics Engineering, Institut Teknologi Sumatera, Indonesia
(3) Department of Informatics Engineering, Institut Teknologi Sumatera, Indonesia
(4) Department of Informatics Engineering, Institut Teknologi Sumatera, Indonesia
(5) Department of Informatics Engineering, Institut Teknologi Sumatera, Indonesia
(6) Faculty of Information and Communication Technology, Universiti Teknikal Malaysia Melaka, Malaysia
Abstract
Purpose: In the learning process, most of the tests to assess learning achievement have been carried out by providing questions in the form of short answers or essay questions. The variety of answers given by students makes a teacher have to focus on reading them. This scoring process is difficult to guarantee quality if done manually. In addition, each class is taught by a different teacher, which can lead to unequal grades obtained by students due to the influence of differences in teacher experience. Therefore the purpose of this study is to develop an assessment of the answers. Automated short answer scoring is designed to automatically grade and evaluate students' answers based on a series of trained answer documents.
Methods: This is ‘how’ you did it. Let readers know exactly what you did to reach your results. For example, did you undertake interviews? Did you carry out an experiment in the lab? What tools, methods, protocols or datasets did you use The method used is TF-IDF-DF and Similarity and scoring computation. Theword weight used is the term Frequency-Inverse Documents Frequency -Document Frequency (TF-IDF-DF) method. The data used is 5 questions with each question answered by 30 students, while the students' answers are assessed by teachers/experts to determine the real score. The study was evaluated by Mean Absolute Error (MAE).
Result: The evaluation results obtained Mean Absolute Error (MAE) with a resulting value of 0.123.
Value: The word weighting method used is the Term Frequency Inverse Document Frequency DocumentFrequency (TF-IDF-DF) which is an improvement over the Term Frequency Inverse Document Frequency (TF-IDF) method. This method is a method of weighting words that will be applied before calculating the similarity of sentences between teachers and students.
Keywords
Full Text:
PDFReferences
H. Sun and Z. Shao, "Research on the Application of Students' Answered Record Analyze Model and Question Automatic Classify Based on K-Means Clustering Algorithm," 2019 10th International Conference on Information Technology in Medicine and Education (ITME), 2019, pp. 494-497, doi: 10.1109/ITME.2019.00116.
G. Adrian, "B. and S", Roxana “Enacting textual entailment and ontologies for automated essay scoring in chemical domain” 16th Int. Symposium on Computational Intelligence and Informatics (CINTI2015) Budapest Hungary arXiv:1511.02669v1 19, November, 2017.
Z. Ke and V. Ng, "Automated essay scoring: A survey of the state of the art", Proc. 28th Int. Joint Conf. Artif. Intell., pp. 6300-6308, Aug. 2019.
S. Drolia, P. Agarwal, S. Rupani and A. Singh, "Automated Essay Rater using Natural Language Processing", International Journal of Computer Applications (, vol. 163, no. 10, pp. 0975-8887). Vol, April 2017.
G. Lv, W. Song, M. Cheng and L. Liu, "Exploring the Effectiveness of Question for Neural Short Answer Scoring System," 2021 IEEE 11th International Conference on Electronics Information and Emergency Communication (ICEIEC)2021 IEEE 11th International Conference on Electronics Information and Emergency Communication (ICEIEC), 2021, pp. 1-4, doi: 10.1109/ICEIEC51955.2021.9463814.
N. Chamidah, M. M. Santoni, H. N. Irmanda, R. Astriratma, L. M. Tua and T. Yuniati, "Word Expansion using Synonyms in Indonesian Short Essay Auto Scoring," 2021 International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS, Jakarta, Indonesia, 2021, pp. 296-300, doi: 10.1109/ICIMCIS53775.2021.9699374.H. A. Abdeljaber, "Automatic Arabic Short Answers Scoring Using Longest Common Subsequence and Arabic WordNet," in IEEE Access, vol. 9, pp. 76433-76445, 2021, doi: 10.1109/ACCESS.2021.3082408.
T. -H. Chang, J. -L. Chen, H. -M. Chou, M. -H. Bai, F. -Y. Hsu and Y. -C. Chen, "Automatic Scoring Method of Short-Answer Questions in the Context of Low-Resource Corpora," 2021 International Conference on Asian Language Processing (IALP), 2021, pp. 25-29, doi: 10.1109/IALP54817.2021.9675160.
U. Hasanah, T. Astuti, R. Wahyudi, Z. Rifai and R. A. Pambudi, "An Experimental Study of Text Preprocessing Techniques for Automatic Short Answer Grading in Indonesian," 2018 3rd International Conference on Information Technology, Information System and Electrical Engineering (ICITISEE), 2018, pp. 230-234, doi: 10.1109/ICITISEE.2018.8720957.
A. Mishra and S. Vishwakarma, "Analysis of TF-IDF Model and its Variant for Document Retrieval," 2015 International Conference on Computational Intelligence and Communication Networks (CICN), 2015, pp. 772-776, doi: 10.1109/CICN.2017.157
M. Ramya and J. A. Pinakas, "Different type of feature selection for text classification", International Journal of Computer Trends and Technology, vol. 10, pp. 102-107, 2017
A. N. Khusna and I. Agustina, "Implementation of Information Retrieval Using Tf-Idf Weighting Method On Detik.Com’s Website," 2018 12th International Conference on Telecommunication Systems, Services, and Applications (TSSA), 2018, pp. 1-4, doi: 10.1109/TSSA.2018.8708744
K Taghipour and HT Ng, "A neural approach to automated essay scoring", EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing Proceedings 1882–1891, 2016.
Pramono, L.H., A.S. Rohman, dan H. Hindersah.2013. Modified Weighting Method in TF*IDF Algorithm for Extracting User Topic Based on Email and Social Media in Integrated Digital Assistant. Joint International Conference on Rural Information & Communication Technology and Electric-Vehicle Technology (rICT & ICeV-T):1-6.
D. Palma and J. Atkinson, "Coherence-Based Automatic Essay Assessment," in IEEE Intelligent Systems, vol. 33, no. 5, pp. 26-36, 1 Sept.-Oct. 2018, doi: 10.1109/MIS.2018.2877278.
P. Lagakis and S. Demetriadis, "Automated essay scoring: A review of the field," 2021 International Conference on Computer, Information and Telecommunication Systems (CITS), Istanbul, Turkey, 2021, pp. 1-6, doi: 10.1109/CITS52676.2021.9618476.
Novia Puji Ririanti, Aji Purwinarko, “Implementation of Support Vector Machine Algorithm with Correlation-Based Feature Selection and Term Frequency Inverse Document Frequency for Sentiment Analysis Review Hotel”, Scientific Journal of Informatics, Vol. 8, No. 2, pp 297, November 2021.
G. Liu, K. Y. Lee, and H. F. Jordan, "TDM and TWDM de Bruijn networks and shufflenets for optical communications," IEEE Trans. Comp., vol. 46, pp. 695-701, June 2017.
J. O. Contreras, S. Hilles and Z. A. Bakar, "Essay Question Generator based on Bloom’s Taxonomy for Assessing Automated Essay Scoring System," 2021 2nd International Conference on Smart Computing and Electronic Enterprise (ICSCEE), Cameron Highlands, Malaysia, 2021, pp. 55-62, doi: 10.1109/ICSCEE50312.2021.9498166.
Gunawansyah, R. Rahayu, Nurwathi, B. Sugiarto and Gunawan, "Automated Essay Scoring Using Natural Language Processing And Text Mining Method," 2020 14th International Conference on Telecommunication Systems, Services, and Applications (TSSA, Bandung, Indonesia, 2020, pp. 1-4, doi: 10.1109/TSSA51342.2020.9310845.
A. A. Putri Ratna, L. Santiar, I. Ibrahim, P. D. Purnamasari, D. Lalita Luhurkinanti and A. Larasati, "Latent Semantic Analysis and Winnowing Algorithm Based Automatic Japanese Short Essay Answer Grading System Comparative Performance," 2019 IEEE 10th International Conference on Awareness Science and Technology (iCAST), Morioka, Japan, 2019, pp. 1-7, doi: 10.1109/ICAwST.2019.8923226.
P. Pathak, S. Raghav, S. Jain and S. Jalal, "Essay Rating System Using Machine Learning," 2021 5th International Conference on Information Systems and Computer Networks (ISCON), Mathura, India, 2021, pp. 1-6, doi: 10.1109/ISCON52037.2021.9702504.
Stephen Robertson, "Understanding Inverse Document Frequency: On Theoretical Arguments for IDF", England Journal of Documentation, vol. 60, pp. 502-520, 2017.
A. Ayob, "Comparison between conventional and digital essay writing assessment system: Consumer concept and user friendly", Research in World Economy, vol. 10, no. 2, pp. 96-101, 2019.
Y. Yang, L. Xia and Q. Zhao, "An Automated Grader for Chinese Essay Combining Shallow and Deep Semantic Attributes," in IEEE Access, vol. 7, pp. 176306-176316, 2019, doi: 10.1109/ACCESS.2019.2957582.
Patidar, A. K., J. Agrawal dan N. Mishra. 2012. Analysis of Different Similarity Measure Functions and Their Impacts on Shared Nearest Neighbor Clustering Approach. International Journal of Computer Applications. 40(16): 1-5.
Nazief, B. A. A. dan M. Adriani. 1996. Confix-Stripping : Approach to Stemming Algorithm for Bahasa Indonesia. International Conference on Information and Knowledge Management, : 560-563
W. Wang and Y. Lu, "Analysis of the Mean Absolute Error (MAE) and the Root Mean Square Error (RMSE) in Assessing Rounding Model", IOP Conference Series: Materials Science and Engineering, 2018.
H. Rababah and A. T. Al-Taani, "An automated scoring approach for Arabic short answers essay questions," 2017 8th International Conference on Information Technology (ICIT), Amman, Jordan, 2017, pp. 697-702, doi: 10.1109/ICITECH.2017.8079930.
Z Ke and V Ng, "Automated essay scoring: A survey of the state of the art", IJCAI International Joint Conference on Artificial Intelligence, pp. 6300-6308, August 2019.
Refbacks
- There are currently no refbacks.
Scientific Journal of Informatics (SJI)
p-ISSN 2407-7658 | e-ISSN 2460-0040
Published By Department of Computer Science Universitas Negeri Semarang
Website: https://journal.unnes.ac.id/nju/index.php/sji
Email: [email protected]
This work is licensed under a Creative Commons Attribution 4.0 International License.