Integrating Fine-Tuned IndoBERT into a Conversational Chatbot for Automated Ticket Categorization

Abstract

Purpose: University IT service desks frequently face operational bottlenecks due to manual ticket classification and high incident volumes. This study aims to develop an automated ticket categorization workflow by fine-tuning the IndoBERT model and embedding it into an interactive chatbot for real-time ticket categorization.

Methods: A dataset of 6,192 Indonesian-language service desk tickets was divided into training (70%), validation (15%), and test (15%) sets. Class-weighted training was applied to address class imbalance, while grid search was performed to optimize the learning rate, batch size, and number of training epochs. The optimal model was then integrated into the Rasa conversational framework via custom actions and connected to the existing web-based service desk through a REST API, with Telegram serving as the user interface.

Result: Fine-tuned IndoBERT achieved a test accuracy of 92.03% and a macro- and weighted-average F1-score of approximately 0.92, outperforming SVM (89.67%), Logistic Regression (88.37%), Random Forest (87.84%), and Naïve Bayes (81.27%). The integrated Telegram chatbot successfully provided automated initial guidance and submitted categorized tickets to the existing service desk platform.

Novelty: While many reviewed higher education chatbots focus on FAQ retrieval or intent recognition, this study establishes an end-to-end framework integrating deep learning contextual text classification with an open-source conversational agent and a web-based IT helpdesk system, supporting automated ticket categorization and submission for academic IT service management.