UTAUT-2, HOT-Fit, and PLS-SEM for User Acceptance and Success of the Face Recognition Feature in CAT BKN Application

Abstract

Purpose: Face recognition feature was implemented in the National Civil Service Agency's Computer-Assisted Test in 2021. There has been no evaluation of the system's acceptance and success. This study aims to measure user acceptance and evaluate the feature's success using the R Shiny application.

Methods: The study utilized 337 respondents from a Google Form-based questionnaire distributed throughout the Regional Office VII of the National Civil Service Agency in Palembang. The hybrid model used was UTAUT-2 and HOT-Fit, with PLS-SEM statistical analysis. Acceptance analysis and feature evaluation were conducted using the developed R Shiny Dashboard.

Results: The findings indicated that 15 of the 26 hypotheses were accepted. Behavioral intention and use behavior significantly influence hedonic motivation and habit. User behavior significantly influences user satisfaction, system quality, service quality, information quality, system use, and organizational structure and environment. As users become more familiar with the technology, their experience improves, and system utilization becomes more effective.

Novelty: The integration of UTAUT-2 and HOT-Fit models within an R Shiny Dashboard was applied to analyze user acceptance and evaluate the face recognition feature in Computer Computer-Assisted Test selection process. The findings provide recommendations for feature development and improving participant face recognition performance. Moreover, the R Shiny Dashboard can be adapted for user experience analysis and system evaluation in other contexts.