Textual Entailment for Non-Disclosure Agreement Contract Using ALBERT Method

Abstract

Purpose: NDA (Non-Disclosure Agreement) is one type of contract letter. An NDA binds two or more parties who all agree that certain information shared or created by one party is confidential. This type of contract serves to protect sensitive information, maintain patent rights, or control the information shared. Reading and understanding a contract letter is a repetitive, time-consuming, and labor-intensive process. Nevertheless, the activity is still crucial in the business world, as it can bind two or more parties under the law. This problem is perfect for Artificial Intelligence using Deep Learning. Therefore, this research aims to test and develop a pretrained language model that is designed for understanding contract letters through Natural Language Inference task.

Method The method used is to train model to perform the language inference task of textual entailment using CNLI (Contract NLI) dataset. ALBERT-base model version that has been tuned to perform textual entailment is used along with LambdaLR for early stopping and AdamW as optimizer. The model is pre-trained with CNLI dataset several times with multiple hyperparameter.

Result: As a result, the ALBERT base model that was used showed an accuracy score of 85 and EM score up to 85.04 percent. Although this score is not the State of the Art of the CNLI benchmark, the trained model can outperform other base versions of model that based on BERT and BART, like SpanNLI BERT-base, SCROLLS (BART-base) and Unlimiformer (BART-base).

Value: ALBERT is a model that focuses on memory efficiency and small size parameters while maintaining performance. This model is suitable for performing tasks that require long context understanding with minimum hardware requirements. Such a model could be promising for the future of NLP in the legal area.