Boundary-Aware Learning for Glioma Detection in MRI Using YOLOv8 Segmentation Supervision

Abstract

Purpose: Since glioma is the most aggressive and infiltrative type of brain tumor, its detection in magnetic resonance imaging (MRI) is especially difficult. Despite the excellent overall accuracy for brain tumor detection with YOLOv8-based object detection, the glioma-specific performance is limited owing to ambiguity of tumor boundaries. This work seeks to elucidate if boundary-aware learning can enhance glioma detection beyond typical bounding box–based approaches.

Methods: This study focuses exclusively on glioma detection using the Cheng brain tumor MRI dataset. YOLOv8 is used as the baseline detector, and boundary-aware learning is implemented through segmentation supervision using YOLOv8-Seg by leveraging pixel-level tumor masks. All the experiments are done in a standardized training environment to allow fair and unbiased comparison.

Result: Experimental evaluation shows that YOLOv8-Seg achieved a detection precision of 0.899, recall of 0.905, and mAP@50 of 0.940, while segmentation results achieved a mask precision of 0.900, recall of 0.904, and mAP@50 of 0.943. For glioma-specific analysis, the model achieved a box mAP@50 of 0.875 and a mask mAP@50 of 0.877. These results indicate that segmentation supervision improves spatial boundary representation even though improvements in conventional detection metrics remain marginal.

Novelty: Unlike the other works that are based on augmentation of the data and performance of better detection, this work has devised a glioma-centric design, and shows bounding box-based detection is insufficient. This work highlights the need for considering boundary aware learning applying the supervision of segmentation in the automated glioma detection system, which can improve the reliability and interpretability of the system.