Comparison of CNN- and Transformer-based Architectures for Automated Oral Epithelium Segmentation on Whole Slide Images

Oral cancer is one of the most commonly found cancers worldwide. Oral Epithelial Dysplasia (OED) is an Oral Potentially Malignant Disorder (OPMD) that can be characterized for preventive oral cancer screening. The standard for OED histological grading is conducted via the epithelial regions of tissue biopsies. However, this procedure is laborious, time-consuming, and subjective; consequently, it is prone to variability due to fatigue and limited expertise. Therefore, this study aims to explore the potential of using Convolutional Neural Network (CNN) and Transformer models for an automated epithelium segmentation algorithm directly from Whole Slide Images (WSIs). This approach can reduce the manual process and support pathologists in grading activities. Accordingly, candidate architectures based on CNN and Transformer are selected: UNet, ResNet50-UNet, VGG19-UNet, Swin-UNet, and MISSFormer. These models are trained using patch-based segmentation to mitigate the high computational cost caused by processing WSIs. The results indicate that UNet, optimized with the ADAM optimizer, demonstrates the best performance in patch-based segmentation with Intersection over Union (IoU) of 0.82 and Dice-Similarity Coefficient (DSC) of 0.87. Furthermore, this model achieves the highest IoU and DSC for tissue-level prediction, scoring 0.88 and 0.94, respectively. According to the experiment, overlapping and non-overlapping patching strategies perform similarly in most of the selected architectures. The latter approach, hence, is suggested for computational efficiency. These results can support enhancing automated epithelium segmentation to provide a reliable tool for assisting pathologists.