A prior segmentation knowledge enhanced deep learning system for the classification of tumors in ultrasound image

Breast and thyroid cancers are prevalent among women worldwide. Ultrasound (US) examination is widely used for the early detection of breast and thyroid cancers. However, due to the blurred tumor boundaries and irregular shapes, the computer-aided diagnosis (CAD) of tumors based on US is challenging. Numerous studies have introduced deep learning-based multi-task learning approaches to address this issue, but these methods may result in feature redundancy and misinformation. Tumor segmentation is a prerequisite step for US CAD, and a higher Dice coefficient is associated with more accurate classification outcomes. Therefore, this paper introduces a novel deep-learning system that fully utilizes segmentation knowledge to boost classification performance. The system starts with a hybrid convolutional neural network (CNN)-Transformer for tumor localization and coarse segmentation, then uses a lightweight CNN-based U-Net to refine segmentation results. Subsequently, segmentation knowledge is harnessed to augment the network input and enhance multimodal feature extraction, resulting in improved classification performance. Our proposed method yielded a Dice coefficient of 83.62% and 77.20% for breast and thyroid tumor segmentation and area under curve (AUC) values of 0.9536 and 0.9475 for their respective classifications. Compared to non-segmentation knowledge-based classification models, our method obtained an increase in AUC of 0.1054 and 0.0566 on the breast and thyroid datasets, respectively. It outperformed the performance of the State-Of-The-Art (SOTA) methods across various datasets. In summary, our proposed system shows promise for application in US tumor analysis and holds potential to be extended to additional diseases and modalities.