AN INTERPRETABLE MULTIMODAL AI MODEL FOR PRE-OPERATIVE TRIAGE OF INDETERMINATE ADNEXAL MASSES: INTEGRATING CLINICAL TEXT ANALYSIS WITH RADIOMIC FEATURES

Abstract

Objective: To develop and validate an interpretable multimodal Artificial Intelligence (AI) model that integrates quantitative radiomic features from ultrasound and semantic features from unstructured clinical notes via Natural Language Processing (NLP) to improve the pre-operative prediction of adnexal mass malignancy. Design: A retrospective cohort, diagnostic accuracy study. Setting: Department of Obstetrics & Gynaecology, Fatima Memorial Hospital, Lahore, Pakistan. Population or Sample: Female patients aged 18 or older who underwent surgical removal of an adnexal mass and had complete pre-operative transvaginal ultrasound images and corresponding unstructured clinical consultation notes available. Methods: A custom Multimodal Neural Network (MNN) was developed, featuring separate sub-networks for radiomic features and BERT-derived clinical text embedding vectors, fused before the final sigmoid output layer. Performance was assessed on an external test set and compared to the IOTA ADNEX model using DeLong's Test. SHapley Additive exPlanations (SHAP) values were used for interpretability. Main Outcome Measures: Area Under the Receiver Operating Characteristic Curve (AUC-ROC), Sensitivity, Specificity, Positive Predictive Value (PPV), and Negative Predictive Value (NPV). Results: The MNN achieved an AUC of 0.93 (95% CI: 0.91–0.95), significantly outperforming the IOTA ADNEX model's AUC of 0.88 (95% CI: 0.86–0.90) ($p < 0.001$ by DeLong's Test). The MNN demonstrated superior Sensitivity ($\text{94.2}\%$) and NPV ($\text{96.8}\%$). SHAP analysis revealed high feature importance for NLP-derived features like "unintentional weight loss". Conclusions: The MNN offers superior diagnostic accuracy, specifically enhanced sensitivity and NPV, compared to established clinical scores for adnexal mass triage, validating the value of integrating implicit information from the clinical narrative. Funding: (No funding was provided).