In this study, we propose a novel large language multimodal models (LLMMs) framework incorporating multimodal data from clinical notes and laboratory results for diabetes risk prediction. We collected five years of electronic health records (EHRs) dating from 2017 to 2021 from Far Eastern Memorial Hospital in Taiwan, which included 1,420,596 clinical notes, 387,392 laboratory results, and more than 1,505 laboratory test items.
By utilizing large language model and deep learning and integrating clinical notes and laboratory results, the proposed framework improves the predictive accuracy of new-onset T2DM while maintaining interpretability through SHAP-based visualizations. Results showed that an area greater than 0.70 was achieved under the receiver operating characteristic curve (AUC) for new-onset T2DM prediction, demonstrating the effectiveness of leveraging textual laboratory data for training and inference in LLMs and improving the accuracy of new-onset diabetes prediction. The findings highlight the potential of LLMMs to assist clinicians in making more informed, timely decisions about patient care.
【MORE】