Multimodal Learning-based Prediction for Nonalcohol Fatty Liver Disease

Nonalcoholic fatty liver disease (NAFLD) is the most common cause of chronic liver disease, and if it is accurately predicted, severe fibrosis and cirrhosis can be prevented. While liver biopsies, the gold standard for NAFLD diagnosis, are intrusive, expensive, and prone to sample errors, noninvasive studies are extremely promising but are still in their infancy due to a dearth of comprehensive study data and sophisticated multimodal data methodologies. This paper proposes a novel approach for diagnosing NAFLD by integrating a comprehensive clinical dataset with a multimodal learning-based prediction method. The dataset comprises physical examinations, laboratory and imaging studies, detailed questionnaires, and facial photographs of a substantial number of participants, totaling more than 6000. This comprehensive collection of data holds significant value for clinical studies. The dataset is subjected to quantitative analysis to identify which clinical metadata, such as metadata and facial images, has the greatest impact on the prediction of NAFLD. Furthermore, a multimodal learning-based prediction method (DeepFLD) is proposed that incorporates several modalities and demonstrates superior performance compared to the methodology that relies only on metadata. Additionally, satisfactory performance is assessed through verification of the results using other unseen data. Inspiringly, the proposed DeepFLD prediction method can achieve competitive results by solely utilizing facial images as input rather than relying on metadata, paving the way for a more robust and simpler noninvasive NAFLD diagnosis.