Application of multi-modal medical image fusion in fundus diseases: a review

doi:10.3877/cma.j.issn.1674-0785.2026.03.012

Abstract

Abstract:

Fundus diseases are a leading cause of vision impairment, and their accurate diagnosis and treatment rely on high-quality medical imaging. Multi-modal medical image fusion (MMIF) integrates various fundus imaging modalities-such as optical coherence tomography, color fundus photography, and fundus fluorescein angiography-to compensate for the limitations of individual modalities in structural and functional assessment. This review systematically summarizes recent advances in the application of MMIF to major fundus diseases, including age-related macular degeneration (AMD), diabetic retinopathy (DR), retinal vein occlusion (RVO), and polypoidal choroidal vasculopathy (PCV), with a focus on lesion identification, disease classification, and differential diagnosis. The review aims to provide a valuable reference for further artificial intelligence-driven research and practice in the field of fundus diseases.

Key words: Multi-modal, Medical image fusion, Fundus diseases

Jiaona Jing, Jiaqi Yao. Application of multi-modal medical image fusion in fundus diseases: a review[J]. Chinese Journal of Clinicians(Electronic Edition), 2026, 20(03): 244-248.

/ / Recommend

Add to citation manager EndNote|Ris|BibTeX

URL: https://zhlcyszz.cma-cmc.com.cn/EN/10.3877/cma.j.issn.1674-0785.2026.03.012

https://zhlcyszz.cma-cmc.com.cn/EN/Y2026/V20/I03/244

References 45

1	Dogra A, Ahuja CK, Kumar S. Image integration procedures in multisensory medical images: a comprehensive survey of the state-of-the-art paradigms [J]. Curr Med Imaging, 2022, 18(5): 476-495.
2	Zhang C, Yang Z, He X, et al. Multimodal intelligence: representation learning, information fusion, and applications [J]. IEEE J Sel Top Signal Process, 2020, 14(2): 478-493.
3	Baltrušaitis T, Ahuja C, Morency LP. Multimodal machine learning: A survey and taxonomy update [J]. IEEE Trans Pattern Anal Mach Intell, 2022, 44(10): 6479-6496.
4	Haribabu M, Guruviah V, Yogarajah P. Recent advancements in multimodal medical image fusion techniques for better diagnosis: An overview [J]. Curr Med Imaging, 2023, 19(7): 673-694.
5	Bousquet E, Duraffour P, Debillon L, et al. Birdshot chorioretinopathy: A review[J]. J Clin Med, 2022, 11(16): 4772.
6	Siak J, Mahendradas P, Chee SP. Multimodal imaging in anterior uveitis [J]. Ocul Immunol Inflamm, 2017, 25(3): 434-446.
7	Acón D, Wu L. Multimodal imaging in diabetic macular edema [J]. Asia Pac J Ophthalmol (Phila), 2018, 7(1): 22-27.
8	Li J, Chen J, Tang Y, et al. Transforming medical imaging with transformers? A comparative review of key properties, current progresses, and future perspectives [J]. Medical Image Analysis, 2023, 85: 102762.
9	Azam MA, Khan KB, Salahuddin S, et al. A review on multimodal medical image fusion: compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics[J]. Comput Biol Med, 2022, 144: 105253.
10	Fonti R, Conson M, Del Vecchio S. PET/CT in radiation oncology [J]. Semin Oncol, 2019, 46(3): 202-209.
11	Xie C, Vardhanabhuti V. PET/CT: nasopharyngeal cancers [J]. PET Clin, 2022, 17(2): 285-296.
12	Tawfik N, Elnemr HA, Fakhr M, et al. Multimodal medical image fusion using stacked auto-encoder in NSCT domain [J]. J Digit Imaging, 2022, 35(5): 1308-1325.
13	Baltrusaitis T, Ahuja C, Morency LP. Multimodal machine learning: A survey and taxonomy [J]. IEEE Trans Pattern Anal Mach Intell, 2019, 41(2): 423-443.
14	贾紫婷. 多模态医学图像融合的关键技术研究 [J]. 计算机时代, 2020(7): 4-6, 11.
15	何俊, 张彩庆, 李小珍, 等. 面向深度学习的多模态融合技术研究综述 [J]. 计算机工程, 2020, 46(5): 1-11.
16	Huang R, Lin Z, Dou H, et al. AW3M: An auto-weighting and recovery framework for breast cancer diagnosis using multi-modal ultrasound [J]. Med Image Anal, 2021, 72: 102137.
17	Nie D, Zhang H, Adeli E, et al. 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients [J]. Med Image Comput Comput Assist Interv, 2016, 9901: 212-220.
18	Maqsood SU. Multi-modal medical image fusion based on two-scale image decomposition and sparse representation [J]. Biomed Signal Process Control, 2020, 57: 101810.
19	Alseelawi N, Hazim HT, Salim Alrikabi HT. A novel method of multimodal medical image fusion based on hybrid approach of NSCT and DTCWT [J]. Int J Online Biomed Eng, 2022, 18(3): 114-133.
20	Thakoor KA, Yao J, Bordbar D, et al. A multimodal deep learning system to distinguish late stages of AMD and to compare expert vs. AI ocular biomarkers [J]. Sci Rep, 2022, 12(1): 2585.
21	Chen M, Jin K, Yan Y, et al. Automated diagnosis of age-related macular degeneration using multi-modal vertical plane feature fusion via deep learning [J]. Med Phys, 2022, 49(4): 2324-2333.
22	Yoo TK, Choi JY, Seo JG, et al. The possibility of the combination of OCT and fundus images for improving the diagnostic accuracy of deep learning for age-related macular degeneration: a preliminary experiment [J]. Med Biol Eng Comput, 2019, 57(3): 677-687.
23	Chou YB, Hsu CH, Chen WS, et al. Deep learning and ensemble stacking technique for differentiating polypoidal choroidal vasculopathy from neovascular age-related macular degeneration [J]. Sci Rep, 2021, 11(1): 7130.
24	Xu Z, Wang W, Yang J, et al. Automated diagnoses of age-related macular degeneration and polypoidal choroidal vasculopathy using bi-modal deep convolutional neural networks [J]. Br J Ophthalmol, 2021, 105(4): 561-566.
25	Cennamo G, Montorio D, Fossataro F, et al. Evaluation of vessel density in disorganization of retinal inner layers after resolved diabetic macular edema using optical coherence tomography angiography [J]. PLoS One, 2021, 16(1): e0244789.
26	Sarao V, Veritti D, Lanzetta P. Automated diabetic retinopathy detection with two different retinal imaging devices using artificial intelligence: a comparison study [J]. Graefes Arch Clin Exp Ophthalmol, 2020, 258(12): 2647-2654.
27	Couturier A, Rey PA, Erginay A, et al. Widefield OCT-angiography and fluorescein angiography assessments of nonperfusion in diabetic retinopathy and edema treated with anti-vascular endothelial growth factor [J]. Ophthalmology, 2019, 126(12): 1685-1694.
28	Gong R, Han R, Guo J, et al. Quantitative evaluation of hard exudates in diabetic macular edema by multicolor imaging and their associations with serum lipid levels [J]. Acta Diabetol, 2021, 58(9): 1161-1167.
29	Hsu MY, Chiou JY, Liu JT, et al. Deep learning for automated diabetic retinopathy screening fused with heterogeneous data from EHRs can lead to earlier referral decisions [J]. Transl Vis Sci Technol, 2021, 10(9): 18.
30	范雯, 王晓玲, 马枭, 等. 基于超广角荧光素眼底血管造影图像行糖尿病视网膜病变分期的多模态深度学习模型研究 [J]. 中华眼底病杂志, 2022, 38(2): 139-145.
31	Hua CH, Kim K, Huynh-The T, et al. Convolutional network with twofold feature augmentation for diabetic retinopathy recognition from multi-modal images [J]. IEEE J Biomed Health Inform, 2021, 25(7): 2686-2697.
32	Nagasawa T, Tabuchi H, Masumoto H, et al. Accuracy of diabetic retinopathy staging with a deep convolutional neural network using ultra-wide-field fundus ophthalmoscopy and optical coherence tomography angiography [J]. J Ophthalmol, 2021, 2021: 6651175.
33	Yannuzzi LA, Shakin JL, Fisher YL, et al. Peripheral retinal detachments and retinal pigment epithelial atrophic tracts secondary to central serous pigment epitheliopathy. 1984 [J]. Retina, 2012, 32 Suppl 1: 1554-1572.
34	Imamura Y, Fujiwara T, Spaide RF. Fundus autofluorescence and visual acuity in central serous chorioretinopathy [J]. Ophthalmology, 2011, 118(4): 700-705.
35	Bujarborua D, Nagpal PN, Deka M. Smokestack leak in central serous chorioretinopathy [J]. Graefes Arch Clin Exp Ophthalmol, 2010, 248(3): 339-351.
36	Xu J, Wan C, Yang W, et al. A novel multi-modal fundus image fusion method for guiding the laser surgery of central serous chorioretinopathy [J]. Math Biosci Eng, 2021, 18(4): 4797-4816.
37	Xu J, Shen J, Jiang Q, et al. Research on the segmentation of biomarker for chronic central serous chorioretinopathy based on multimodal fundus image [J]. Dis Markers, 2021, 2021: 1040675.
38	Abdolrahimzadeh S, Di Pippo M, Ciancimino C, et al. Choroidal vascularity index and choroidal thickness: potential biomarkers in retinitis pigmentosa [J]. Eye (Lond), 2023, 37(9): 1766-1773.
39	Ayton LN, Guymer RH, Luu CD. Choroidal thickness profiles in retinitis pigmentosa [J]. Clin Exp Ophthalmol, 2013, 41(4): 396-403.
40	Jauregui R, Park KS, Duong JK, et al. Quantitative comparison of near-infrared versus short-wave autofluorescence imaging in monitoring progression of retinitis pigmentosa [J]. Am J Ophthalmol, 2018, 194: 120-125.
41	Liu T, Ling C, Hahn L, et al. Prediction of visual impairment in retinitis pigmentosa using deep learning and multimodal fundus images [J]. Br J Ophthalmol, 2023, 107(10): 1484-1489.
42	Yassin SH, Wang Y, Freeman WR, et al. Efficacy and accuracy of artificial intelligence to overlay multimodal images from different optical instruments in patients with retinitis pigmentosa [J]. Clin Exp Ophthalmol, 2023, 51(5): 446-452.
43	Kang EY, Yeung L, Lee YL, et al. A Multimodal imaging-based deep learning model for detecting treatment-requiring retinal vascular diseases: model development and validation study [J]. JMIR Med Inform, 2021, 9(5): e28868.
44	He X, Deng Y, Fang L, et al. Multi-modal retinal image classification with modality-specific attention network [J]. IEEE Trans Med Imaging, 2021, 40(6): 1591-1602.
45	Li X, Jia M, Islam MT, et al. Self-supervised feature learning via exploiting multi-modal data for retinal disease diagnosis [J]. IEEE Trans Med Imaging, 2020, 39(12): 4023-4033.

[1]	Yuxin Cao, Zhuojun Mao, Jiahe Liang, Jiangpu Yi, Zekai Zhang, Wenshuai Ma, Yuntao Chen, Xiaoqian Li, Yuxin Zhang, Tiesheng Cao, Lijun Yuan. Application of three-dimensional printed heart model in simulating clinical teaching of left atrial appendage closure [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2024, 21(06): 602-607.
[2]	Liuqing Peng, Peng Han, Nan Li, Shuyuan Liang, Xiang Fei. Value of multi-modal ultrasound in differential diagnosis of benign and malignant focal lesions of the salivary gland [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2022, 19(12): 1366-1372.
[3]	Xu Zhang, Dan Liu, Pintong Huang. Value of multi-modal ultrasound in diagnosis of cervical lymphoma [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2020, 17(10): 987-992.
[4]	Kunhe Li, Yi Li, Zhen Chen, Xianjun Huang, Liangcan Xiao. Comparison of treatments for throat complications after fast-track gastrointestinal surgery [J]. Chinese Archives of General Surgery(Electronic Edition), 2015, 09(06): 475-478.
[5]	Chunguang Yang, Basen Li, Qi Mei, Zhiquan Hu. The diagnosis and treatment of metastatic prostate cancer: a case repot and review of the literature [J]. Chinese Journal of Endourology(Electronic Edition), 2019, 13(06): 370-373.
[6]	Jingbo Li, Jianing Zhu, Luda Song, Yukun Luo, Qiuyang Li. Advances in application of multimodal ultrasound imaging in diagnosis of renal tumors [J]. Chinese Journal of Kidney Disease Investigation(Electronic Edition), 2023, 12(06): 349-353.
[7]	Huan Chen, Menghan Li, Weihong Yu. Pay attention to the application of artificial intelligence in constructing a three-tier screening and diagnosis-treatment system for ocular fundus diseases [J]. Chinese Journal of Ophthalmologic Medicine(Electronic Edition), 2025, 15(03): 129-134.
[8]	Xinyue Zhang, Shuai Han, Shushi Zhang, Wenchen Li, Shuyan Zhang. Application of intracranial pressure monitoring technique in the treatment of traumatic brain injury [J]. Chinese Journal of Neurotraumatic Surgery(Electronic Edition), 2023, 09(04): 246-252.
[9]	Chuangxi Liu, Hao Lin, Tianyou Pu, Qu Wang, Yunbiao Xiong. Practice of minimally invasive concept in microneurosurgery [J]. Chinese Journal of Neurotraumatic Surgery(Electronic Edition), 2020, 06(04): 248-250.
[10]	Yun Zhou, Yonggang Li, Bo Li, Qi Fang, Hui Wang. Clinical efficacy analysis of mechanical thrombectomy in patients with wake-up stroke based on multi-modal CT imaging [J]. Chinese Journal of Interventional Radiology(Electronic Edition), 2021, 09(01): 70-74.

Please choose a citation manager

Content to export