Facial Recognition for Disease Diagnosis Using a Deep Learning Convolutional Neural Network: A systematic Review and Meta-Analysis

    


    


    Figure 7: Flowchart of face recognition technology, depicting the steps involved in identifying and verifying individuals based on facial features. The image was created using the Microsoft PowerPoint tool.

    

Application of Deep Learning in Facial Recognition of Genetic Diseases,

Since 2017, deep learning has been used in face recognition mainly for diagnosing genetic diseases. Gene mutations can lead to changes in the patient’s face, and many genetic diseases are reflected in facial features. In 2019, Yaron Gurovich and Y air Hanani et al. published an article in the Journal of Nature Medicine. They identified the facial phenotype of genetic disorders based on deep learning, which pushed the identification of genetic disorders through face recognition to the climax. In this paper, a facial image analysis framework called DeepGestalt is proposed, which uses computer vision and deep learning algorithms to quantify the similarity of hundreds of syndromes. In the initial experiment, DeepGestalt performed better than clinicians. The model was trained on a dataset consisting of more than 17000 images, which represent more than 200 syndromes and are managed by a community-driven phenotyping platform [34].

Angelman Syndrome (AS), also known as angel syndrome, arises from various genetic mechanisms that disrupt the maternal expression of the UBE3A gene, including deletion, Paternal Uniparental Disomy (UPD), UBE3A pathogenic variants, or imprinting defects. AS is characterized by developmental delays, excessive laughter, speech impairments, distinctive EEG patterns in epilepsy, microcephaly, and various facial features, including a large mouth, prominent tongue, midfacial depression, and protrusion [35]. Researchers analyzed image and molecular data of 261 AS patients aged from 10 months to 32 years using the facial recognition system DeepGestalt, and the analysis results demonstrated significant differences in distinguishing deletion subtypes from UPD or imprinting defects, with less differentiation among UBE3A pathogenic variant subtypes [36].

Cornelia de Lange syndrome (CdLS) is a genetic condition with severe neurodevelopmental issues, including mental retardation and distinctive facial features. It affects around 1/10,000 to 1/30,000 people [37]. Common CdLS facial features include thick eyebrows, long eyelashes, a small nose, and a drooping mouth. Some patients develop normally with a standard head circumference [38]. The application of DeepGestalt and Face2Gene technology is significantly conducive to genetic disease diagnosis and management. In a study of 49 individuals with known CdLS gene variants, Face2Gene showed an accuracy of 97.9% in clinical CdLS diagnosis [39].

Down’s syndrome or 21-trisomy syndrome, is a disease caused by additional staining on chromosome 21 [40]. Children who suffer from this disease have obvious special facial signs, such as wide eye distance, low nasal root, small eye fissure, lateral oblique, salivation, etc. Bosheng Qin and Letian Liang developed a recognition method for Down’s syndrome using facial images and the CNN network [41]. The training of the network is divided into two main steps: using 10562 subjects’ faces to form a general face recognition network, and then using the trained network for transfer learning. The data set of transfer learning is composed of 148 patients with Down’s syndrome and 257 healthy subjects. The accuracy of the CNN network in identifying Down syndrome was 95.87 % [42]. Turner’s Syndrome (TS) is a disease caused by partial or complete deletion of the X chromosome in female individuals [43]. The most common facial deformities of TS include a high arched palate, low posterior interline, pigmented nevus, small jaw deformity, and posterior rotation ears [44,45]. Zhouxian Pan and Zhen Shen et al. established a facial diagnosis model for TS. The training data included 170 TS patients and 1053 healthy subjects. 2 TS patients and 35 controls were used to test the efficacy in a real clinical environment. Experimental results showed that the average sensitivity and specificity of the study were 96.7 % and 97.0 %, respectively [46].

Williams-Beuren Syndrome (WBS), also known as cocktail syndrome, has a typical ‘elves’ facial profile. The face of ‘Little Elf’ is characterized by a wide forehead, periorbital edema, flat nose, etc [47]. Hui Liu and Zi Hua Mo conducted a study involving 104 WBS children, 145 healthy subjects, and 91 patients with other types of genetic syndrome. The classification performance of the face recognition model was evaluated by fivefold cross-validation and compared with human experts. The evaluation results showed that the VGG-19 model achieved the best performance, and the accuracy and precision were 92.7% and 94%, respectively [48]. Hang Yang and Xin-Rong Hud trained automatic facial recognition models on datasets composed of Noonan syndrome patients, healthy children, and subjects with several other malformation syndromes. A new DCNN framework with the arc loss function (DCNN-Arcface) was constructed, and transfer learning and data expansion were adopted in the training process. The DCNN-Arcface model was compared with two traditional machine learning models, the DCNN-CE model, and six doctors. Experimental results indicated that the DCNN-Arcface model achieved high accuracy in distinguishing NS from non-NS [48].

Genetic Syndromes (GSs) are a series of symptoms caused by genetic diseases [49]. Each specific genetic syndrome has specific characteristics according to the development affected by abnormal genes or chromosomes. Many GSs have obvious facial deformities, and facial gestalt can be used as a diagnostic tool for identifying syndromes [50]. Dian Hong and Ying Yi Zheng et al. collected 456 positive photos of 228 GS children and 228 healthy children. Meanwhile, the VGG-16 model was pre-trained by adopting the transfer learning method. Then, the VGGG-16 model was compared with five doctors, and the highest accuracy of the VGGG-16 model in screening GSs was 88% [51]. Rong-Min Baek et al. utilized a deep learning algorithm, the Multitask Cascaded Convolutional Neural Network (MTCNN), to develop a highly accurate facial recognition model for diagnosing Velocardiofacial Syndrome (VCFS). The model exhibited a high accuracy ranging from 94.03% to 99.78%, demonstrating its potential for early detection and treatment of rare genetic diseases in children, thereby offering support to healthcare professionals.

Application of Deep Learning Method in Facial Recognition of Non-Genetic Diseases,

In daily social activities, people tend to judge others’ health status according to their appearance. Numerous studies have shown that healthy people are often considered more attractive. This can be explained by Darwin’s evolutionary theory: this preference evolves because these ‘attractive’ features provide a signal of the quality of life, especially physical health, and keeping away from people who are considered less attractive (unhealthy) may prevent infectious diseases [52]. Most studies focused on the association between health status and facial features, such as skin color, facial obesity, and symmetry [53,54].

Facial features of acromegaly include tooth spacing widened, protrusion, frontal bone increased, nose increased, zygomatic arch prominent, and skin thickening [55]. Xiangyi Kong and Shun Gong et al. trained several popular machine learning algorithms such as Language Models (LM), K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Random Forests (RT), CNN and EM (Expectation Maximization) with a dataset consisting of 527 patients with acromegaly and 596 normal subjects. Then, the trained model was evaluated by a single dataset, and it was found that the CNN model achieved the highest accuracy of 96 % and specificity of 96 % [56]. As for children with autism, their faces have several characteristics: wider upper face, larger eyes, relatively short middle part of the face (including cheeks and nose), and a wide mouth [57,58]. Fawaz Waselallah Alsaade developed a model to help the community and psychiatrists detect autism through facial feature experiments. The pre-training models of Exception, VGG-19, and NASNET Mobile were used for classification tasks. Then, these models were tested on a dataset consisting of 2940 face images. The Exception model achieved the highest accuracy of 91%.

The most obvious early symptoms of Parkinson's disease include trembling, limb stiffness, motor dysfunction, and gait abnormalities. Many patients with Parkinson’s disease often have facial rigidity, no expression, reduced binocular rotation, and reduced blinks [59]. For these patients, it seems that their faces are wearing a stiff mask, so this symptom is also called ‘mask face’. Bo Jin and Yue Qu et al. collected facial expression videos from patients with Parkinson’s disease and matched controls. Then, relative coordinates and position jitter were adopted to extract facial expression features (facial expression amplitude and jitter of facial small muscles) from the key points of the face. Traditional machine learning algorithms and advanced deep learning algorithms were adopted to diagnose Parkinson’s disease. Experimental results indicated that applying the Long- and Short-Term Model (LSTM) neural network to extract key features can achieve an accuracy of 86%. Premature senescence syndrome refers to the phenomenon of physiological aging, physical decline, and psychological weakness in middle-aged people due to various reasons. A general framework for the diagnosis of neonatal premature failure syndrome was proposed in our previous study. Dhairya Chandra proposed an improved VGG-16 model, and they used a variety of machine learning algorithms and feature extraction tools. Results showed that the VGG-16 model obtained the best accuracy of 99.8%.

Facial dermatosis patients such as acne vulgaris, psoriasis, rosacea, and seborrheic dermatitis suffer from serious physiological and social problems. Evgin Goceri studied five common facial skin diseases that can lead to anxiety, depression, and even suicide. The pre-trained DenseNet201 structure and a new loss function were used to classify skin lesions. Experimental results indicated that the method can classify lesions efficiently (the highest accuracy is 95.24%) [60]. Facial neuritis, commonly known as facial paralysis, is a disease characterized by motor dysfunction of facial expression muscles. The general symptom is a skewed mouth and eye, and patients often cannot complete the most basic movements such as raising eyebrows, closing eyes, and drumming mouth [61]. Anping Song and Zuoyu Wu trained a single CNN in an end-to-end manner, with only pixels and disease labels as inputs of the model. The CNN was trained using the dataset of 1049 clinical images. With the help of neurologists, the dataset was divided into seven categories according to the classification criteria. The results of the model matched the level of neurologists, and the classification accuracy was 97.5%.

Cancer has become the second leading cause of death in the world. Most cancers are caused by gene mutation, which affects metabolism and leads to facial changes [62]. Bin Liang and Na Yang et al. collected facial images of cancer patients and established a dataset of cancer facial images. Meanwhile, according to the gender and age distribution of the cancer face image dataset, the face image dataset of non-cancer patients was established by randomly selecting images from the public Megage dataset. A residual neural network was constructed to classify cancer and non-cancer cases. Besides, the guided gradient weighted class activation mapping was used to reveal the related features. The study collected 8124 facial images of cancer patients in total. Experimental results indicated that the average facial obesity of male and female cancer patients is more obvious than that of non-cancer patients. On the test data set, the model accuracy is 82 % [63,64]. The typical features of beta-thalassemia include small foramen, epicanthus, low nose, flat middle face, short nose, smooth middle lip, thin upper lip, and underdeveloped mandible. BO JIN and LEANDRO CRUZ used relatively small datasets for computer-aided facial diagnosis of single and multiple diseases (beta-thalassemia, hyperthyroidism, Down’s syndrome, and leprosy). Through deep transfer learning of face recognition, the Top-1 accuracy can reach more than 90%, which is better than the performance of traditional machine learning methods and clinicians [65].

The prevalence of coronavirus 2019 is one of the most severe challenges facing the global health system today. Oleg Kit identified people under 50 years old who were susceptible to 2019 coronavirus disease by a non-invasive method. The pre-trained Dlib face recognition model was employed to analyze 525 facial photos of 2019 patients with different symptoms of the coronavirus disease. Meanwhile, a CNN was used to obtain the face description vector. On the test data set, the accuracy of binary classification for the individual severity of the coronavirus disease is 84%. Constitution classification is the basis and core of constitution research in traditional Chinese medicine. Er-Yang Huan and Gui-Hua Wen proposed a method for constitution classification based on transfer learning. First, the DenseNet-169 model trained with ImageNet was applied [66]. Then, the DenseNet-169 was carefully modified according to the physical characteristics, and the modified model was trained with clinical data to obtain the physical identification network ConstructionNet [67]. To further improve the classification accuracy, the model was combined with VGG-16, Inception v3, and DenseNet-121. Finally, the model was tested according to the idea of ensemble learning, and the composition type of the input face image was determined.

In 2022, researchers including Xinru Kong developed a depression recognition method using facial images and deep convolutional neural networks, which can effectively distinguish depressed patients from healthy individuals through binary classification. The trained neural network models achieved high accuracy, with the accuracy of the FCN model reaching 98.23%, demonstrating their potential for rapid and precise automatic depression identification [68]. Xiaofei Zhang and his colleagues proposed using a deep learning algorithm to recognize Schizophrenia (SCZ) patients based on their facial images. The study is divided into two parts, and the trained CNN achieved an accuracy of 95.18% in classifying "healthy control" or "SCZ patient." The research findings suggest that facial expressions have the potential to serve as indicators of SCZ and could be applied to recognition on mobile devices in clinical and everyday life.

Main Methods,

Since 2017, an increasing number of deep learning methods have been applied to identify diseases based on facial features. There are different innovations in the CNN architecture, mainly including parameter optimization, regularization, structural reorganization (convolution layer, pooling layer, and activation function), loss function, and fast processing. Note that different innovative approaches have been explored based on the survey of the latest CNN technologies for disease recognition and classification [69].

Meanwhile, three methods are adopted to design new CNN architectures. One is the structural modification of traditional models for diagnosing Down Syndrome, Noonan Syndrome, and other genetic disorders. Another is transfer learning used to diagnose Williams-Beuren syndrome, Autism spectrum disorder, progeria syndrome, facial dermatological disorders, beta-thalassemia, etc. The other is the construction and validation of deep learning models based on existing studies, such as the models developed for diagnosing genetic disorders, Corneliade Lange syndrome, Langer syndrome, and Angelman syndrome. Besides, other methods such as logistic regression, SVM, random forest, and machine learning are used to diagnose Parkinson's disease [70].

Features of the Ddisease Facial Dataset,

As a data-driven method, the face recognition method based on deep learning requires a large amount of training data, and the development of datasets also reflects the progress of face recognition technology. Since 2017, the technology of artificial intelligence has been successively applied to recognize human faces to identify diseases. Meanwhile, many disease datasets have been successively collected, such as the London Medical Database (LDM), the Face2Gene and the elife database, the Kaggle platform, and the FNP dataset. Specifically, LMD contains genetic and clinical photographs and information, which has been collected for decades. In 2016, LMD was specifically integrated into the Face2Gen library to broaden its application scenario. Next Generation Phenotyping (NGP) of FDNA is used by 70% of the world's geneticists in more than 2,000 clinical sites in 130 countries /regions to capture, construct, and analyze complex human physiological data to generate operable genomes. The FDNA database includes unprecedented deep phenotype and genotype information related to more than 10,000 diseases, which are derived from patients in the real world through extensive user networks. Kaggle was released in 2010, and it is an online platform for data mining and forecasting competitions. The dataset of autistic children came from the Kaggle platform, with a total of 2,940 face images, including 1470 face images of autistic children and 1470 face images of healthy children [58]. The FNP dataset comes from clinical images from the Department of Rehabilitation, Shanghai Tenth People's Hospital. Facial nerve paralysis (FNP dataset). It includes images of 377 males and 483 females, where 136 patients are aged under 40, 302 patients are middle-aged, and 422 patients are elderly.

Healthy Subject Data Set,

Numerous face datasets have been published for healthy subjects, including LFW, [71] YTF, [72] CASIA-Webface, [73] VGG2, [74] IJB-A, [75] UMDFace, [76] MegaFace, [77] CelebA, [78] CFP-FP, [79] IQIYI, [80] MS1M-IBUG, [81] Age Database-30, [82] CALFW, [71] IJB-B, [83] MS1M-ArcFace, [84] IMDB-Face, [85] Celeb500k, [86,87] CPLFW, [88] IJB-C, [89] Glint-Mini, Asian-Celeb, [91] and DeepGlint. [92] These datasets offer ample sample data for training face recognition algorithms. Table 3 provides a detailed overview of each dataset, including its name, release time, total number of face images, and description.

  


    
Data set

    
Time of publication

    
Number of face images and videos

    
Identity

  

  


    
LFW    (Zheng    et al., 2017) 

    
2007

    
13233

    
5749

  

  


    
YTF(Herrera    and Wen, 2020) 

    
2011

    
3 495 videos

    
3 495

  

  


    
CASIA-Webface    [(Yi    et al., 2014)] 

    
2014

    
494414

    
10575

  

  


    
VGG2    (Deng    et al., 2018) 

    
2015

    
3310000

    
9131

  

  


    
IJB-A[(Klare    et al., 2015)] 

    
2015

    
5712 face images and 2085 videos

    
500

  

  


    
UMDFace    [(Bansal    et al., 2017)] 

    
2016

    
367,888

    
8,277

  

  


    
MegaFace( Kemelmacher-Shlizerman et al., 2016) 

    
2016

    
4.7M

    
672000

  

  


    
CelebA    (Liu    et al., 2018) 

    
2016

    
202,599

    
10,177

  

  


    
CFP-FP    (Deng    et al., 2019) 

    
2016

    
7000

    
500

  

  


    
IQIYI(Wang    and Lobato, 2019) 

    
2016

    
500000 videos

    
5000

  

  


    
MS1M-IBUG    (Zhu    et al., 2021) 

    
2017

    
3.8M

    
85K

  

  


    
Age    Database-30 (Moschoglou    et al., 2017) 

    
2017

    
12240

    
570

  

  


    
CALFW    (Zheng    et al., 2017) 

    
2017

    
13233

    
5749

  

  


    
IJB-B    (Whitelam    et al., 2017) 

    
2017

    
11,754 face images and 7,011 videos

    
1,845

  

  


    
MS1M-ArcFace    (Razzhigaev    et al., 2020) 

    
2018

    
5.8M

    
85K

  

  


    
IMDB-Face    (Wang    et al., 2018) 

    
2018

    
1700000

    
59000

  

  


    
Celeb500k    (Cao    et al., 2018; Deng et al., 2020) 

    
2018

    
50000000

    
500000

  

  


    
CPLFW    (Zheng    and Deng, 2018) 

    
2018

    
13233

    
5749

  

  


    
IJB-C(Maza    et al., 2018) 

    
2018

    
138,000 face images and 11,000 videos

    
28,936

  

  


    
Glint-Mini    (Liu    et al., 2022) 

    
2020

    
5.2M

    
91K

  

  


    
Asian-Celeb    (Le    and Kakadiaris, 2020) 

    
2020

    
2800000

    
94000

  

  


    
DeepGlint    (Deng    et al., 2019b) 

    
2020

    
6.75M

    
181K

  

Table 3: Healthy Subject Data Set.