Phys Med Rehabil Int. 2022; 9(2): 1204.
Risk Assessment System of Fall in the Elderly Using Artificial Intelligence and Cloud Computing
Blasco-García JD¹, Pavón-Pulido N²*, López-Riquelme JA², Feliu-Batlle JJ², Nieto-Galera R³ and Herrero MT¹
1Clinical and Experimental Neuroscience (NiCE), University of Murcia, Spain
2Department of Electrical Engineering and Electronic Technology, Technical University of Cartagena, Spain
3AFA Levante. C/ Alameda San Antón, 29. 30205 Cartagena, Spain
*Corresponding author: Nieves Pavón-Pulido, Department of Automation, Electrical Engineering and Electronic Technology, Technical School of Industrial Engineering, Technical University of Cartagena, Cartagena, Murcia, Spain
Received: July 14, 2022; Accepted: August 10, 2022; Published: August 17, 2022
This paper presents a Cloud-based online tool for helping health professionals to predict the risk of falling in the elderly by using the well-known Tinetti’s Test. This tool implements a Deep Learning-based method for allowing several Tinetti scale’s items to be automatically estimated, simply using a conventional camera or a recorded video. From these sources of information, patients’ skeleton is recognized and their movements analyzed by applying some geometric calculations, which provide an objective risk assessment. Results are represented as a set of plots easily interpretable by experts. Several tests, in a controlled environment, have been carried out to validate the accuracy and reliability of the system. Moreover, some tests have been also made with real elderly patients, whose results have been evaluated by therapists. The benefits of using such remote tool for assessing (objective) fall risk, from a usability point of view, are also highlighted.
Keywords: Artificial intelligence; Tinetti scale; Falling risk; Telemedicine; Cloud computing
IoT: Internet of Things; IMU: Inertial Measurement Unit; ML: Machine Learning; AI: Artificial Intelligence
The COVID-19 pandemic has highlighted the need of improving health care systems through Digital Transformation, mainly regarding the use of online platforms capable of making remote attention easier, in a global connected world, where Cloud Computing and Internet of Things (IoT), have emerged as essential tools in many areas. This transformation could be particularly positive for vulnerable elderly people (dependent and multi-pathological), their families as well as caregivers .
A common risk that elderly people face is that related to falls which can cause chronic disabilities and economic burden  (particularly dangerous because many patients could seriously worsen their health status as a consequence of a fall). Then, it is essential to prevent these situations, and therapists use standard tests for assessing such risk, mainly based on the observation of patients’ movements. However, this procedure requires a face-to-face relationship and this is not always viable or easy. Moreover, simple observation is, in some measure, subjective, and it is not possible to properly store such measurements for further validation or comparison.
There exist some research lines focused on developing solutions  to incorporate sensors (accelerometers, Inertial Measurement Units (IMUs) or RGB-D devices), into the diagnostic process and automating fall risk assessment [4-11]. However, designing an automatic usable system capable of helping professionals to objectively assess the risk of falling (by remotely applying well-known standard tests), is still a challenge, especially if an artificial vision system, based on a conventional camera, is pretended to be used without the need of wearing other complementary sensors.
This paper presents a Cloud-based application, capable of evaluating the risk of falling in elderly people, which automatically detects and analyzes the human motion and applies the standard Tinetti’s Test for assessing such risk [12,13], taking a sequence of images, obtained by using a conventional camera, as input. The results are stored in the Cloud and they are presented to the therapist through graphical plots displayed in a WEB-based dashboard.
The outline of the paper is as follows. Section 2 describes the system architecture and details how the system works. Section 3 includes the obtained results and a discussion about the system performance. Section 4 addresses the conclusions and future work.
Materials and Methods
Figure 1: General scheme of the system architecture.
Then the system performs skeleton tracking (a technique that enables the motion of human beings to be tracked), and generates a sequence of time stamped positions and angles of the links and joints of the skeleton model, which is used for obtaining a set of scores, defined by the Tinetti’s Test, by automatically calculating the value of several geometric features of interest, such as the arms separation, the legs separation or the leaning trunk, among others.
Human Body Segmentation Stage
In COCO topology (the current standard for representing the pose of the human body ), the key points are only located in the ankle and wrist joints, lacking information on the scale and orientation of the hands and feet. Therefore, the solution “Pose” uses “BlazePose” , a new topology of 33 key points of the human body (Figure 2), useful when estimation of more complex postures is required.
Figure 2: Pose Reference Points.
From the three-dimensional coordinates of the joints, obtained for each frame, after body segmentation, a set of parameters are calculated for automatically estimating the scores described by the Tinetti’s Test.
Such parameters should be selected by considering which human movements are anomalous and which could lead to increment the risk of falling. Several tests recognized for their effectiveness, specifically those of the Tinetti scale, have been chosen for helping in such selection.
Arms Separation: The use or not of arms (while walking or standing), is a good indicator of good balance and gait coordination . Items 2, 9 and 15 (Tinetti scale), consider the position of the arms.
It is then interesting to evaluate the position of the arms respect to the trunk by calculating the parameters a and , which respectively define the separation of the right arm and the left arm respect to the trunk (Figure 3):
Figure 3: 3D points used for calculating the angle between the arms and the trunk.
Where A, B, C, D, E and F are the 3D points respectively located at SHOULDERRIGHT, ELBOWRIGHT, HIPRIGHT, SHOULDERLEFT, ELBOWLEFT, and HIPLEFT.
Legs Separation: The separation of the legs (and feet), is also considered by the items 5 and 8 (Tinetti’s Test), related to balance and gait evaluation. The angle that defines the separation between legs has been selected as a relevant parameter, since if the patient spreads the feet while standing or walking, the angle will increase, and the risk of unbalance will be higher.
Since the “Pose” solution from MediaPipe only offers the 3D points: G (HIPRIGHT) and H (HIPLEFT), corresponding to the hips, and the 3D points: I (FOOTRIGHT) and J (FOOTLEFT), corresponding to the feet, the midpoint M (CENTER) located at the segment that joins such points (Figure 4), is obtained for calculating the angle which provides the needed information about the separation of legs:
Figure 4: Points needed to calculate the angle formed by the legs.
Leaning Trunk: Maintaining an upright posture is a sign of a good state of balance and this is related to the inclination of trunk. The Tinetti scale considers this feature, then the proposed system calculates two angles as relevant parameters, by taking into account the position of the knees (Figure 5). By considering A as the 3D point that represents the SHOULDERRIGHT, B the one for HIPRIGHT, C the one for KNEERIGHT, and D, E and F as the 3D points for SHOULDERLEFT, HIPLEFT and KNEELEFT, respectively, the angles a and β are calculated:
Figure 5: Points and angles for calculating the inclination of the trunk.
Trunk Swing: The swaying of the trunk is one of the traditionally most recognizable signs of loss of balance (see items 4, 5 and 15 of the Tinetti scale), both in balance and when walking. It has been empirically verified that measuring the difference in shoulder height is an effective method to estimate sway, then, “OFFSET” is another relevant parameter which is calculated according to:
Where yA and yB are the y-coordinates of the 3D points A and B, which represent the SHOULDERRIGHT and the SHOULDERLEFT (Figure 6). The representation of the OFFSET value as a time series provides useful information about balance to the therapist.
Figure 6: Points needed to measure the swing of the trunk.
Results and Discussion
Once the system calculates each parameter described above, a set of scores are computed, compatible with scoring scale used in Tinetti’s Test  (Table 1).
25_ < 45
55_ < 85
15_ < 30
Uneven shoulders (OFFSET)
< 0.05 m
Table 1: Scoring system for transforming parameter values into Tinetti-inspired scores.
Table 2 shows how the features used for selecting the relevant parameters are scored according to the Tinetti score system. For example, if the patient does not use the arms to balance, the score is the highest. That means patient’s balance is suitable during the performance of the trial. Therefore, if the system detects that the arm angle is low or even 0, and the time series with the value of the arm angle through time does not suffer high variations, the system could estimate a good level of balance for such trial. The system provides a higher score if the arm angle is lower, since, according to the Tinetti score system, a low separation of arms means that the patient is not using them for maintaining balance. Similar criteria are used for deciding the scores related to the rest of parameters. As parameters are continuously calculated, they are stores as time series, represented as graphical plots ready to be analyzed by the therapist. The scores are continuously calculated according to the time stamped parameters values. A final mean value for each parameter is also calculated as the result of each trial, from each time series.
Separation of the arms
The patient does not use arms to balance.
The patient partially uses arms to help himself.
The patient uses his arms fully.
Leg spacing (feet)
The patient doesn’t need to spread his feet to support himself in a stable way.
There is an appreciable separation of the feet in the search for support.
There is marked distance between the feet.
Seated trunk tilt
The patient maintains an upright position while sitting.
There is a slight tilt of the body forward.
The patient is unable to stand by himself.
There is no sway in posture.
There is perceptible wiggle of the body.
There is accented rocking of the trunk.
Table 2: Parameters scores according to the Tinetti scoring system.
A trial is defined as a set of exercises proposed by the therapist to the patient, such as, walking in a straight line during a time, sitting down or getting up. The evaluation procedure of the proposed system consisted of two stages, one for validating the system (carried out by the researchers), and another carried out with a group of five real patients with neurodegenerative diseases. Trials consisted of performing a walking exercise in which three of the parameters were evaluated (separation of the arms, legs and trunk sway), and an exercise in which it was necessary to remain seated in a chair (for measuring the inclination of the trunk).
Figure 7 shows the results obtained after analysing the arm angle parameter during a trial performed in laboratory. The rest of parameters were also analysed, as Figure 8, Figure 9 and Figure 10 show.
Figure 7: Arms Separation.
Figure 8: Legs Separation.
Figure 9: Leaning Trunk.
Figure 10: Trunk Swing.
This experiment allowed the system to be previously validated before being applied over real patients and it demonstrated that the system reliably calculates the needed parameters from a set of single frames acquired by a conventional camera, in real time.
The system represents not only the time series, but the 3D skeleton of the subject, which is also stored in the Cloud. This helps the therapist to properly understand the graphical plot, maintaining the privacy of patients (raw recorded videos or video calls are not remotely stored).
Table 3 shows the average of the parameter’s values measured in the exercises and their corresponding score for real patients. Such results have been validated by a physical therapist specialized in rehabilitation, who has determined the usefulness of the system as a diagnostic tool and the plausibility of the results.
Table 3: Parameters values and scores measured in patients with neurodegenerative diseases.
The proposed system provides a remote Cloud-based application that, when it is not possible to work face to face, enables therapists to evaluate the risk of falling in the elderly, contributing to the Digital Transformation of Health system. Online systems should be developed to offer patients and health professionals usable digital spaces where remotely to carry out early diagnose tasks, better personalized attention and a better tracking of health status, mainly for elderly patients, who are one of the most vulnerable segments of population.
The proposed system (validated in a laboratory environment and with real patients), is aligned to such goal, since it allows therapists to remotely analyze the level of patients’ balance by using the Tinetti scoring system, through a video call or by using a previously recorded video with a conventional camera, in a reliable manner and accomplishing the required standards of privacy and security. It provides valuable information which allows the risk of falling to be anticipated, contributing to implement actions of preventive medicine, focused on avoiding dangerous situations for vulnerable elderly people.
As the system is still a prototype, it has not been tested with a statistically significant number of patients and therapists. Therefore, performing this kind of tests is one of the main future works. Additionally, it is necessary to demonstrate if the system is suitable in terms of usability and accessibility, for example, in elderly residences or with elderly people who live at home, with professional and not professional caregivers. The research team is also working in the development of a set of trials for validating the proposed system in such different scenarios.
Conflicts of Interest
No conflicts of interest.
This work has been partially supported by the project 21668/ PDC/21 Program Proof of Concept included in the “ProgramaSéneca 2021” Region of Murcia, Spain. This article is also the result of the activity carried out in the research groups: NEUROCOR (Neurotechnology, Control and Robotics) at the Technical University of Cartagena, Spain, and NiCE (Clinical and Experimental Neuroscience) at the School of Medicine, Campus Mare Nostrum, University of Murcia, Spain. The Alzheimer and other neurodegenerative dementias association “AFAL Cartagena y Comarca” deserves a special mention in this work for its support in providing contacts for carrying out trials with real patients.
- Plaza M, Avilés O, Aperador W (2012) Computer Vision Technology on Biomechanical Diagnosis. Proceedings of the 2012 International Conference on Image Processing, Computer Vision, and Patter Recognition (IPCV’12) 1:78-82.
- Montero-Odasso MM, Kamkar N, Pieruccini-Faria F, Osman A, Sarquis- Adamson Y, Close J, et al. Evaluation of Clinical Practice Guidelines on Fall Prevention and Management for Older Adults. JAMA Network Open. 2021;4(12):e2138911. doi:10.1001/jamanetworkopen.2021.38911
- Shany T, Redmond SJ, Narayanan MR, Lovell NH (2012) Sensors-based wearable systems for monitoring of human movement and falls. IEEE Sensors 12(3):658–670.
- Ayed I, Ghazel A, Jaume-I-Capó A, Moya-Alcover G, Varona J, Martínez- Bueso P. Feasibility of Kinect-Based Games for Balance Rehabilitation: A Case Study. Journal of Healthcare Engineering. 2018;2018:1-8. doi:10.1155/2018/7574860
- Ayed I, Jaume-i-Capó A, Martínez-Bueso P, Mir A, Moyà-Alcover G (2021) Balance Measurement Using Microsoft Kinect v2: Towards Remote Evaluation of Patient with the Functional Reach Test. Applied Sciences, 11:6073:1-21.
- Colagiorgio P, Romano F, Sardi F, Moraschini M, Sozzi A, Bejor M, et al. Affordable, automatic quantitative fall risk assessment based on clinical balance scales and Kinect data. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2014;:3500-3503. doi:10.1109/EMBC.2014.6944377
- Ortega-Bastidas P, Aqueveque P, Gómez B, Saavedra F, Cuerda RCL. Use of a Single Wireless IMU for the Segmentation and Automatic Analysis of Activities Performed in the 3-m Timed Up & Go Test. Sensors (Basel, Switzerland). 2019;19(7):1647. doi:10.3390/s19071647
- Pereiro AX, Campos-Magdaleno M, Navarro-Pardo E, Juncos-Rabadán O, Facal D. Normative scores for the Timed Up & Go in a Spanish sample of community-dweller adults with preserved functionality. Atencion Primaria. 2021;53(7):102065. doi:10.1016/j.aprim.2021.102065
- Rivolta MW, Aktaruzzaman M, Rizzo G, Lafortuna CL, Ferrarin M, Bovi G, et al. Evaluation of the Tinetti score and fall risk assessment via accelerometrybased movement analysis. Artificial intelligence in medicine. 2019;95:38-47. doi:10.1016/j.artmed.2018.08.005
- Roqueta C, Jaime ED, Miralles R, Cervera AM. Experiencia en la evaluación del riesgo de caídas: Comparación entre el test de Tinetti y el Timed Up & Go. Revista Española de Geriatría y Gerontología. 2007;42(6):319-327. doi:10.1016/S0211-139X(07)73570-9
- Similä H, Mäntyjärvi J, Merilahti J, Lindholm M, Ermes M. Accelerometry- Based Berg Balance Scale Score Estimation. IEEE Journal of Biomedical and Health Informatics. 2014;18(4):1114-1121. doi:10.1109/JBHI.2013.2288940
- Omaña H, Bezaire K, Brady K, Davies J, Louwagie N, Power S, et al. Functional Reach Test, Single-Leg Stance Test, and Tinetti Performance- Oriented Mobility Assessment for the Prediction of Falls in Older Adults: A Systematic Review. Physical therapy. 2021;101(10). doi:10.1093/ptj/pzab173
- Guevara CR, Lugo LH (2012) Validez y confiabilidad de la Escala de Tinetti para población colombiana. Revista Colombiana de Reumatologia, 19(4):218–233.
- Lugaresi C, Tang J, Nash H, McClanahan C, Uboweja E, Hays M, Zhang F, Chang C-L, Yong MG, Lee J, Chang W-T, Hua W, Georg M, Grundmann M, MediaPipe: A Framework forBuildingPerception Pipelines, arXiv:1906.08172.
- Cocodataset.org. (n.d.). COCO - Common Objects in Context. https:// cocodataset.org/#home. Accessed August 18, 2021.
- Bazarevsky V, Grishchenko I, Raveendran K, Zhu T, Zhang F, Grundmann M (2020) BlazePose: On-device Real-time Body Pose tracking, arXiv:2006.10204. https://doi.org/10.48550/arXiv.2006.10204.
- Swanson CW, Fling BW. Associations between gait coordination, variability and motor cortex inhibition in young and older adults. Experimental Gerontology. 2018;113:163-172. doi:10.1016/j.exger.2018.10.002
Citation: Blasco-García JD, Pavón-Pulido N, López-Riquelme JA, Feliu-Batlle JJ, Nieto-Galera R and Herrero MT. Risk Assessment System of Fall in the Elderly Using Artificial Intelligence and Cloud Computing. Phys Med Rehabil Int. 2022; 9(2): 1204.