Risk Assessment System of Fall in the Elderly Using Artificial Intelligence and Cloud Computing

Blasco-Garcia JD; Pavon-Pulido N; Lopez-Riquelme JA; Feliu-Batlle JJ; Nieto-Galera R; Herrero MT

Research Article

Phys Med Rehabil Int. 2022; 9(2): 1204.

Risk Assessment System of Fall in the Elderly Using Artificial Intelligence and Cloud Computing

Blasco-García JD¹, Pavón-Pulido N²*, López-Riquelme JA², Feliu-Batlle JJ², Nieto-Galera R³ and Herrero MT¹

¹Clinical and Experimental Neuroscience (NiCE), University of Murcia, Spain

²Department of Electrical Engineering and Electronic Technology, Technical University of Cartagena, Spain

³AFA Levante. C/ Alameda San Antón, 29. 30205 Cartagena, Spain

*Corresponding author: Nieves Pavón-Pulido, Department of Automation, Electrical Engineering and Electronic Technology, Technical School of Industrial Engineering, Technical University of Cartagena, Cartagena, Murcia, Spain

Received: July 14, 2022; Accepted: August 10, 2022; Published: August 17, 2022

Abstract

This paper presents a Cloud-based online tool for helping health professionals to predict the risk of falling in the elderly by using the well-known Tinetti’s Test. This tool implements a Deep Learning-based method for allowing several Tinetti scale’s items to be automatically estimated, simply using a conventional camera or a recorded video. From these sources of information, patients’ skeleton is recognized and their movements analyzed by applying some geometric calculations, which provide an objective risk assessment. Results are represented as a set of plots easily interpretable by experts. Several tests, in a controlled environment, have been carried out to validate the accuracy and reliability of the system. Moreover, some tests have been also made with real elderly patients, whose results have been evaluated by therapists. The benefits of using such remote tool for assessing (objective) fall risk, from a usability point of view, are also highlighted.

Keywords: Artificial intelligence; Tinetti scale; Falling risk; Telemedicine; Cloud computing

Abbreviations

IoT: Internet of Things; IMU: Inertial Measurement Unit; ML: Machine Learning; AI: Artificial Intelligence

Introduction

The COVID-19 pandemic has highlighted the need of improving health care systems through Digital Transformation, mainly regarding the use of online platforms capable of making remote attention easier, in a global connected world, where Cloud Computing and Internet of Things (IoT), have emerged as essential tools in many areas. This transformation could be particularly positive for vulnerable elderly people (dependent and multi-pathological), their families as well as caregivers [1].

A common risk that elderly people face is that related to falls which can cause chronic disabilities and economic burden [2] (particularly dangerous because many patients could seriously worsen their health status as a consequence of a fall). Then, it is essential to prevent these situations, and therapists use standard tests for assessing such risk, mainly based on the observation of patients’ movements. However, this procedure requires a face-to-face relationship and this is not always viable or easy. Moreover, simple observation is, in some measure, subjective, and it is not possible to properly store such measurements for further validation or comparison.

There exist some research lines focused on developing solutions [3] to incorporate sensors (accelerometers, Inertial Measurement Units (IMUs) or RGB-D devices), into the diagnostic process and automating fall risk assessment [4-11]. However, designing an automatic usable system capable of helping professionals to objectively assess the risk of falling (by remotely applying well-known standard tests), is still a challenge, especially if an artificial vision system, based on a conventional camera, is pretended to be used without the need of wearing other complementary sensors.

This paper presents a Cloud-based application, capable of evaluating the risk of falling in elderly people, which automatically detects and analyzes the human motion and applies the standard Tinetti’s Test for assessing such risk [12,13], taking a sequence of images, obtained by using a conventional camera, as input. The results are stored in the Cloud and they are presented to the therapist through graphical plots displayed in a WEB-based dashboard.

The outline of the paper is as follows. Section 2 describes the system architecture and details how the system works. Section 3 includes the obtained results and a discussion about the system performance. Section 4 addresses the conclusions and future work.

Materials and Methods

The system (Figure 1), which helps the therapist to remotely conduct and evaluate the Tinetti’s Test, has been designed using a Client/Server-based software architecture hosted in Google Cloud, where the server side allocates a remote data warehouse implemented by using the Google Data store, which allows the results of the tests to be safely stored, maintaining the privacy of the patient. In particular, an application running in the Google App Engine Standard Framework offers their services through the Google Endpoints Framework. Additionally, a WEB client (written in HTML5, JavaScript and CSS), enables therapists to access a dashboard (where their patients’ data and tests results are properly presented), and to remotely capture patients’ images through a single video call or by providing a video previously recorded. The video sequence is processed by a set of client-side modules written in JavaScript, which make use of the MediaPipe Machine Learning (ML) framework [14]. Such sequence should show the patient carrying out the specific movements needed by the Tinetti’s Test.

Figure 1: General scheme of the system architecture.


    

    

    Figure 1: General scheme of the system architecture.

Then the system performs skeleton tracking (a technique that enables the motion of human beings to be tracked), and generates a sequence of time stamped positions and angles of the links and joints of the skeleton model, which is used for obtaining a set of scores, defined by the Tinetti’s Test, by automatically calculating the value of several geometric features of interest, such as the arms separation, the legs separation or the leaning trunk, among others.

Human Body Segmentation Stage

Once the sequence of raw images is provided, the body segmentation stage allows human links and joints to be accurately identified by using the JavaScript-based “Pose” solution provided by the MediaPipe framework, which offers “high-fidelity body pose tracking, inferring 33 3D landmarks and background segmentation mask on the whole body from RGB video frames”. It also converts a 2D input into a 3D output by using ready-to-use Artificial Intelligence (AI) models.

In COCO topology (the current standard for representing the pose of the human body [15]), the key points are only located in the ankle and wrist joints, lacking information on the scale and orientation of the hands and feet. Therefore, the solution “Pose” uses “BlazePose” [16], a new topology of 33 key points of the human body (Figure 2), useful when estimation of more complex postures is required.

Figure 2: Pose Reference Points.


    

    

    Figure 2: Pose Reference Points.

Geometric Calculations

From the three-dimensional coordinates of the joints, obtained for each frame, after body segmentation, a set of parameters are calculated for automatically estimating the scores described by the Tinetti’s Test.

Such parameters should be selected by considering which human movements are anomalous and which could lead to increment the risk of falling. Several tests recognized for their effectiveness, specifically those of the Tinetti scale, have been chosen for helping in such selection.

Arms Separation: The use or not of arms (while walking or standing), is a good indicator of good balance and gait coordination [17]. Items 2, 9 and 15 (Tinetti scale), consider the position of the arms.

It is then interesting to evaluate the position of the arms respect to the trunk by calculating the parameters a and , which respectively define the separation of the right arm and the left arm respect to the trunk (Figure 3):

Figure 3: 3D points used for calculating the angle between the arms and the trunk.


    

    

    Figure 3: 3D points used for calculating the angle between the arms and

the trunk.

Where A, B, C, D, E and F are the 3D points respectively located at SHOULDER_RIGHT, ELBOW_RIGHT, HIP_RIGHT, SHOULDER_LEFT, ELBOW_LEFT, and HIP_LEFT.

Legs Separation: The separation of the legs (and feet), is also considered by the items 5 and 8 (Tinetti’s Test), related to balance and gait evaluation. The angle that defines the separation between legs has been selected as a relevant parameter, since if the patient spreads the feet while standing or walking, the angle will increase, and the risk of unbalance will be higher.

Since the “Pose” solution from MediaPipe only offers the 3D points: G (HIP_RIGHT) and H (HIP_LEFT), corresponding to the hips, and the 3D points: I (FOOT_RIGHT) and J (FOOT_LEFT), corresponding to the feet, the midpoint M (CENTER) located at the segment that joins such points (Figure 4), is obtained for calculating the angle which provides the needed information about the separation of legs:

Figure 4: Points needed to calculate the angle formed by the legs.


    

    

    Figure 4: Points needed to calculate the angle formed by the legs.

Where:

$https://latex.codecogs.com/svg.image?M=\frac{G+H}{2}$

Leaning Trunk: Maintaining an upright posture is a sign of a good state of balance and this is related to the inclination of trunk. The Tinetti scale considers this feature, then the proposed system calculates two angles as relevant parameters, by taking into account the position of the knees (Figure 5). By considering A as the 3D point that represents the SHOULDER_RIGHT, B the one for HIP_RIGHT, C the one for KNEE_RIGHT, and D, E and F as the 3D points for SHOULDER_LEFT, HIP_LEFT and KNEE_LEFT, respectively, the angles a and β are calculated:

Figure 5: Points and angles for calculating the inclination of the trunk.


    

    

    Figure 5: Points and angles for calculating the inclination of the trunk.

Trunk Swing: The swaying of the trunk is one of the traditionally most recognizable signs of loss of balance (see items 4, 5 and 15 of the Tinetti scale), both in balance and when walking. It has been empirically verified that measuring the difference in shoulder height is an effective method to estimate sway, then, “OFFSET” is another relevant parameter which is calculated according to:

Where y_A and y_B are the y-coordinates of the 3D points A and B, which represent the SHOULDER_RIGHT and the SHOULDER_LEFT (Figure 6). The representation of the OFFSET value as a time series provides useful information about balance to the therapist.

Figure 6: Points needed to measure the swing of the trunk.


    

    

    Figure 6: Points needed to measure the swing of the trunk.

Results and Discussion

Once the system calculates each parameter described above, a set of scores are computed, compatible with scoring scale used in Tinetti’s Test [13] (Table 1).

Table 1: Scoring system for transforming parameter values into Tinetti-inspired scores.








  

    

    SCORES

  

  

    0

    1

    2

  

  

    Arm angle 

    45

    25_ < 45

    < 25

  

  

    Trunk angle 

    < 55

    55_ < 85

    85

  

  

    Leg angle 

    30

    15_ < 30

    < 15

  

  

    Uneven shoulders (OFFSET) 

    0.1 m

    0.05_x<0.01 m

    < 0.05 m






Table 1: Scoring system for transforming parameter values into Tinetti-inspired

scores.

Table 2 shows how the features used for selecting the relevant parameters are scored according to the Tinetti score system. For example, if the patient does not use the arms to balance, the score is the highest. That means patient’s balance is suitable during the performance of the trial. Therefore, if the system detects that the arm angle is low or even 0, and the time series with the value of the arm angle through time does not suffer high variations, the system could estimate a good level of balance for such trial. The system provides a higher score if the arm angle is lower, since, according to the Tinetti score system, a low separation of arms means that the patient is not using them for maintaining balance. Similar criteria are used for deciding the scores related to the rest of parameters. As parameters are continuously calculated, they are stores as time series, represented as graphical plots ready to be analyzed by the therapist. The scores are continuously calculated according to the time stamped parameters values. A final mean value for each parameter is also calculated as the result of each trial, from each time series.

Table 2: Parameters scores according to the Tinetti scoring system.








  

    Parameter

    SCORES

  

  

    Separation of the arms

    The    patient does not use arms to balance.


      The patient partially uses arms to help himself.


      The patient uses his arms fully.

    = 2


      = 1


      = 0

  

  

    Leg spacing (feet)

    The    patient doesn’t need to spread his feet to support himself in a stable way.


      There is an appreciable separation of the feet in the search for support.


      There is marked distance between the feet.

    = 2


      = 1


      = 0

  

  

    Seated trunk tilt

    The    patient maintains an upright position while sitting.


      There is a slight tilt of the body forward.


      The patient is unable to stand by himself.

    = 2


      = 1


      = 0

  

  

    Trunk swing

    There is    no sway in posture.


      There is perceptible wiggle of the body.


      There is accented rocking of the trunk.

    = 2


      = 1


      = 0






Table 2: Parameters scores according to the Tinetti scoring system.

A trial is defined as a set of exercises proposed by the therapist to the patient, such as, walking in a straight line during a time, sitting down or getting up. The evaluation procedure of the proposed system consisted of two stages, one for validating the system (carried out by the researchers), and another carried out with a group of five real patients with neurodegenerative diseases. Trials consisted of performing a walking exercise in which three of the parameters were evaluated (separation of the arms, legs and trunk sway), and an exercise in which it was necessary to remain seated in a chair (for measuring the inclination of the trunk).

Figure 7 shows the results obtained after analysing the arm angle parameter during a trial performed in laboratory. The rest of parameters were also analysed, as Figure 8, Figure 9 and Figure 10 show.

Figure 7: Arms Separation.


    

    

    Figure 7: Arms Separation.

Figure 8: Legs Separation.


    

    

    Figure 8: Legs Separation.

Figure 9: Leaning Trunk.


    

    

    Figure 9: Leaning Trunk.

Figure 10: Trunk Swing.


    

    

    Figure 10: Trunk Swing.

This experiment allowed the system to be previously validated before being applied over real patients and it demonstrated that the system reliably calculates the needed parameters from a set of single frames acquired by a conventional camera, in real time.

The system represents not only the time series, but the 3D skeleton of the subject, which is also stored in the Cloud. This helps the therapist to properly understand the graphical plot, maintaining the privacy of patients (raw recorded videos or video calls are not remotely stored).

Table 3 shows the average of the parameter’s values measured in the exercises and their corresponding score for real patients. Such results have been validated by a physical therapist specialized in rehabilitation, who has determined the usefulness of the system as a diagnostic tool and the plausibility of the results.

Download PDF

Citation: Blasco-García JD, Pavón-Pulido N, López-Riquelme JA, Feliu-Batlle JJ, Nieto-Galera R and Herrero MT. Risk Assessment System of Fall in the Elderly Using Artificial Intelligence and Cloud Computing. Phys Med Rehabil Int. 2022; 9(2): 1204.

Home

Journal Scope

Editorial Board

Instruction for Authors

Submit Your Article

		PATIENTS
		P1	P2	P3	P4	P5
PARAMETERS	Arm angle	21°	32°	23°	19°	47°
	Score	2	1	2	2	0
	Trunk angle	91°	98°	101°	94°	82°
	Score	2	2	2	2	1
	Leg angle	8°	27°	24°	19°	30°
	Score	2	1	1	1	0
	OFFSET	0.02m	0.03m	0.02m	0.07m	0.12m
	Score	2	2	2	1	0

Risk Assessment System of Fall in the Elderly Using Artificial Intelligence and Cloud Computing

Abstract

Abbreviations

Introduction

Materials and Methods

Human Body Segmentation Stage

Geometric Calculations

Results and Discussion

Conclusion

Limitations

Conflicts of Interest

Acknowledgments

References