Special Article - Stroke Recovery and Rehabilitation
Austin J Cerebrovasc Dis & Stroke. 2016; 3(2): 1046.
Increasing Access to Cost Effective Home-Based Rehabilitation for Rural Veteran Stroke Survivors
Hous ley SN¹, Garlow AR¹, Ducote K¹, Howard A¹, Thomas T¹, Wu D¹, Richards K¹ and Butler AJ1,2,3*
¹Department of Physical Therapy, Georgia State University, USA
²Department of Veteran’s Affairs, Atlanta Rehabilitation Research and Development Center of Excellence, USA
³Neuroscience Institute, Joint Center for Advanced Brain Imaging, Center for Behavioral Neuroscience, Georgia State University, USA
*Corresponding author: Butler AJ, B.F. Lewis School of Nursing and Health Professions, Department of Physical Therapy, 140 Decatur Street, Suite 819, Atlanta, GA 30303, USA
Received: July 16, 2016; Accepted: August 23, 2016; Published: August 25, 2016
Abstract
Introduction: An estimated 750,000 Americans experience a stroke annually. Most stroke survivors require rehabilitation. Limited access to rehabilitation facilities has a pronounced burden on functional outcomes and quality of life. Robotic devices deliver reproducible therapy without the need for real-time human oversight. This study examined the efficacy of using homebased, telerobotic-assisted devices (Hand and Foot Mentor: HM and FM) to improve functional ability and reduce depression symptoms, while improving access and cost savings associated with rehabilitation.
Methods: Twenty stroke survivors performed three months of home-based rehabilitation using a robotic device, while a therapist remotely monitored progress. Baseline and end of treatment function and depression symptoms were assessed. Satisfaction with the device and access to therapy were determined using qualitative surveys. Cost analysis was performed to compare home-based, robotic-assisted therapy to clinic-based physical therapy.
Results: Compared to baseline, significant improvement in upper extremity function (30.06%, p= 0.046), clinically significant benefits in gait speed (29.03%), moderate improvement in depressive symptoms (28.44%) and modest improvement in distance walked (30.2%) were observed. Participants indicated satisfaction with the device. Home-based robot therapy expanded access to post -stroke rehabilitation for 35% of the people no longer receiving formal services and increased daily access for the remaining 65%, with a cost savings of $2,352 (64.97%) compared to clinic-based therapy.
Conclusion: Stroke survivors made significant clinically meaningful improvements in the use of their impaired extremities using a robotic device in the home. Home-based, robotic therapy reduced costs, while expanding access to a rehabilitation modality for people who would not otherwise have received care.
Keywords: Stroke; Telerehabilitation; Veterans; Home-based; Rural
Abbreviations
HM: Hand Mentor; FM: Foot Mentor; UE: Upper Extremity; LE: Lower Extremity; AROM: Active Range of Motion; FIM: Functional Independence Measure; ADL: Activities of Daily Living; MCID: Minimal Clinically Important Difference; NIHSS: National Institutes of Health Stroke Scale; VA: Veterans Affairs; RUCA: Rural-Urban Commuting Areas; ARAT: Action Research Arm Test; 6MWT: 6 Minute Walk Test; 10MWT: 10 Meter Walk Test; CES-D: Center for Epidemiologic Studies Depression; HIPPA: Health Insurance Portability and Accountability Act of 1996; CIMT: Constraint- Induced-Movement-Therapy
Introduction
Stroke is one of the leading causes of long -term disability [1] with an estimated 795,000 incidences of stroke in the United States annually [2]. Of the approximate 665,000 survivors [3], 80% experience moderate to severe upper extremity (UE) impairments [4] and two-thirds experience lower extremity (LE) impairments [5]. Most require long-term rehabilitation to regain functional capacities required to perform activities of daily living and ambulation. This represents an approximately $34 billion cost to the healthcare system with an estimated average yearly rehabilitation cost of $11,689 per stroke survivor following acute and subacute rehabilitation discharge [6]. This substantial burden to the healthcare system has emphasized the need to investigate opportunities to improve care for stroke survivors while reducing mounting costs.
To date, best practice for successful rehabilitation often involves intensive, repetitive practice that actively engages the participant in goal-oriented and task-specific activities to regain functional capacities in upper and lower extremities [7]. Unfortunately, the quality of stroke services for rural patients is suboptimal and limited access to rehabilitation facilities has a pronounced burden on functional outcomes and quality of life. A recent study demonstrated, using logistic modeling, that rural stroke survivors were less likely to receive stroke rehabilitant therapy than their urban counterparts [8]. Moreover, with the prevalence of stroke being predicted to increase by almost 25% by 2030 [9] and rural populations being identified as being particularly vulnerable to stroke [10], there is a great need to develop accessible, cost effective therapy to minimize functional disability and optimize functional motor recovery for rural stroke survivors.
Robot-assisted therapy is a promising option for improving voluntary upper extremity (UE) movement in stroke survivors with finite access to conventional therapy [11-13]. Additionally, several recent studies have concluded that robotic assisted therapy improves lower extremity (LE) strength and locomotor function [14,15]. Recent advances in robot-assisted therapy have greatly increased the level of function patients can achieve. Successful rehabilitation techniques involve highly intensive, repetitious practice that actively engages the participant in goal-oriented and task-specific activities. Many studies have observed that home-based, robotic-assisted therapy demonstrate equivalent outcomes compared to one-on-one therapeutic delivery [11,16,17]. The results of these studies indicate that robot-assisted therapy provides reliable, reproducible treatment while measuring performance without the need for real-time human oversight [18].
Although the goals of using robotic assistive devices are to improve active range of motion (AROM), strength, and function in the distal musculature of stroke survivors is promising, these modalities are underutilized in the home. Therefore, combining telemedicine with in -home robot-assisted therapy (telerehabilitation) for people with residual impairment following stroke has the potential to reduce barriers while proving cost-effective, consistently highquality treatment to patients with limited access to rehabilitation clinics because of location or availability of treatment modalities [19]. This relatively new idea of telerehabilitation is defined as the provision of rehabilitation services at distance using information and communication technologies [20]. A recent systematic review examining studies published after 2000 cites positive outcomes for patients and caregivers who have utilized telerehabilitation [21]. Additionally, caregivers and patients report high levels of overall satisfaction and acceptance of telerehabilitation interventions. Recently, a prospective, single blinded, multisite, randomized controlled trial successfully paired robot-assisted therapy and a telerehabilitation intervention [22]. Equivalent outcomes were observed in the dose-equivalent robot-assisted therapy group and the usual and customary care group. Further, specific evaluation of LE robotic intervention found that 12 weeks of home-based rehabilitation elicited improvements in locomotor function and strength [14]. Recently, preliminary data investigating robotic telerehabilitation in stroke survivor’s homes, reported improvements in residual upper and lower limb impairments, while reductions in the cost of care decreased the burden on the healthcare system [15]. However, to date, a paucity of evidence regarding the efficacy and cost effectiveness of telerehabilitation interventions in rural stroke survivors persists and a knowledge gap exists as to what effect participation in telerehabilitation has on utilization of available therapy.
This study aimed to examine the efficacy of using a homebased, tele robotic-assisted device to: improve functional ability, reduce depression symptoms, and create a satisfactory experience, increase access to, and monitor participant utilization of cost efficient rehabilitation when compared to the cost of clinic - based therapy for rural stroke survivors.
Materials and Methods
Participants
The following inclusion criteria were utilized to screen volunteers for consideration for either hand or foot telerehabilitation: (1) between the ages of 45 and 90; (2) unilateral ischemic or hemorrhagic stroke within the previous 24 months; (3) Persistent UE or LE paresis as defined by having a score on the National Institutes of Health Stroke Scale (NIHSS) of 1-3 and a Functional Independence Measure (FIM) score of 17-88 that limited activities of daily living [23]; (4) Participants possessed some degree of upper or lower extremity voluntary activity, as indicated by the ability to move their proximal and/or distal joints against gravity.
Exclusion criteria were the following: (1) Those with clinically significant comprised mental status within three days of enrollment; (2) Severe receptive or expressive aphasia, as indicated by a score of 2 on item #11 Extinction and Inattention, a score o f 2 on item #8 Sensory, or a score of ≥1 on item #9 Best Language of the NIHSS, respectively; (3) Participants who were not independent before their stroke; (4) not able to follow simple instructions to operate the robotic user interface; (5) Prior Botox injections within six months of enrollment; (6) Additionally, due to the physical nature of the robotic rehabilitation therapy, volunteers with significant UE or LE contractures or injuries limiting the use of the more affected side were excluded.
Recruitment was centered on the Atlanta Veterans Affairs (VA) Medical Center. VA physicians and nurse practitioners screened a total of 31 veteran stroke survivors for eligibility and interest. A total of 20 mostly rural and highly rural stroke survivors with mean age of 67 (±11.4) years and a mean time since stroke of 20.4 months, ranging from 1.8-136.7 months met inclusion criteria and were enrolled. The 20 participants were comprised of: 1 female, 19 male, ten participants with UE impairments, and ten participants with LE impairments.
The VA-Office of Rural Health defines urban, rural, and highly rural by the Rural-Urban Commuting Areas (RUCA) system. Urban is defined as “Census tracts with at least 30 percent of the population residing in an urbanized area as defined by the Census Bureau.” “Highly Rural” is defined by as “Sparsely populated areas, less than 10 percent of the working population commutes to any community larger than an urbanized cluster. Rural is defined as “Land areas not designed as Urban or Highly Rural”.
Intervention
Following enrollment, participants were granted use of either a foot or hand, home-based robotic rehabilitation device. Home-based robotic rehabilitation was delivered using either the Hand (HM) or Foot Mentor (FM) ™ devices (Motus Nova, Inc. Atlanta, GA 30345). A study therapist trained in HM or FM arranged in-home setup and training with the volunteers. Each person was instructed to start at lower daily activity levels (one hour), progressing to the standard two -hour therapy dosage within the first week, for the three-month study duration. Due to the scheduling flexibility of the robotic device, participants were able to complete the two hours of daily prescribed robotic rehabilitation in any permutation. A review article by Linder, et al. [24] provides a more detailed rationale for the training principles and exercise dosage prescription for robotic-assisted therapy. Linder, et al. [24] also provides a detailed explanation of the implication of spasticity and tone on participation in robotic therapy.
The HM and FM devices were designed for use by individuals with residual upper and lower extremity impairments after stroke. The goal of using the device is to improve AROM and strength in the distal musculature of the paretic limb of patients with hemiparesis and weakness secondary to stroke through highly intensive, task-specific, and interactive practice [25,26]. The participants use their affected wrist or ankle to complete game-like training programs to challenge motor control initially at an easy level, requiring only a small degree of wrist or ankle motion. However, as participant’s motor control consistently improves (eight successes in ten attempts), the robotic device progresses difficulty levels, requiring greater AROM to achieve the goal. Conversely, if the user is experiencing difficulty (less than eight successful attempts out of ten), the device will decrease the difficulty level.
A study therapist remotely monitored daily metrics on a clinical dashboard through a secure server. Participant performance, including daily usage time and number of daily cycles (total and for each individual program), resistance to passive movement, passive range of motion angles and active range of motion angles achieved were monitored and discussed with each participant on weekly phone calls.
Clinical measures
Pre-assessment of all clinical measures was completed prior to receiving the robotic rehabilitation device by a study therapist trained in the use of standardized clinical measures. Following completion of the home-based therapy, post-intervention assessment of all the clinical measures was completed to assess changes from baseline. In addition, a qualitative satisfaction questionnaire was completed at the post - intervention assessment to ascertain participant perceptions of the home -based therapy.
Upper extremity functional limitations were evaluated using the Action Research Arm Test (ARAT) [27]. Quality of movement is scored on a 4-point ordinal scale (0-3), with a score of 3 indicating normal performance of the task within 5 s and a score of 0 indicating the inability to perform any part of t he task within 60 seconds. With a maximum score of 57, indicating normal performance, the test is comprised of 19 items divided into 4 subscales: grasp, grip, pinch, and gross movement. The ARAT is a valid and reliable tool for UE deficits following stroke [28,29] with previously defined minimal clinically important difference (MCID) [30].
Lower extremity functional status was assessed using the 10-meter walk test (10MWT) to measure gait speed [31] and the 6-minute walk test (6MWT) to measure gait performance over short distances [32]. Both the 10MWT and the 6MWT are valid and reliable measures to assess lower extremity function following stroke, with previous studies reporting MCIDs of 0.06 m/s [33] and 34.4m [34] respectively.
The Functional Independence Measure (FIM) instrument is a reliable and valid measure to assess motor and cognitive disability for stroke survivors as it relates to burden of care [35,36]. The FIM instrument is comprised of 18 items divided into two statistically and clinically separate indicators, of which 13 assess disability in motor functions and 5 in cognitive functions. The total scoring ranges from 18 (minimum) to 126 (maximum) and ranges for the motor and cognitive subscales are 13 to 91 and 5 to 35, respectively. Each item is scored on 7 levels of performance independence (7 is total independence, 1 is total dependence).
The Center for Epidemiologic Studies Depression (CES-D) scale is a questionnaire used to screen for depressive symptomology due to the possible impact on the quality of life of stroke survivors [37,38]. The test consists of 20 questions that capture how well a patient is coping emotionally. Scores range from 0-60 with scores greater than 16 indicating the patient is at risk for depression. The CES-D has been found reliable and valid for the subacute stroke population [39].
Usage and utilization
Both robotic devices are programmed to record a variety of relevant participant usage data. Daily usage and performance data were collected including: number of therapy sessions uses over the entire study duration, daily and average usage time. Overall device utilization is determined by calculating the ratio of number of uses over the number of days the devices are in the home. Device data were accessible remotely via secure server and adherence to the Health Insurance Portability and Accountability Act of 1996 (HIPAA) privacy rules were strictly maintained.
Satisfaction
Following completion of the home-based telerehabilitation therapy, a self-report questionnaire was distributed to participants to assess satisfaction and to provide greater insight into any unanticipated challenges of home-based robotic telerehabilitation. Both questionnaires included positive and negative statements about the device and required respondents to select one of seven choices anchored by: 0 = strongly disagree, 3 = neutral and 6 = strongly agree. Satisfaction questionnaires were evaluated based on how strongly patients agreed or disagreed with positive and negative statements. One additional question was included where participants were asked to state what they would change about the device.
Access
To assess the impact of the home-based, telerehabilitation on increasing access to therapy, participants current and past exposure to occupational or physical therapy was collected during initial assessment. Investigators asked participants either in person or via telephone whether or not they were participating in therapy during the time period they were using the device to determine the effect of the robotic therapy on Veteran access to treatment.
Cost analysis
Cost analysis was performed to compare home-based, telerehabilitation therapy to clinic-based physical therapy. Device and deployment costs were calculated, including the cost of home delivery, support, monitoring and connection, and pickup. Monthly maintenance and server connection costs were added to the device costs and were amortized to zero over the course of an expected usable lifetime of five years. Costs of in-home delivery were totaled across the 90-day treatment period and compared clinic-based physical therapy. Projected outpatient therapy transit and therapist costs for three one-hour sessions held weekly at the Atlanta VAMC were based on average patient mileage reimbursements and the projected cost of a physical therapist taken from the 2015 State of Georgia Bureau of Labor Statistics [40].
Statistical analysis
Data were checked for accuracy against data entry forms and expressed as means, SDs, and ranges using Microsoft Excel. Deidentified usage data for each participant were extracted from the secure server and entered into the Microsoft Access database. All remaining analyses were completed using SPSS, version 22 (IBM, Armonk, NY). Mean and percent change scores from baseline were calculated for each measure, along with 95% confidence intervals. Changes in functional outcome scores from baseline were analyzed using paired t-tests. Changes from baseline for all clinical outcome measures were compared to the corresponding estimated values for the minimum clinically important difference (MCID) in chronic stroke for these measures. Finally, mean scores were calculated for participant responses to surveys measuring satisfaction with use of the robotic devices. The level of significance was set at p≤0.05 and all tests were 2-tailed. All data met the assumptions of the tests used to analyze them. Summary data are presented as mean±standard deviation, unless otherwise noted.
Results
Figure 1 shows the flow of the participants through each stage of the study. Twenty, mostly rural and highly rural Veteran stroke survivors (67.0±11.4) years old at enrollment) with UE hemiparesis resulting from unilateral stroke (mean time since stroke of 20.4±9.26) months) met inclusion criteria and were enrolled in this study. All 20 participants showed UE or LE impairment secondary to corticospinal tract infarcts. Overall the home-based, telerehabilitation therapy was safe and well tolerated. No adverse events occurred. Participant demographic and geographic location information is presented in Table 1. Nineteen participants completed the study, consisting of three months of home-based, telerehabilitation. One participant in the HM group dropped out after device deployment due to medical reasons unrelated to the study. This participant did not complete postintervention assessments and the data was excluded from all analyses. Means and SD for clinical outcome measures for participants at baseline and postintervention are presented in Table 2.
Figure 1: Participant flow through each stage of the study.
Robotic-Assisted Therapy
Upper extremity, n=10
Lower extremity, n=10
Total, n=20
Male, gender, n (%)
9 (90)
10 (100)
19 (95)
White/African American
9/1
8/2
17/3
Education (years)
12.8
12.4
13
Mean time since stroke, months (range)
21.5 (1.8-136.7)
19.4 (3.5-67.7)
20.4 (1.8-136.7)
Mean age (years) at enrollment, mean (SD)
63.4 (9.1)
70.6 (12.7)
67 (11.4)
Dominant Right hand/foot, n (%)
10 (100)
10 (100)
20 (100)
6 (60)
4 (40)
10 (50)
Withdrew after baseline visit
1
0
1
Distance from VAMC miles (SD)
76.08
(35.0)
53.64
(35.5)
63.32
(35.8)
Time device in home in days (SD)
119.6
(39.4)
105.4
(43.3)
112.9
(40.8)
Table 1: Baseline characteristics of participants and subdivided into device groups. Distance from VA medical center (VAMC) represents the mean distance for each participant to the closest VAMC that can provide comparable clinic based Physical Therapy.
Outcome Measure
Baseline
Post-intervention
Mean difference (%)
95% CI
Sig, p
FIM
97.67
96.13
-1.53 (1.57)
-8.45 to 5.38
0.642
CES-D
12.41
8.88
-3.53 (28.44)
-8.61 to 1.55
0.160
10 meter walk test (m/s)
0.31
0.40
0.09 (29.03) ŧ
-0.11 to 0.29
0.197
6 minute walk test (m)
59.70
77.73
18.03 (30.20)
-12.06 to 48.13
0.153
ARAT (unaffected)
50.22
55.78
5.56 (11.07)
-5.89 to 17.00
0.295
ARAT (affected)
30.67
39.89
9.22 (30.06) ŧ
0.38 to 18.07
0.046*
*p<0.05; ŧ = MCID; CI: Confidence Interval; FIM: Functional Independence Measure; CES-D:
Epidemiologic Studies Depression; ARAT: Action Research Arm Test
Table 2: Clinical outcome measures at baseline and three months post-intervention.
Clinical measures
Compared to baseline, affected UE ARAT scores improved by an average of 9.22 (26.13) points. The observed change represents a statistically (30.06%, p= 0.046) significant improvement in upper extremity function. Further, the average improvement surpassed the previously validated MCID of 5.7 [41] indicating a clinically meaningful change in upper extremity function.
On average, participants using the FM device demonstrated a 29.03% increase in gait speed, from 0.31 to 0.40m/s. The observed change in gait speed represents a small but clinically significant improvement (0.09m/s, p=0.197). Based on previously validated stratification on gait speed and predicted functional walking capacity [42], the changes observed following the home-based telerehabilitation demonstrates the stroke survivors improved from home ambulators to limited community ambulators.
For participants who used the FM device, modest, non-significant (30.2%, p=0.153) improvements in total distance walked were observed from baseline (59.7±51.0) to post-intervention (77.7±32.9).
Self reported depressive symptoms over the past week, as assessed by the CES-D, demonstrated a moderate (28.44%, p=0.160) decrease from baseline (12.4±14.2) to post-intervention (8.9±9.3), indicating a lower prevalence of depressive symptoms at the final assessment.
Across all participants using either the FM or HM, no change (p=0.642) in FIM scores was observed from baseline (97.7) to postintervention (96.1).
Usage and utilization
Summaries of device exposure, usage, and utilization are presented in Table 3. Throughout the three-month study period, participants were given access to either the HM or FM device for an average of 106 days (exposure). Participants completed an average of 30.6 training sessions throughout the study. However, relatively wide individual variability was observed, with participants using the device as little as two and as many as 75 sessions. For each daily usage, participants performed robotic telerehabilitation an average of 90.6 min (range 15-153.6 min). Given the above information, participants utilized either the HM or the FM devices an average of 29.3% of the days the device was in the home. Large heterogeneity in device utilization was observed, ranging from 2.75% to 76.53%.
ID
Mean daily therapy minutes
Usage (sessions)
Exposure (days)
Utilization
1
100.3
66
105
62.86%
2
85.2
9
105
8.57%
3
121.1
15
105
14.29%
5
15.0
3
109
2.75%
6
113.4
14
110
12.73%
8
153.6
73
98
74.49%
9
62.7
75
98
76.53%
11
106.9
8
43
18.60%
13
75.5
2
78
2.56%
19
72.36
41
209
19.62%
Total
90.61
30.6
106
29.3%
Table 3: Usage, exposure, and device utilization. Usage is defined as the number of uses over the duration the device was in the home. Exposure is defined as the time the device was in the home. Utilization is the ratio of usage over exposure and represents how frequently the device was used on a daily basis.
Satisfaction
Following completion of the intervention, 17 participants responded to 15 statements to measure satisfaction with the HM or FM device (Table 4). Overall, participants indicated satisfaction with the device and their overall improvement. For most of the statements, a response of “6” is the most positive response; however for four of the statements a response of “0” is the most positive (strongly disagreeing with the statement). Subjects’ responses on these four questions indicated that they either slightly or strongly disagreed with negative statements about the device. For the positive statements, participants indicated that they either slightly or strongly agreed (mean≥4.0). Participants generally disagreed with the negative statements (mean ≤ 2.0), however the mean response to the statement concerning donning and doffing the peripheral hand or foot piece was 3.5, indicating that participants had difficulty with this aspect of using the HM or FM.
Statement
Mean (SD)
%
1. The instructions for using the Hand or Foot Mentor were clear and easy to understand
5.94 (0.25)
99%
5.19
(0.83)
91%
3. My function was improved
4.44
(1.79)
82%
4. The games were appropriate
5.16
(1.23)
86%
5. The games were hard to see
1.09
(2.03)
27%
6. This therapy challenged me
4.81
(1.68)
74%
7. This therapy was too difficult
1.14
(1.56)
9%
8. This therapy was too easy
2.00
(1.96)
31%
9. The device was user-friendly
4.86
(1.70)
85%
10. I got bored with the games
2.00
(2.42)
39%
11. I enjoyed playing the games
5.43
(1.02)
91%
12. The pace of therapy was just right
5.00
(1.57)
85%
13. I had trouble donning and doffing the Hand or Foot piece
3.53
(2.22)
61%
14. Overall, I am satisfied with the progress I made using the Hand/Foot Mentor
5.09
(1.21)
89%
15. Therapy with the Hand/Foot Mentor met my expectations
5.13
(1.31)
91%
Table 4: Exit interview comprised of subjective statements aimed at capturing satisfaction with using either the HM or FM device. Items, means (SD), and percent agreement for participant responses to satisfaction survey on a 0-6 Likert Scale, where 0 is Strongly Disagree and 6 is Strongly Agree, n=17 (of 20).
Access
Following the study therapist phone or in -person survey, it was determined that 7 (35%) participants were no longer receiving formal physical or occupational therapy services. The remaining 13 (65%) participants were receiving some form of formal services. Eight (40%) participants were receiving outpatient therapy services, of which, three (15%) sought out private services outside of the VA health system. The remaining five (25%) participants received home health care physical or occupational therapy. Participants reported between utilizing formal therapy services between two times per month and two times per week. For all participants mean access to therapy services was estimated as 1.2 session per week or 14.7 sessions for the duration of the study (assuming four weeks per month). Therefore, when considering the device usage data (Table 4), the home-based, robotic therapy intervention increased or extended access for all participants. Exploratory analysis to assess the affect of the home-based, telerehabilitation on those participants who are already receiving formal services demonstrates that the present study increases access from 22.6 sessions to 53.2 sessions, representing a 135.4% increase in rehabilitation exposure.
Cost analysis
Total home-based, telerehabilitation costs were calculated based on the cost of equipment; device maintenance and data connection; home delivery, support and pickup; including weekly clinician followup and monitoring. These costs were compared against clinical based outpatient therapy at the VA medical center defined as three onehour sessions per week for 90 days. The estimated monthly equipment cost of $82.14 and $54.76 for the foot and hand robotic devices, respectively, was amortized over an expected five-year lifetime (Table 5). This was combined with the cost of an annual maintenance and hosting contract of $1,199 per device to cover device repairs and the data connection and online portal for therapist monitoring of patient progress. Corresponding therapist transit costs were calculated based on a study average round-trip distance of 164 (48.68) miles using the standard government mileage reimbursement rate of $0.415 per mile (43) and 144.8 (39.7) minutes. The therapist costs of $65.23 an hour consisted of the cost of time to deploy a two-person team to the participant homes for device installation and orientation, repairs or device reorientation due to user error, device pickup, and weekly therapist monitoring and telephone based follow-up calls.
Projected outpatient therapy transit and therapist costs for three one -hour sessions held weekly at the Atlanta VA medical center were based on the mean distance from a participating VA clinic (63.3±35.8 miles) and patient mileage reimbursements and the projected cost of a physical therapist at $38.56 taken from the State of Georgia Bureau of Labor Statistics [40].
Final cost analysis reveals three months of home-based, telerehabilitation costs the VA health care system an average of $1,268.07 per Veteran, compared to an average of $3,619.95 per Veteran for outpatient clinic based therapy. This analysis reveals an average of $2,352 (64.97%) in savings compared to clinic-based therapy per stroke survivor. Further, the inclusion of home-based telerehabilitation leads to a return of approximately $2.85 of therapy on every dollar spent by the VA health system.
Discussion
This study aimed to evaluate the efficacy of using telerehabilitation via home-based robotic-assisted therapy to improve function and reduce depression symptoms for rural stroke survivors. Additionally, this study aimed to assess the effects home-based robotic-assisted therapy has on access, utilization and cost effectiveness for rural stroke survivor’s rehabilitation when compared to clinic -based therapy. The results of this study indicate that home-based, roboticassisted telerehabilitation can elicit meaningful improvements in UE function, gait speed, walking distance and reduce depression symptoms. Participants indicated overall satisfaction with the device and their progress. It was also demonstrated that homebased, robotic-assisted telerehabilitation expanded access for those survivors no longer receiving therapy and increased access for those already receiving therapy. Finally, this study documents that homebased, robotic-assisted telerehabilitation provides a substantial cost savings when compared to clinic -based therapy.
Clinical measures
This study demonstrates that individuals along varying time points of stroke recovery, as indicated by mean time since stroke ranging from 1.8-136.7 months, made meaningful improvements in their ability to perform upper and lower extremity functional tasks by using home based, robot-assisted therapy.
Statistically significant improvements in UE motor functioning were observed at end of treatment, as evidenced by mean improvement in affected UE ARAT score of 9.2 points. Additionally, this change surpasses the previously validated MCID [39], indicating this change would be detectable and meaningful to the stroke survivors. Although, substantial heterogeneity in mean time since stroke presents a confounding factor for drawing conclusions, a previous study by v ander Lee, et al. [39] documented considerable responsiveness to improvement in UE functioning across time, strengthening the observed 9.2 point improvement observed in this study. It also must be noted that normative ARAT scores for stroke survivors six month and 3.6 years post stroke are 41.3 and 29.2 out of 57 respectively, which demonstrates a negative sloping trend as time since stroke increases. Taken together, the observed baseline scores in this study demonstrate that the stroke survivors began this study with less than average UE functional abilities.
Small but clinically meaningful improvements in gait speed were observed from baseline (0.31 m/s) to post-intervention (0.40 m/s) following use of the home-based, robotic-assisted telerehabilitation intervention. Gait speed has been shown to be a significant predictor of disability following stroke and as survivors recover gait speed they experience substantially better function and quality of life [44]. Previous studies have defined categorizations for different gait speeds [45] and have validated those categorizations ability to delineate functional abilities [46]. People walking at speeds of <0.4 m/s are household ambulators, people walking at speeds of ≥0.4 m/s but <0.8 m/s are limited community ambulators, and people walking at speeds of ≥0.8 m/s are able to walk in the community without substantial limitations. Schmid, et al. demonstrated that categorization transitions represent clinically meaningful changes for stroke survivors [44], a finding that is especially poignant for household ambulators. They theorized that household ambulators possibly had more severe strokes and were more sensitive to the adverse sequelae that exist when mobility is restricted. Although the current study observed a relatively small improvement in gait speed (0.09 m/s), this change represents a transition into a higher functional gait speed category that indicates home-based, robotic-assisted telerehabilitation may improve functional independence and may decrease the adverse sequelae seen when stroke survivors are home bound.
Although the stroke survivors in this study demonstrated only modest improvement in functional walking capacity at postintervention (30.2%), this finding must be taken into the context of the participant’s abilities. Initially, the cohort was largely restricted to household ambulation, suggesting their LE functional abilities were below average. This concept is further strengthened when this cohort’s initially walking capacity (59.7±51.0 meters) is compared to previously documented normative data for stroke survivor’s walking capacity (408±132 meters) [47]. Therefore it can be suggested that although the small absolute change observed (18.03 m) did not reach the previously described MCID of 34.4 meters [48], the relative change of 30.2% might actually reflect an improvement in the participants walking capacity. This idea was previous suggested by Flansbjer, et al. who noted that a change as small as 13% may indicate clinically significant improvements [49].
A systematic review by Hackett, et al. [50] found that approximately one-third of all stroke survivors suffer from depression. Early stroke rehabilitation has been shown to reduce the risk for depression by approximately 43%, with Hou, et al. citing increased endorphins levels, improved fitness, and social interaction as possible mechanisms [51]. This study documents only modest improvements (28.44%) in CES-D from baseline (12.4±14.2) to post-intervention (8.9±9.3), indicating improvements were less than previously reported (43%) when compared to similar interventions. However, this data represents mean scores across all subjects. When considering only those likely to be diagnosed with depression, we found that five survivors (20%) out of 20 scored 16 or higher on the CES-D, indicating depressive symptoms were likely present [52]. At post intervention, exploratory analysis reveals that three of the five survivors CES-D scores decreased to below the threshold for depression. This change represents a 60% improvement in clinically significant depression scores that exceeds the previously observed improvements.
Usage and utilization
Evidence has been accumulating that post-acute stroke rehabilitation services are not equally delivered among populations [53]. These disparities in health care utilization may have a deleterious effect on functional outcomes following discharge from the acute and subacute rehabilitation setting [54], thus making investigations into methods to improve usage and utilization imperative to optimize outcomes. Therapy utilization in acute rehabilitation is well documented; however, usage and utilization is less understood in the subacute and chronic phases of stroke recovery. One study examining usage and utilization in 88 post-acute rehabilitation providers found that stroke survivors utilizing home healthcare agencies delivered an average of 21 therapy visits during a 90-day episode of care or in other terms, 23.3% utilization, where utilization is the number of therapy sessions during the episode of care [55]. Additionally, the authors found that stroke survivors utilizing clinic-based outpatient therapy received an average of 40 therapy visits per 90-day episode of care (44% utilization) [55]. When considering the significance of this difference in utilization the authors adjusted for baseline differences and found that survivors utilizing outpatient care (i.e. higher utilization) had better outcomes at 90 days. This hypothesis is supported by previous literature that has observed positive dose -response relationships with increasing amounts of therapy [56-57]. Although we documented large heterogeneity in utilization, ranging from 2.75% to 76.53%, mean utilization was 30.6 therapy sessions across the mean 106-day episode of care (29.3%). Although this rate of utilization indicates that the home-based, robotic-assisted telerehabilitation has a lower rate than the previously discussed clinic-based therapy delivery method it was higher than the documented home healthcare utilization. It must also be noted that the stroke survivors in this study were encouraged to continue previously initiated therapy services. When these data are considered in exploratory analysis, the home-based, robotic assisted telerehabilitation delivery method is found to increase the therapy utilization from 9.62% to 38.9%.
A recent meta-analysis exploring dosing strategies following stroke has documented 30-60 minutes, five to seven days per week [58] as optimal. However, this meta-analysis largely excluded literature exploring the effects of constraint-induced-movementtherapy (CIMT). Although CIMT is a developing field, preliminary evidence shows promising trends for improvements in functional capacity with longer daily treatment dose [59]. Therefore, the daily usage documented in this study (90.6min) is more likely to provide similar stimuli for adaptive neuroplasticity responses that are thought to be observed in CIMT and other studies [57].
Satisfaction
Participants in this study largely expressed satisfaction with the Hand/Foot Mentor devices and with their progress throughout the episode of care. Participants articulated positive statements, such as, “It kept me walking,” “It exercised my hand, because most of the time my hand would just be sitting in my lap.” “I liked beating the games.” Several stroke survivors commented on how much they “had to focus while playing the games,” which is a central pillar of a successful neurorehabilitation program [60].
Comments of dissatisfaction generally focused on limited game selection. Three stroke survivors expressed difficulty donning and doffing the HM device citing the need for caregiver assistance, while one stroke survivor cited difficulty donning the FM device due to long -standing knee osteoarthritis. This participant required caregiver help to bend the involved knee. These difficulties did not prevent any stroke survivor from continuing to use the device.
Access
Within stroke care there has been an implicit assumption that after the acute and subacute rehabilitation phases, where an individual may receive dedicated services for two to six months that the individual has reached a plateau. At this point there is thought to be a residual level of impairment where restorative strategies are abandoned in favor compensatory strategies. However, there is a growing body of evidence that is challenging this entrenched assumption. Frequently, significant recovery is often possible well after six months with intensive therapy [61]. Therefore, a shift in resource allocation toward continued rehabilitation delivery after the previously believed plateau period may provide benefits to functional outcomes and long term healthcare expenditures by reducing dependence on long-term medical services. Several factors influence this access to rehabilitation following a stroke including distance to a participating clinic and caregiver availability. From its inclusion into stroke rehabilitation, the benefits for robotic-assisted therapy have primarily been observed while under direct supervision of a therapist [11]. However, studies have recently demonstrated that home-based, robotic-assisted therapy to provide equivalent functional improvements with minimal therapist supervision [15,18,22]. Therefore the home-based model for robotic-assisted therapy has the potential to improve access to beneficial rehabilitation by reducing caregiver burden, travel time, and cost of travel. Although the majority of our cohort (13 out of 20 or 65%) was receiving some form of therapy, several sought out private pay services outside the VA healthcare system. This detail underlines restrictions frequently seen in the VA healthcare system where regional clinics services are limited, and patients may experience backlogs in scheduling appointments. For this subset of the cohort, the HM and FM devices allowed patients to perform rehabilitation activities at the home, augmenting scheduling and outpatient restrictions. This resulted in an increased access to rehabilitation from 22.6 sessions to 53.2 sessions during the course of this study. For patients receiving these parallel services, the inclusion of homebased, robotic-assisted therapy increased rehabilitation exposure by 135.4%. The remaining members of the cohort (7 out of 20 or 35%) were not receiving formal services because benefits were exhausted. For these stroke survivors, the inclusion of the home-based, roboticassisted therapy increase access to rehabilitation from 0 visits to 30.6 over the course of the study.
Additionally, our results underscore the prospect for home -based, robotic-assisted therapy to improve access for rural stroke survivors by reducing dropout rates for rehabilitation services. In this study dropout rates were approximately 5% (1 out of 20 stroke survivors) a Figure 2 lower than the previously documented rates of 10-20% in similar studies [22,62].
Figure 2: Three-month cost of home-based, robotic telerehabilitation compared to clinic based outpatient therapy, based on three, one-hour weekly physical therapy sessions in the outpatient clinic.
Cost analysis
Previous robotic-assisted therapy studies have evaluated the cost of daily therapy, which report costs of $120 per session [11]. This is largely due to the fact that most require therapist oversight of this daily cost, operating the robotic device accounts for approximately $20. By comparison, the HM and FM systems do not require a therapist to be present. Additionally, when comparing just robotic device costs, the HM and FM devices reduce the average daily cost from $20 to $2.25, with the HM and FM individual costs at $1.83 and $2.74 respectively.
In the VA healthcare system, the monthly cost of deploying a hand or foot device at $409 and $436.38, respectively, are considerably lower than the projected cost of comparable outpatient therapy at $1,206.65 (Table 5). This represents an average cost savings of 64.97%. The largest savings were in the elimination of repeated in-person therapist costs at $521.13 per month versus a one-time installation, pickup and weekly monitoring cost at $85.37 per month, a reduction of 83.62% (Table 5). Because patients were no longer required to drive an average of 1496.72 miles (representing $621.14 per month in mileage reimbursements at $0.415/mile), transit costs dropped to $180.71 per month or a savings of 75.35%.
The results of this study demonstrate a substantial (64.97%) savings for the VA healthcare system. Although, as the Department of Veterans Affairs prepares to provide rehabilitation services to the estimated 15,000 annual stroke survivors [63,64], home-based, robotic-assisted therapy’s wider applicability to provide efficacious, cost saving rehabilitation to the civilian and commercial healthcare system is not well understood. To begin to evaluate the effects of home-based, robotic-assisted therapy on consumer third party payers, one must consider the current constraints on rehabilitation. Since the Balanced Budget Act of 1997, Congress has placed an annual therapy cap on rehabilitation services that is currently set at $1,960. Additionally, third-party payers typically limit reimbursement to 20 therapy sessions, after which benefits may be exhausted. This general outline approximates per session costs at $98. One large-scale study that directly evaluated the costs and utilization of therapy services for 88 post-acute rehabilitation providers found that stroke survivors utilizing home health therapy received an average of 21 sessions at an average cost of $3,879 or $184.71 per session [55]. While those utilizing clinic-based, outpatient therapy received an average of 40 therapy visits at a cost to third-party payers of $1,689 or $42.23 per session [55]. The results of this study revealed an average cost of $1,268.07 over 30.6 sessions or approximately $41.44 per session. Related to the previously discussed costs evaluations, the results of this study demonstrate lower costs compared to clinic-based, outpatient therapy when delivered in the non-government healthcare system. When explored further, it must be noted that the absolute costs of the home - based, robotic-assisted therapy is $420.93 lower for a 16 day longer episode of care. Therefore, if overall utilization of the HM and FM devices can be raised to match the 44.4% (40 sessions during a 90-day episode of care) described by Kramer, et al. [55], this study would have documented an average of 47.1 sessions during the 106-day exposure resulting in a $26.92 per session cost. Further exploratory analysis reveals that this would represent a 36.3% cost savings compared to the current most economically efficient estimate of rehabilitation delivery.
Limitations
While we made considerable effort to design a sound study there are several limitations. First, although the target treatment time was initially set at two hours of daily therapy over course the three - month study and weekly therapist involvement attempted to alleviate this challenge, large heterogeneity in participation still persisted. Second, recruitment was focused around the Veteran populations in the southeast United States resulting in our participants consisting of a much greater number of men than women. Therefore, the external validity of our results is limited. Thirdly, a single group study design was utilizes, instead of two-group randomized controlled trial. Studies without a placebo or randomized comparison group may leave our results open to many possible interpretations and explanations. Lastly, while the results potentially represent a 64.97% and 36.3% reduction of outpatient therapy costs (for VA healthcare and civilian insurance respectively) across both arm and foot devices for stroke survivors these are estimated costs. This study was not designed to directly compare costs and had no control group. Further, although the results of this study highlight the potential for home -based, robotic-assisted telerehabilitation to increase access to rehabilitation for stroke survivors, the authors acknowledge that this intervention is intended to augment human therapist services not replace them.
Future studies with the HM and FM should utilize larger sample sizes, and involve non -Veteran participants with heterogeneous levels of impairment may help elucidate our initial observations.
Additionally, future studies will address heterogeneous training volume by holding the training dose constant across the entire intervention. Monitoring cumulative training time and allowing the number of sessions to increase or decrease to accommodate the literature supported recommended dosing for UE rehabilitation [65], will ensure a dose match across participants and experimental groups
Conclusion
Stroke survivors made clinically and statistically significant improvements in the use of their impaired extremities using a robotic device in the home. Home-based, robotic-assisted therapy reduced costs, while expanding access to a rehabilitation modality for stroke survivors who would not otherwise have received care.
Acknowledgment
Funding/Support
This material was based on work supported by the U.S. Department of Veterans Affairs, Office of Rural Health under Award Number N31-FY13Q1-00-P00621. The content is solely the responsibility of the authors and does not necessarily represent the official view of the U.S. Department of Veterans Affairs. We would also like to thank all participants in this study for their time and valuable feedback.
References
- Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, et al. Heart disease and stroke statistics--2015 update: a report from the American Heart Association. Circulation. 2015; 131: e29-e322.
- Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Borden WB, et al. Heart disease and stroke statistics--2013 update: a report from the American Heart Association. Circulation. 2013; 127: 143-152.
- CDC N. Underlying Cause of Death 1999-2013 on CDC WONDER Online Database, released.
- Mazurek JM, Schleiff PL, Wood JM, Hendricks SA, Weston A. Centers for Disease Control and Prevention (CDC). Notes from the Field: Update: Silicosis Mortality - United States, 1999-2013. MMWR Morb Mortal Wkly Rep. 2015; 64: 653-654.
- Lloyd-Jones D, Adams R, Carnethon M, De Simone G, Ferguson TB, Flegal K, et al. Heart disease and stroke statistics--2009 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation. 2009; 119: 480-486.
- Jørgensen HS, Nakayama H, Raaschou HO, Olsen TS. Recovery of walking function in stroke patients: the Copenhagen Stroke Study. Arch Phys Med Rehabil. 1995; 76: 27-32.
- Godwin KM, Wasserman J, Ostwald SK. Cost associated with stroke: outpatient rehabilitative services and medication. Top Stroke Rehabil. 2011; 1: 676-684.
- Duncan PW, Zorowitz R, Bates B, Choi JY, Glasberg JJ, Graham GD, et al. Management of adult stroke rehabilitation care a clinical practice guideline. Stroke. 2005; 36: e100-e143.
- Jia H, Cowper DC, Tang Y, Litt E, Wilson L. Postacute Stroke Rehabilitation Utilization: Are There Differences Between Rural-Urban Patients and Taxonomies? The Journal of Rural Health. 2012; 28: 242-247.
- Ovbiagele B, Goldstein LB, Higashida RT, Howard VJ, Johnston SC, Khavjou OA, et al. Forecasting the future of stroke in the united states a policy statement from the american heart association and american stroke association. Stroke. 2013; 44: 2361-2375.
- Pearson TA, Lewis C. Rural epidemiology: insights from a rural population laboratory. Am J Epidemiol. 1998; 148: 949-957.
- Lo AC, Guarino PD, Richards LG, Haselkorn JK, Wittenberg GF, Federman DG, et al. Robot-assisted therapy for long-term upper-limb impairment after stroke. N Engl J Med. 2010; 362: 1772-1783.
- Waldner A, Tomelleri C, Hesse S. Transfer of scientific concepts to clinical practice: recent robot-assisted training studies. Funct Neurol. 2009; 24: 173-177.
- Volpe BT, Krebs H, Hogan N, Edelsteinn L, Diels CM, Aisen ML. Robot training enhanced motor outcome in patients with stroke maintained over 3 years. Neurology. 1999; 53: 1874-1876.
- Neurology Section Platform Presentations 2015 Combined Sections Meeting. Journal of Neurologic Physical Therapy. 2015; 39: 68-69.
- Butler A, Bay C, Wu D, Richards K, Buchanan S, Yepes M. Expanding tele-rehabilitation of stroke through in-home robot-assisted therapy. Int J Phys Med Rehabil. 2014; 2: 2.
- Kutner NG, Zhang R, Butler AJ, Wolf SL, Alberts JL. Quality-of-life change associated with robotic-assisted therapy to improve hand motor function in patients with subacute stroke: a randomized clinical trial. Physical therapy. 2010; 90(4): 493-504.
- Kwakkel G, Kollen BJ, van der Grond J, Prevo AJ. Probability of regaining dexterity in the flaccid upper limb: impact of severity of paresis and time since onset in acute stroke. Stroke. 2003; 34: 2181-2186.
- Cramer SC, Nudo R, editors. Brain repair after stroke: Cambridge University Press; 2010.
- Winstein CJ, Stein J, Arena R, Bates B, Cherney LR, Cramer SC, et al. Guidelines for Adult Stroke Rehabilitation and Recovery A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke. 2016; 47: e98-e169.
- Russell TG. Physical rehabilitation using telemedicine. J Telemed Telecare. 2007; 13: 217-220.
- Johansson T, Wild C. Telerehabilitation in stroke care--a systematic review. J Telemed Telecare. 2011; 17: 1-6.
- Wolf SL, Sahu K, Bay RC, Buchanan S, Reiss A, Linder S, et al. The HAAPI (Home Arm Assistance Progression Initiative) Trial A Novel Robotics Delivery Approach in Stroke Rehabilitation. Neurorehabilitation and neural repair. 2015; 29: 958-968.
- Oczkowski WJ, Barreca S. The functional independence measure: its use to identify rehabilitation needs in stroke survivors. Arch Phys Med Rehabil. 1993; 74: 1291-1294.
- Linder SM, Reiss A, Buchanan S, Sahu K, Rosenfeldt AB, Clark C, et al. Incorporating robotic - assisted telerehabilitation in a home program to improve arm function following stroke: a case study. J Neurol Phys Ther. 2013; 37: 125-132.
- Langhorne P, Bernhardt J, Kwakkel G. Stroke rehabilitation. Lancet. 2011; 377: 1693-1702.
- Chang WH, Kim YH2. Robot-assisted Therapy in Stroke Rehabilitation. J Stroke. 2013; 15: 174-181.
- Lyle RC. A performance test for assessment of upper limb function in physical rehabilitation treatment and research. International Journal of Rehabilitation Research. 1981; 4: 483-492.
- Yozbatiran N, Der-Yeghiaian L, Cramer SC. A standardized approach to performing the action research arm test. Neurorehabil Neural Repair. 2008; 22: 78-90.
- Hsieh CL, Hsueh IP, Chiang FM, Lin PH. Inter-rater reliability and validity of the action research arm test in stroke patients. Age Ageing. 1998; 27: 107-113.
- Lang CE, Edwards DF, Birkenmeier RL, Dromerick AW. Estimating minimal clinically important differences of upper-extremity measures early after stroke. Arch Phys Med Rehabil. 2008; 89: 1693-1700.
- Collen FM, Wade DT, Bradshaw CM. Mobility after stroke: reliability of measures of impairment and disability. Int Disabil Stud. 1990; 12: 6-9.
- Flansbjer UB, Holmbäck AM, Downham D, Patten C, Lexell J. Reliability of gait performance tests in men and women with hemiparesis after stroke. J Rehabil Med. 2005; 37: 75-82.
- Perera S, Mody SH, Woodman RC, Studenski SA. Meaningful change and responsiveness in common physical performance measures in older adults. J Am Geriatr Soc. 2006; 54: 743-749.
- Tang A, Eng JJ, Rand D. Relationship between perceived and measured changes in walking after stroke. J Neurol Phys Ther. 2012; 36: 115-121.
- Kidd D, Stewart G, Baldry J, Johnson J, Rossiter D, Petruckevitch A, Thompson AJ. The Functional Independence Measure: a comparative validity and reliability study. Disabil Rehabil. 1995; 17: 10-14.
- Beninato M, Gill-Body KM, Salles S, Stark PC, Black-Schaffer RM, Stein J. Determination of the minimal clinically important difference in the FIM instrument in patients with stroke. Arch Phys Med Rehabil. 2006; 87: 32-39.
- Berg A, Palomäki H, Lehtihalmes M, Lönnqvist J, Kaste M. Poststroke depression: an 18-month follow-up. Stroke. 2003; 34: 138-143.
- Kim P, Warren S, Madill H, Hadley M. Quality of life of stroke survivors. Qual Life Res. 1999; 8: 293-301.
- Parikh RM, Eden DT, Price TR, Robinson RG. The sensitivity and specificity of the Center for Epidemiologic Studies Depression Scale in screening for post-stroke depression. Int J Psychiatry Med. 1988; 18: 169-181.
- Georgia SOEaWE.
- van der Lee JH, Beckerman H, Lankhorst GJ, Bouter LM. The responsiveness of the Action Research Arm test and the Fugl-Meyer Assessment scale in chronic stroke patients. J Rehabil Med. 2001; 33: 110-113.
- Bowden MG, Balasubramanian CK, Behrman AL, Kautz SA. Validation of a speed-based classification system using quantitative measures of walking performance poststroke. Neurorehabilitation and neural repair. 2008; 22: 672-675.
- https://www.va.gov/HEALTHBENEFITS/vtp/beneficiary_travel.asp
- Schmid A, Duncan PW, Studenski S, Lai SM, Richards L, Perera S, et al. Improvements in speed-based gait classifications are meaningful. Stroke. 2007; 38: 2096-2100.
- Perry J, Garrett M, Gronley JK, Mulroy SJ. Classification of walking handicap in the stroke population. Stroke. 1995; 26: 982-989.
- Bowden MG, Balasubramanian CK, Behrman AL, Kautz SA. Validation of a speed-based classification system using quantitative measures of walking performance poststroke. Neurorehabil Neural Repair. 2008; 22: 672-675.
- Wevers LE, Kwakkel G, van de Port IG. Is outdoor use of the six-minute walk test with a global positioning system in stroke patients' own neighbourhoods reproducible and valid? J Rehabil Med. 2011; 43: 1027-1031.
- 49. Tang A, Eng JJ, Rand D. Relationship between perceived and measured changes in walking after stroke. J Neurol Phys Ther. 2012; 36: 115-121.
- Flansbjer UB, Holmbäck AM, Downham D, Patten C, Lexell J. Reliability of gait performance tests in men and women with hemiparesis after stroke. J Rehabil Med. 2005; 37: 75-82.
- Hackett ML, Yapa C, Parag V, Anderson CS. Frequency of depression after stroke: a systematic review of observational studies. Stroke. 2005; 36: 1330-1340.
- Hou WH, Liang HW, Hsieh CL, Hou CY, Wen PC, Li CY. Effects of stroke rehabilitation on incidence of poststroke depression: a population-based cohort study. J Clin Psychiatry. 2013; 74: e859-e866.
- Shinar D, Gross CR, Price TR, Banko M, Bolduc PL, Robinson RG. Screening for depression in stroke patients: the reliability and validity of the Center for Epidemiologic Studies Depression Scale. Stroke. 1986; 17: 241-245.
- Pradon D, Roche N, Enette L, Zory R. Relationship between lower limb muscle strength and 6-minute walk test performance in stroke patients. J Rehabil Med. 2013; 45: 105-108.
- Bode RK, Heinemann AW, Semik P, Mallinson T. Patterns of therapy activities across length of stay and impairment levels: peering inside the "black box" of inpatient stroke rehabilitation. Arch Phys Med Rehabil. 2004; 85: 1901-1908.
- Kramer A, Holthaus D, Goodrish G, Epstein A. A study of stroke post-acute care costs and outcomes: final report. USDHHS. 2006.
- Kwakkel G, Wagenaar RC, Twisk JW, Lankhorst GJ, Koetsier JC. Intensity of leg and arm training after primary middle-cerebral-artery stroke: a randomised trial. Lancet. 1999; 354: 191-196.
- Lohse KR, Lang CE, Boyd LA. Is more better? Using metadata to explore dose-response relationships in stroke rehabilitation. Stroke. 2014; 45: 2053-2058.
- Wang H, Camicia M, Terdiman J, Mannava MK, Sidney S, Sandel ME. Daily treatment time and functional gains of stroke patients during inpatient rehabilitation. PM R. 2013; 5: 122-128.
- Pollock A, Baer G, Campbell P, Choo PL, Forster A, Morris J, et al. Physical Rehabilitation Approaches for the Recovery of Function and Mobility After Stroke Major Update. Stroke. 2014; 45: e202.
- McIntyre A, Viana R, Janzen S, Mehta S, Pereira S, Teasell R. Systematic review and meta - analysis of constraint-induced movement therapy in the hemiparetic upper extremity more than six months post stroke. Top Stroke Rehabil. 2012; 19: 499-513.
- Pajaro-Blázquez M, Pons JL. Research highlights in neurorehabilitation. J Neuroeng Rehabil. 2014; 11: 21.
- Taub E, Miller NE, Novack TA, Cook EW 3rd, Fleming WC, Nepomuceno CS, et al. Technique to improve chronic motor deficit after stroke. Arch Phys Med Rehabil. 1993; 74: 347-354.
- Blanton S, Morris DM, Prettyman MG, McCulloch K, Redmond S, Light KE, Wolf SL. Lessons learned in participant recruitment and retention: the EXCITE trial. Phys Ther. 2006; 86: 1520-1533.
- O'Donnell JC, Hamilton BB. Stroke rehabilitation management in the Department of Veterans Affairs: impact of patient referral source on outcomes. Arch Phys Med Rehabil. 1997; 78: 929-937.
- Management of Stroke Rehabilitation Working Group. VA/DOD Clinical practice guideline for the management of stroke rehabilitation. J Rehabil Res Dev. 2010; 47: 1-43.
- Oujamaa L, Relave I, Froger J, Mottet D, Pelissier JY. Rehabilitation of arm function after stroke. Ann Phys Rehabil Med. 2009; 52: 269-293.