Bundling implementation strategies supports outcome measure adoption in stroke rehabilitation: preliminary findings

Abstract

Background

Over 80% of people who experience a stroke present with residual impairment of the upper extremity, such as the shoulder, elbow, wrist, and hand. However, rarely do stroke rehabilitation practitioners (e.g., occupational therapists) use standardized outcome measures to objectively evaluate upper extremity function. Accordingly, the purpose of this project was to develop a bundle of implementation strategies that supports practitioners’ adoption of the Fugl-Meyer Assessment of the Upper Extremity in stroke rehabilitation practice.

Methods

We used tenets of Implementation Mapping to guide the development of our implementation strategy bundle. We partnered with one, large academic health system serving over 200 stroke patients annually through intensive rehabilitation care. Strategies were selected and developed through a multi-method process that included a review of the literature, qualitative input from our health system’s practitioners and managers, and expert consultation. We also specified the hypothesized implementation “mechanisms” our strategies intended to change. Practitioners’ adoption (yes/no) of the Fugl-Meyer Assessment was calculated by analyzing electronic health record documentation of the 6-month time frame before strategies were deployed compared to the 6-month time frame after deployment.

Results

Practitioners were exposed to the following implementation strategies to support Fugl-Meyer adoption: conduct educational meetings, prepare outcome measure champions, provide equipment, develop training materials, and adapt documentation systems. In the 6-months before deployment of our implementation strategies, practitioners implemented the Fugl-Meyer with 14.8% of stroke patients. In the six months after deployment, adoption of the Fugl-Meyer increased to 73.8% (p < .001).

Conclusions

When systematically developed in collaboration with health system partners, a bundle of implementation strategies may support outcome measure adoption in stroke rehabilitation. Improving the use of standardized outcome measures is of paramount importance in stroke rehabilitation to objectively monitor patients’ progress or decline, to demonstrate the value of rehabilitation services for enhancing patients’ recovery, and to advocate for continued reimbursement for rehabilitation care. Future opportunities lie in further specifying the mechanisms through which implementation strategies are intended to work and how those mechanisms contribute to strategy effectiveness.

Trial registration

NCT registration: NCT04888416; May 06, 2021.

Contributions to the literature

• Multi-method approaches, such as systematic literature searches, practitioner focus groups, and listening sessions, can strengthen the needs assessment phase of Implementation Mapping.

• A combination – or bundle – of tailored implementation strategies is needed to support standardized outcome measure adoption in the stroke rehabilitation field, particularly strategies that target barriers at the individual- and inner-level contexts.

• In addition to specifying the hypothesized “mechanisms” through which strategies work, opportunities lie in how these mechanisms contribute to strategy effectiveness.

Background

Stroke is the leading cause of long-term disability in the United States, and over 80% of stroke survivors are living with impairments of the upper extremity [1, 2]. Though not an exhaustive list, these impairments may manifest in the forms of neuromuscular weakness, altered sensory processing, abnormal motor planning, and poor coordination [3, 4]. As a result of these impairments, stroke survivors are often referred to rehabilitation practitioners who can provide evidence-based interventions (e.g., constraint induced movement therapy, functional electrical stimulation) designed to maximize function of the upper extremity [5].

To understand the need for, and response to, evidence-based interventions for the post-stroke upper extremity, it is critical that rehabilitation practitioners implement standardized outcome measures of sensory and motor function. In stroke rehabilitation, standardized outcome measures such as the Fugl-Meyer Assessment of the Upper Extremity [6] are valid and reliable tools that objectively capture upper extremity performance. Findings from such tools can help communicate patients’ functional changes to other disciplines, healthcare systems, or payers, and also inform decisions about intervention plan modifications [7]. Indeed, the Fugl-Meyer Assessment (henceforth referred to as the “Fugl-Meyer”) has been recommended for use by international panels of stroke experts [8].

Despite the value of standardized outcome measures, seldom have they been consistently implemented in rehabilitation care [9, 10], with rates of Fugl-Meyer Assessment use as low as 5% among practitioners in the United States [11]. Certainly, there is a pressing need to develop implementation strategies – or the methods and techniques that are used to support uptake of empirically-supported practices [12] – that can optimize outcome measure implementation by rehabilitation practitioners. Thus, the purpose of this paper is to provide a transparent description of how our team developed I-STROM (Implementation STRategies for Outcome Measurement) – a bundle of implementation strategies to support Fugl-Meyer adoption in the stroke rehabilitation setting. We describe our collaborative development approach as well as preliminary findings after bundle deployment and considerations for future rehabilitation studies.

Methods

Study context

We partnered with one large health system in the Midwest region of the United States to implement the Fugl-Meyer Assessment with rehabilitation practitioners who evaluated and treated the post-stroke upper extremity. Annually, our partner health system served approximately 200 stroke survivors through intensive rehabilitation services (e.g., inpatient rehabilitation). All study activities described below were approved by the Institutional Review Board at The Ohio State University (#2021H0162) and align with the Standards for Reporting Implementation Studies (StaRI) statement [13].

Development of I-STROM – implementation mapping

We followed the tenets of Implementation Mapping to guide the development of I-STROM. In general, Implementation Mapping is a five-step approach to selecting and developing strategies that can improve the use of evidence-based practices (e.g., outcome measures) in real-world care [14]. The five steps of Implementation Mapping are listed in Table 1 alongside their general descriptions. Below, we describe our methods for carrying out each step during I-STROM’s development.

Table 1 Description of implementation mapping steps

Step 1: Conduct a needs assessment

To conduct our needs assessment, we held one-on-one listening sessions with four rehabilitation managers at our partner health system to understand the potential sources of support they perceived their staff would need in order to consistently implement the Fugl-Meyer with their stroke patients. We also discussed opportunities to inform staff about the purposes of the I-STROM project and obtain their input on the types of strategies that should be deployed to enhance Fugl-Meyer implementation.

Step 2. Identify implementation determinants and outcomes

Prior to the launch of the present study, we conducted a scoping review that aimed to identify the determinants influencing outcome measure implementation in stroke rehabilitation. Implementation determinants – or barriers – that were identified through this scoping review process were vetted with rehabilitation practitioners in March-April 2021 via focus groups. We held three different focus groups with a total of 21 practitioners. During these focus groups, which were approximately 40-minutes in duration, practitioners identified simple and complex challenges that have hindered their efforts to implement standardized outcome measures. Lastly, to determine priority outcomes if I-STROM, we concluded each focus group by asking practitioners, “What do you want to change as a result of being part of this project?”

Focus group data were recorded and professionally transcribed in preparation for analysis. Using directed content analysis, two team members (professional doctoral students) coded barriers according to language from the Consolidated Framework for Implementation Research [15], a typology of theory-derived constructs that can either support or impede implementation of evidence-based practices. Discrepancies in coding were resolved during weekly meetings with the team lead who had expertise coding qualitative data to implementation frameworks and taxonomies [16, 17]. The team lead independently coded focus group data to the Implementation Outcomes Framework [18] to determine which outcome(s) were appropriate to measure before and after I-STROM deployment.

Step 3. Select implementation strategies

Selection of implementation strategies was conducted using a two-step process. First, we entered major implementation barriers identified from our scoping review into Waltz et al.’s barrier-to-strategy matching tool [19]. This barrier-to-strategy matching tool is an interactive spreadsheet that provides expert-endorsed strategies, from the Expert Recommendations for Implementing Change (ERIC) taxonomy [20], perceived to overcome implementation barriers. Next, during subsequent focus groups with acute care, inpatient, and outpatient practitioners, we vetted these strategies to confirm or refute their relevance for supporting outcome measure adoption. Practitioners were also given the opportunity to suggest additional implementation strategies which were categorized according to the ERIC taxonomy. Focus group data were analyzed by means of directed content analysis (see procedures used in Step 2).

Step 4: produce and deploy implementation strategies and materials

Procedures completed in Steps 1–3 allowed us to confirm implementation strategies to be included in I-STROM for deployment at our partner health system. Development of materials were led by two research team members (LAJ and LRW) between December 2021 – March 2022. Strategies were then deployed in April 2022.

Step 5. Evaluate implementation outcomes

We collected data from our partner health system’s electronic health record database to conduct a preliminary evaluation of implementation outcomes. Electronic health record data was abstracted for stroke patients who had been referred to rehabilitation between October 1, 2021 – September 30, 2022. We then calculated Fugl-Meyer adoption (no vs. yes) during the 6-month periods pre- and post- I-STROM strategy deployment. Specifically, we calculated the proportion of stroke survivors referred to rehabilitation who completed the Fugl-Meyer (as documented by practitioners) and compared pre-post proportions using the McNemar test.

Results

Step 1: Conduct a needs assessment

Listening sessions with managers indicated two primary themes. First, the rehabilitation department at our partner health system had encouraged practitioners to implement the Fugl-Meyer but had not systematically deployed health system-wide strategies to support adoption. Accordingly, our partners welcomed I-STROM to further promote outcome measure use by practitioners. Second, listening sessions revealed that the Fugl-Meyer was one of the most appropriate outcome measures to implement given that more time- and resource-intensive outcome measures would not be feasible. At the conclusion of these listening sessions, our health system partners identified two individuals to serve as early adopter “champions” who could enhance the implementation climate – or the perceived shared value – for using the Fugl-Meyer in routine practice. One champion was the lead practitioner on the stroke rehabilitation unit and could help coordinate the deployment of I-STROM activities. The other champion was perceived as a natural peer leader given their 15 + years of experience working with the stroke population and their familiarity with the health system infrastructure.

Step 2: identify implementation determinants and outcomes

Findings from our scoping review have been published elsewhere and identified major barriers to Fugl-Meyer implementation [21]. Focus group data with practitioners confirmed that the following six barriers from our scoping review were most salient to Fugl-Meyer use within their health system: lack of available resources, reduced access to information about the intervention or tool, lack of knowledge, decreased compatibility with routine workflow and resources, limited appropriateness based on patient needs (i.e., severely impaired patients in acute care), and minimal perceived relative advantage to having practitioners implement these outcome measures rather than other commonly used measurement tools (e.g., nine-hole peg test; dynamometry). Further, focus group participants expressed that their primary outcome of interest was to increase “adoption” (also known as “uptake” or “intentional use”) of the Fugl-Meyer as measured by practitioners’ documentation of Fugl-Meyer scores in stroke patients’ electronic health records.

Step 3: select implementation strategies

We entered these six implementation barriers into the barrier-to-strategy matching tool [19] which yielded a total of nine highly-recommended strategies to support implementation. During subsequent focus groups, practitioners confirmed that the following strategies were most relevant to support their implementation needs: conduct educational meetings, develop educational materials, and prepare outcome measure champions. However, practitioners also suggested two additional strategies to overcome implementation barriers: (1) modify the electronic health record systems to ease documentation of Fugl-Meyer scores and (2) provide standardized equipment to implement the Fugl-Meyer appropriately. Accordingly, the final version of I-STROM consisted of five implementation strategies, described below.

Step 4: produce and deploy implementation materials

Below we list I-STROM’s five implementation strategies and describe their production and deployment between December 2021 – April 2022. Figure 1 depicts these strategies, the barriers they intended to overcome, and the hypothesized mechanisms through which each strategy was intended to work [22].

Fig. 1

Logic model to support Fugl-Meyer Assessment adoption. Dotted lines indicate associations between barriers and strategies as well as strategies and hypothesized mechanisms. Figure informed by the Implementation Research Logic Model [22]

Conduct educational meetings

A total of 11 rehabilitation practitioners – all of whom were occupational therapists – participated in a 90-minute, in-person educational session on Fugl-Meyer administration and interpretation. The session was led by an expert (LRW) in the Fugl-Meyer who previously had trained an international consortium of practitioners on Fugl-Meyer implementation. The in-person session was then augmented with recorded videos at the request of practitioners who preferred recorded material that could be accessed at their discretion rather than participating in “booster” in-person training sessions over time.

Develop educational materials

Educational materials included printed instructions for how to administer each item of the Fugl-Meyer, electronic training manuals (accessible via a QR code) that depicted the equipment to be used during Fugl-Meyer administration and how patients should be positioned, and videos demonstrating how to implement the Fugl-Meyer with two stroke survivors – one with severe upper extremity impairments and one with moderate impairments.

Prepare outcome measure champions

The primary responsibility of the primary champion was to identify and communicate implementation challenges and successes to the research team throughout the project period. Challenges and successes could be communicated in-person during semi-annual I-STROM meetings but were most commonly communicated through email correspondence. The other champion also received one 60-minute advanced training in Fugl-Meyer administration and was instructed to provide immediate guidance, as able, to frontline practitioners who needed assistance with implementing or interpreting results of the Fugl-Meyer Assessment.

Provide physical equipment

Practitioners were provided with identical Fugl-Meyer kits that were equipped with required items (e.g., pencil; index card) as well as a laminated QR code that, when scanned with a mobile device, retrieved an electronic version of the Fugl-Meyer training manual. Lastly, kits also contained a laminated scoring card and dry erase marker which allowed practitioners to manually document patients’ scores on each Fugl-Meyer item if they were not able to electronically document scores when administering the outcome measure in real-time.

Adapt documentation systems

Four web-conferencing calls were held between the research team, outcome measure champions, and our partner health system’s information technology (IT) department to modify the electronic health record system. These 30-minute conferencing calls allowed our champions to express their Fugl-Meyer documentation needs with feedback from the IT department to determine what system changes were feasible to execute and would reduce practitioners’ documentation burden.

Step 5: evaluate implementation outcomes

Prior to the deployment of I-STROM, 88 stroke patients were referred to intensive rehabilitation services in the 6-month time frame between October 1, 2021 – March 31, 2022. Of those referred, rehabilitation practitioners adopted the Fugl-Meyer with 13 stroke patients (14.8%). In the 6-months after I-STROM had been deployed (April 1, 2022 – September 30, 2022), 103 stroke patients were referred to intensive rehabilitation services, and practitioners adopted the Fugl-Meyer with 76 stroke patients (73.8%; p < .001). See Fig. 2 for a timeline of all Implementation Mapping activities.

Fig. 2

Timeline of Implementation Mapping activities

Discussion

By working closely with our health system partners, our team applied methods from Implementation Mapping [14] and developed a bundle of implementation strategies to support Fugl-Meyer adoption. The five implementation strategies deployed in the present project were heavily informed by the needs and preferences expressed by our partner health system’s practitioners. Had we solely relied on our scoping review findings and the barrier-to-strategy matching tool recommendations, we likely would have deployed strategies that were mismatched to the needs prioritized by our partners. While we recommend that other teams review the literature for common implementation barriers and harness tools and resources to assist in strategy selection, strategies should ultimately be tailored to needs of local stakeholders [23, 24]. We postulate that the significant changes we observed in practitioners’ implementation behaviors (from 14.8% to 73.8% adoption) were attributed to the systematic tailoring approach we used throughout strategy development.

Though preliminary findings from the present study suggest that I-STROM may have enhanced practitioners’ adoption of the Fugl-Meyer over a 6-month period, it remains unclear how these strategies led to desired results. In other words, while we prospectively specified potential mechanisms through which each of our strategies promoted adoption (Fig. 1), we did not directly measure the degree to which these mechanisms changed as a result of deploying our bundle of strategies. However, we can extrapolate from other empirical work and theory to potentially explain how each of our strategies “worked” to improve implementation. The development of educational materials, for instance, has enhanced practitioners’ knowledge and skills towards using evidence-based practices, particularly when those materials are coupled with educational meetings that increase practitioners’ perceived value of an intervention or tool [25, 26]. We also prepared outcome measure champions to address the barriers of low relative advantage (i.e., the notion that a new practice is more advantageous than current practices) and a lack of knowledge and beliefs about the Fugl-Meyer’s value. Similarly, prior literature suggests that the use of champions, or those who perform “championing activities” (e.g., communicating across teams, providing guidance to peers), can help improve practitioner buy-in, foster positive attitudes, and enhance collective self-efficacy towards the use of a particular intervention or tool [27, 28]. Nonetheless, future studies with larger samples should be designed to measure the mechanisms underlying I-STROM, thereby elucidating our understanding of why implementation strategies may – or may not – be effective.

Study limitations

Although our study makes a valuable contribution to the rehabilitation and implementation science bodies of literature, it is not without limitations. For instance, given that this was a pilot project with preliminary findings, we cannot make claims about the effectiveness of our strategies that were examined through an observational study design. Moreover, strategies were developed and deployed within one health system and only occupational therapy practitioners participated in I-STROM activities. Thus, while our results may not be fully transferrable to other health systems, our process of tailoring strategies to meet health systems’ needs may be replicable.

Conclusions

Consistent with prior literature, a combination of tailored implementation strategies is likely necessary to enhance outcome measure adoption, and such tailoring should be conducted in collaboration with health system partners. Further, while it is helpful to specify hypothesized mechanisms of implementation strategies, we must continue to expand our understanding of how these strategies “work” and within which contexts.