Scaling-up an evidence-based intervention for osteoarthritis in real-world settings: a pragmatic evaluation using the RE-AIM framework

Background Scaling-up and sustaining effective healthcare interventions is essential for improving healthcare; however, relatively little is known about these processes. In addition to quantitative experimental designs, we need approaches that use embedded, observational studies on practice-led, naturally occurring scale-up processes. There are also tensions between having adequately rigorous systems to monitor and evaluate scale-up well that are proportionate and pragmatic in practice. The study investigated the scale-up of an evidence-based complex intervention for knee and hip osteoarthritis (ESCAPE-pain) within ‘real-world’ settings by England’s 15 Academic Health Science Networks (AHSNs). Methods A pragmatic evaluation of the scale-up of ESCAPE-pain using the RE-AIM framework to measure Reach, Effectiveness, Adoption, Implementation and Maintenance. The evaluation used routine monitoring data collected from April 2014 to December 2018 as part of a national scale-up programme. Results Between 2014 and 2018, ESCAPE-pain was adopted by over 110 clinical and non-clinical sites reaching over 9000 people with osteoarthritis. The programme showed sustained clinical effectiveness (pain, function and quality of life) and high levels of adherence (78.5% completing 75% of the programme) within a range of real-world settings. Seven hundred seventy people (physiotherapists and exercise professionals) have been trained to deliver ESCAPE-pain, and 84.1% of sites have continued to deliver the programme post-implementation. Conclusions ESCAPE-pain successfully moved from being an efficacious “research intervention” into an effective intervention within ‘real-world’ clinical and non-clinical community settings. However, scale-up has been a gradual process requiring on-going, dedicated resources over 5 years by a national network of Academic Health Science Networks (AHSNs). Whilst the collection of monitoring and evaluation data is critical in understanding implementation and scale-up, there remain significant challenges in developing systems sufficiently rigorous, proportionate and locally acceptable.


Background
Scaling-up and sustaining evidence-based health interventions is essential to achieve widespread improvements in the quality of care [1,2]. However, we have a poor understanding of how effective complex healthcare interventions transition from a trial to being implemented at scale-up and sustained in real-world settings [3][4][5][6][7][8][9][10]. Scale-up needs to be supported by effective monitoring and evaluation systems [4,11,12], but there are challenges in balancing adequately rigorous systems to monitor and evaluate scale-up with the need for proportionate and pragmatic approaches [12,13].
Worldwide, osteoarthritis (OA) is one of the most prevalent causes of pain and disability, and an estimated 6.83 million people consult for knee or hip OA in the UK [14,15]. Yet current management is largely sub-optimal, and the burden on individuals and society remains high [16][17][18]. Although rehabilitation professionals (such as physical and occupational therapists) understand the need to implement evidenced-based interventions, their ability to implement new knowledge into clinical practice remains limited [19][20][21]. ESCAPE-pain is a complex evidence-based intervention (EBI) for people with knee or hip OA that combines structured education and selfmanagement strategies with an individualised exercise regime in line with clinical guidelines [22]. Due to the evidence demonstrating ESCAPE-pain's clinical and cost effectiveness [23][24][25][26], the programme was adopted by England's 15 Academic Health Science Networks (AHSNs) as a priority for scale-up nationally.
The study aimed to evaluate the scale-up of ESCAPEpain as a complex EBI by a network of AHSNs in England. Tied to this, we discuss the role of pragmatic data collection and monitoring in efforts to scale EBIs within real-world settings.

Study design
This is a pragmatic evaluation of the scale-up of ESCAPE-pain using the RE-AIM framework [27,28]. By pragmatic, we refer to an approach that is based in practice (rather than taking a research or theoretical perspective) using routine monitoring data collected as part of an AHSNs' national programme from April 2014 to December 2018. Table 1 outlines how the RE-AIM framework has been applied within the study to measure Reach, Effectiveness, Adoption, Implementation and Maintenance.

ESCAPE-pain programme
ESCAPE-pain is an EBI integrating education and exercise for people with chronic knee and/or hip pain and OA, which promotes self-management to improve quality of life and function [23][24][25][26]. People attend in groups of 10-12 people, twice a week, over 6 weeks (12 sessions). Each session is led by a trained facilitator and comprises 20-25 min of structured education about OA and selfmanagement strategies, and 30-45 min of exercise. Details of the programme are available at http://www.escape-pain. org/. ESCAPE-pain is underpinned by a randomised controlled trial and economic evaluation [23][24][25][26].

Scaling-up ESCAPE-pain
NHS England established 15 Academic Health Science Networks (AHSNs) to help accelerate the spread and

Data collection
The AHSNs collect routine data to monitor the scale-up of ESCAPE-pain, which were used to measure outcomes for each domains of the RE-AIM framework. AHSNs receive no participant identifiable data, i.e. providers anonymise all data prior to submitting it. Reach-The number of participants attending each cohort of ESCAPE-pain and the joint affected (i.e. hip or knee OA). Demographic data are not collected. There are no local prevalence data for hip and knee OA available at the level of individual sites to be able to determine a reliable denominator. Nationally, there are an estimated 4.11 million cases of knee OA and 2.46 million cases of hip OA [29].
Effectiveness-Pre-/post-programme clinical outcomes for participants measured using the Knee/Hip Injury and Osteoarthritis Outcome Score (KOOS and HOOS) subscales of pain, activities of daily living (ADL) and quality of life (QoL) [30,31].
Adoption-The number of sites and the type of setting (e.g. clinical outpatients, non-clinical community), provider organisation (e.g. NHS, local authority/council, charity, leisure/fitness centre) and professional (e.g. physiotherapist, therapy assistant or fitness instructor) delivering ESCAPE-pain. Implementation-Self-reported compliance with the core components of ESCAPE-pain, namely (i) a 1-h session twice a week for 6 weeks (i.e. 12 sessions), (ii) each session contains exercise and structured education, (iii) the programme follows a cohort structure and (iv) the programme must be delivered by a trained facilitator. Facilitators' self-reported levels of understand of the programme and ability to implement and deliver the programme via a routine post-training questionnaire. Participant adherence measured by the number of people completing the programme. Completion was defined as participants attending 75% of sessions, to match the level of adherence within the clinical trial [23,24,26].
Maintenance-The number of sites continuing or ceasing to deliver ESCAPE-pain at < 1 year, 1-2 years and < 2 years post-implementation. It is not possible to report on maintenance at an individual level because long-term follow-up data for clinical outcomes are not collected.

Data analysis
Clinical outcome data were available for 3664 people with knee OA from 72 sites and 209 people with hip OA from 33 sites. Only participants with pre-and post-outcome data were included in the analysis. Data from all sites were analysed as a single dataset. Paired t test was used to determine the mean difference for each subscale, and effect size was calculated using Cohen's D. Data were analysed using R v3.5.1. Data are presented as mean change in KOOS or HOOS points (confidence intervals, CI), where an increase in scores indicates an improvement. All other data were analysed using descriptive statistics.

Reach
Nine thousand one hundred fifty people with hip and knee OA have participated in the ESCAPE-pain programme between April 2014 and December 2018 (Fig. 1).

Effectiveness
Pre-and post-rehabilitation data were only available from 3614 people with knee and 209 people with hip pain who completed the ESCAPE-pain programme (i.e. defined as attending ≥ 75% of sessions) ( Table 2). Participants saw improvements in pain by 7.6 (CI 7.  Table 2). The Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) function score was used as the primary outcome for the ESCAPE-pain clinical trial [24]. It is possible to calculate the WOMAC function score from the KOOS [31]. WOMAC function score showed an improvement of − 5.49 (95% CI − 7.78,    (Table 4), although a smaller dataset was available for non-clinical settings.

Implementation
Adherence data were available for 6072 participants; 78.5% (n = 4767) of participants completed 75% of sessions. Seven hundred seventy people (or facilitators) have been trained to deliver ESCAPE-pain (488 physiotherapists, 282 fitness instructors). Facilitators (n = 665) agreed (13%) or strongly agreed (87%) they understood what ESCAPE-pain was and how to implement it, 29% agreed and 71% strongly agreed they felt able to deliver ESCAPE-pain. All sites self-report compliance against the programme's core 4 components when implementing ESCAPE-pain.

Maintenance
As of December 2018, 84.1% of sites continue to deliver ESCAPE-pain post-implementation (116 out of a total of 138 sites). Table 5 shows the number of sites delivering/ceasing ESCAPE-pain after the programme was implemented by three time categories (i.e. < 1, 1-2 and > 2 years). Of all the sites that have implemented ESCAPE-pain (n = 138), it was stopped after less than 1 year by 11 (8%) sites. Of those sites delivering ESCAPE-pain at December 2018 (n = 116), it has been delivered for more than 2 years by 24 (20.7%) sites.

Discussion
The widespread implementation of healthcare innovations usually takes many years; many initiatives fail [3] or are not translated into practice [32]. We used RE-AIM to evaluate the national scale-up of ESCAPE-pain, a complex healthcare intervention for knee and hip OA, by England's 15 Academic Health Science Networks (AHSNs). Since 2014, ESCAPE-pain has been adopted in over 110 sites, reaching over 9000 people with knee or hip OA. Whilst this is promising, national prevalence data for knee (4.11 million) and hip (2.46 million) OA in England indicate a need for continuing to expand reach [29]. Although it is delivered predominately by physiotherapists, a growing number of exercise professionals are now delivering the programme. It has been adopted across an expanding range of settings beyond the original model tested in the trial (from NHS outpatient departments to non-clinical community venues), and by a diverse number of providers and funding arrangements. This shows that complex healthcare interventions of this kind can be scaled-up successfully into similar types of settings and professions and 'spread-out' into different contexts [3,5].
Monitoring demonstrates that ESCAPE-pain has been scaled-up; however, it is important to determine what (exactly) has been implemented (i.e. fidelity) and whether it is effective (i.e. delivering intended outcomes) [33,34]. Measuring intervention fidelity and quality within a national scale-up process has been pragmatic in its approach. A mandatory 1-day training course was developed as a strategy to help safeguard that ESCAPE-pain was implemented with fidelity (i.e. by building knowledge and skills to implement and deliver the programme) [35]. In addition, all sites implementing ESCAPE-pain were required to self-report compliance with the programme's core components-sites that do not report compliance are  not considered to be delivering ESCAPE-pain. Participants' adherence levels were comparable with those observed within the original trial where 55% of participants completed 10 or more sessions in the ESCAPE-pain group intervention [23,24]. Individual level long-term follow-up data are not collected as part of the AHSNs' national programme, which means that it is not possible to determine if the selfmanagement strategies and subsequent benefits delivered by ESCAPE-pain are maintained by participants. At an organisational level, the number of sites continuing to deliver the programme is high, suggesting it is largely sustained in practice settings once implemented. There is debate in the literature about what constitutes sustainability (e.g. continued delivery of intervention components, extent of integration, realisation of outcomes, duration) [8,34]. In the case of ESCAPE-pain, the majority of sites are < 1-year post-implementation; therefore, the extent of long-term sustainability is to be seen.
As interventions move from highly resourced, controlled research conditions into 'real-world' settings, there is a risk effectiveness can be reduced due to the intervention's essential core components being incorrectly implemented [34,36]. Therefore, it is important to continue to monitor the effectiveness of interventions as they are implemented in different contexts [12,13,34]. Critically for ESCAPE-pain, on-going data collection demonstrated that the programme's effectiveness has been maintained as it spread from a controlled [24], cloistered trial setting, into very different 'real-world' clinical and community settings. However, interpretation of effectiveness needs to recognise the potential impact of missing data as people with poorer outcomes may be underrepresented.
The systematic, on-going monitoring of scale-up demonstrated by AHSNs for ESCAPE-pain is uncommon [12,13,34,37], despite calls to evaluate the widespread implementation of self-management programmes for OA, like ESCAPE-pain [37]. However, data collection has been difficult as staff (both clinical and non-clinical) in sites often lack systems to routinely collect data, have little time and may be unable or reluctant to collect data. Although the AHSNs have created systems to ease the burden of collecting and analysing data, there is no way of enforcing data return. Consequently, data return is sporadic. All sites have returned data, but not all sites return data all the time (i.e. as requested quarterly), and the sites returning data vary overtime. In addition, the completeness of data returned varies. This results in limitations for reporting scale-up.
Other limitations are that implementation outcomes relied on self-reported and indirect measures (e.g. compliance with core components of ESCAPE-pain, numbers trained, facilitators' ability implement and deliver ESCAPE-pain). However, impartial observation of implementation across a large number of geographically dispersed sites was not feasible. Whilst these measures do not guarantee the programme was implemented and delivered with fidelity or quality, they provided a pragmatic approach to monitoring. A further challenge going forward is that as the number of sites expands it is essential that systems and processes underpinning monitoring (e.g. data collection, quality controls, analysis and reporting) continue to be rigorous and sustainable (i.e. feasibly resourced) [12,13].

Conclusions
An evidence-based complex intervention can be implemented at scale successfully: achieving reach and maintaining quality. Importantly, ESCAPE-pain's clinical effectiveness has been sustained as it has transitioned into a diverse range of 'real-world' settings, beyond those tested in the original trial. However, scale-up has been a gradual process requiring on-going, dedicated resources over 5 years by a national network of AHSNs.
Whilst the collection of data for monitoring and evaluation is critical in understanding implementation and scale-up, there are significant challenges in developing systems sufficiently rigorous, proportionate and locally acceptable.
Abbreviations ADL: Activities of daily living; AHSN: Academic Health Science Network; EBI: Evidence-based intervention; ESCAPE-pain: Enabling Self-management and Coping with Arthritic Pain using Exercise; HOOS: Hip Injury and Osteoarthritis Score; KOOS: Knee Injury and Osteoarthritis Score; OA: Osteoarthritis; QOL: Quality of life; RE-AIM: Reach, Effectiveness, Adoption, Implementation and Maintenance