Skip to main content
Download PDF
Download ePub

Applying RE-AIM to examine the impact of an implementation facilitation package to scale up a program for Veterans with chronic obstructive pulmonary disease

Implementation Science Communications volume 4, Article number: 143 (2023) Cite this article

Abstract

Background

US Veterans are four times more likely to be diagnosed with chronic obstructive pulmonary disease (COPD) compared to the civilian population with no care model that consistently improves Veteran outcomes when scaled. COPD Coordinated Access to Reduce Exacerbations (CARE) is a care bundle intended to improve the delivery of evidence-based practices to Veterans. To address challenges to scaling this program in the Veterans’ Health Administration (VA), the COPD CARE Academy (Academy), an implementation facilitation package comprised of five implementation strategies was designed and implemented.

Methods

This evaluation utilized a mixed-methods approach to assess the impact of the Academy’s implementation strategies on the RE-AIM framework implementation outcomes and the extent to which they were effective at increasing clinicians’ perceived capability to implement COPD CARE. A survey was administered one week after Academy participation and a semi-structured interview conducted 8 to 12 months later. Descriptive statistics were calculated for quantitative items and thematic analysis was used to analyze open-ended items.

Results

Thirty-six clinicians from 13 VA medical centers (VAMCs) participated in the Academy in 2020 and 2021 and 264 front-line clinicians completed COPD CARE training. Adoption of the Academy was indicated by high rates of Academy session attendance (90%) and high utilization of Academy resources. Clinicians reported the Academy to be acceptable and appropriate as an implementation package and clinicians from 92% of VAMCs reported long-term utilization of Academy resources. Effectiveness of the Academy was represented by clinicians’ significant increases (p < 0.05) in their capability to complete ten implementation tasks after Academy participation.

Conclusions

This evaluation found that the use of implementation facilitation paired with additional strategies enhanced the capacity of clinicians to implement COPD CARE. Future assessments are needed to explore post-academy resources that would help VAMCs to strategize localized approaches to overcome barriers.

Peer Review reports

Background

Chronic obstructive pulmonary disease (COPD) is an irreversible, progressive, and debilitating respiratory illness characterized by airway inflammation and airflow limitation [1, 2]. It is the fourth leading cause of death and disability [3] and the third leading cause of hospitalizations in the US It is estimated that COPD will become the leading global cause of death by 2033 [4, 5]. The US Veteran population is especially vulnerable to COPD as Veterans are four times more likely to be diagnosed with it, more susceptible to its complications, and have a higher COPD mortality rate compared to the civilian population [6].

Although COPD is not fully reversible, it is treatable when evidence-based approaches to its management such as medication optimization, adherence review, inhaler technique, and symptomatic assessment are used [7]. While these best practices are well-established in the literature and clinical guidelines, it remains a challenge to embed these recommendations into routine primary care delivery models [8, 9]. As a result, only one-third of US patients with COPD receive evidence-based treatment [9]. Barriers to implementing and scaling COPD evidence-based best practices include lack of informatics infrastructure, limited staffing and practitioner engagement, and high workload [9, 10].

COPD care bundles, which combine multiple evidence-based clinical interventions for COPD management into one service [11] have demonstrated positive patient outcomes [12,13,14,15,16,17,18,19,20] Yet there is a need to better understand the best approaches to promote scale-up of COPD bundles across multiple settings [15].

This evaluation explores whether a virtual implementation package, COPD Coordinated Access to Reduce Exacerbations (CARE) Academy (Academy), built capacity for implementing a COPD care bundle in the Veterans’ Health Administration (VA). The VA is the largest integrated health care system in the United States with 171 VA Medical Centers (VAMCs) with unique processes, cultures, priorities, and geographic barriers that can make scaling best practices difficult.

Initial design of the Academy began in 2018 with the development of a clinical training program that was refined and tested across two VAMCs [16]. The program was found to have a positive impact on clinician confidence and interprofessional collaboration; however, clinicians reported they needed additional guidance and resources to overcome logistical barriers to implementing COPD CARE [17]. Furthermore, a national implementation team, including experts in pharmacy and COPD management, recognized that different implementation strategies were needed to scale COPD CARE more rapidly.

To address these obstacles and promote effective service reach, the national implementation team developed a more comprehensive implementation package over a 12-month period with support from VA experts in Dissemination & Implementation (D&I) science [21]. The Academy is a 5-week virtual training program led by national COPD CARE experts who serve as external facilitators. It is comprised of five discrete implementation strategies [22] including implementation facilitation (IF) (Fig. 1), which is an interactive approach to addressing implementation challenges through forming supportive relationships [23]. The Academy provides facilitation through cohort-based learning by convening clinicians from multiple VAMCs working together to implement the COPD CARE bundle at their respective VAMCs. The Academy trains clinicians and provides them with resources to serve as internal facilitators and support implementation at their VAMCs. The external facilitators are included to promote discussion and shared problem solving. Virtual discussions, guided implementation resources, informatics support, and clinical training support were integrated within the IF model (Fig. 1).

Fig. 1
figure 1

Core components of the Academy

Full size image

Evaluation conceptual framework

We applied the RE-AIM conceptual framework [24, 25] to measure the impact of the Academy on building capacity for implementing COPD CARE. RE-AIM emphasizes translating evidence-based interventions into practice and has five domains and associated measures for examining an intervention’s reach (R), effectiveness (E), adoption (A), implementation (I), and maintenance (M). RE-AIM was selected as a guiding framework to assess the Academy impact due to its inclusion of implementation outcomes in addition to effectiveness outcomes [24].

The specific aims of this evaluation were to assess the impact of the Academy on clinicians’ capability to implement COPD CARE (effectiveness) and to assess the reach, adoption, implementation, and maintenance of the Academy as it relates to clinicians’ capability for implementation. Future evaluations are planned to examine more distal outcomes, such as the effect of the Academy on the implementation of COPD CARE as indicated by increased use of best practices and improvements in Veteran care.

Methods

Design

This quality improvement evaluation utilized a mixed-methods approach to obtain retrospective feedback from clinicians about their perceptions of the Academy and its impact on their capability to implement COPD CARE. This evaluation was determined not to meet the federal definition of research and qualified for a quality improvement exemption.

Setting and sample

The Academy was implemented in two cohorts in Fall 2020 and Spring 2021. Cohort one involved five Midwestern VAMCs and Cohort two involved eight VAMCs from the West and East coasts and the Southwest. Two approaches were used to identify the VAMCs: [1] the Academy was promoted through a national VA website known for promoting promising practices to VA leaders [26], and [2] program developers strategically engaged with Clinical Pharmacy Executives across the VA to identify VAMCs with a strong interest in the program. VAMCs that agreed to participate were instructed to complete a pre-implementation workbook, which involved identifying implementation team members, including an implementation lead, responsible for guiding the overall process at their site and a clinician lead responsible for conducting the front-line clinician training.

Data sources

Three primary data sources were used for this evaluation: [1] administrative records, [2] a survey, and [3] a semi-structured interview and are described in Table 1.

Table 1 Data source, description, and timeframe
Full size table

Measures

Descriptions and data sources for each RE-AIM domain are detailed in Table 2.

Table 2 Description and data source for RE-AIM domain measures
Full size table

Data analysis

Descriptive statistics were calculated for all quantitative survey items and thematic analysis was used to summarize the qualitative open-ended items. To assess changes in self-reported capability, the non-parametric Wilcoxon signed-rank test was used for the 10 Likert scale items. No adjustments for repeated testing were made and an alpha level of 0.05 was used. IBM SPSS Statistics (Version 28) [27] was used for the statistical analysis. For structured interview data, descriptive statistics were calculated to summarize the eight Likert scale items and frequencies were calculated for the dichotomous yes/no items. An independent evaluator with no direct affiliation to the VA conducted a thematic analysis of the open-ended items separately, and a consensus of final themes was agreed on through group discussion with the project lead. The thematic analysis was conducted using NVivo [28]. Initially, an inductive approach was taken using open coding. Survey and interview findings are presented within the RE-AIM framework to identify indicators of the RE-AIM effectiveness and implementation outcomes for each domain. We used the SQUIRE 2.0 reporting guidelines when writing this paper [29].

Results

The results are presented for each RE-AIM domain.

Reach

Thirty-six clinicians from 13 VAMCs participated in the Academy in Fall 2020 and Spring 2021. Across all 13 sites, 264 front-line clinicians completed the COPD CARE clinician training. This group included 130 pharmacists, 117 nurses, and 17 other front-line clinicians (e.g., respiratory therapists).

Effectiveness

Sixteen clinicians who served as clinician leads and implementation leads from the 13 VAMCs responded to the survey. Thirteen (81%) respondents were Clinical Pharmacist Practitioners and three (19%) reported other professions (i.e., inpatient care med-surg Tele Nurse). Clinicians reported significant increases in their capability to complete implementation efforts after participation in the Academy across ten items representing implementation tasks (p < 0.05) (Table 3).

Table 3 Changes in clinician capability to complete implementation tasks before and after Academy participation
Full size table

Adoption

Over 90% of clinicians responding to the survey reported complete or nearly complete attendance at all five of the Academy weekly discussions. Clinicians from 12 (92%) VAMCs participated in the semi-structured interview; one VAMC declined the request for an interview. Interviewed clinicians reported high utilization of Academy resources, with the workbooks being used by clinicians at all 12 (100%) VAMCs, followed by 11 (92%) VAMCs using the live virtual debrief meetings and the COPD CARE resources available through a shared network drive. Interview responses indicated that clinicians at three-fourths or more of VAMCs reported using the Academy weekly emails (83%), weekly YouTube videos (75%), and monthly post-Academy follow-up meetings (75%).

Implementation

Acceptability

Clinicians’ perceptions of the Academy content and delivery approach suggest they were satisfied with these aspects of the Academy [30]. Interview findings indicate that clinicians at all 12 (100%) sites viewed the Academy content to be complete and covering critical aspects of implementing the COPD CARE service and clinicians at 83% of VAMCs reported the approach to delivering Academy content was effective.

Clinicians reported that they valued the team-based support aspect of the Academy including the opportunities to be part of the virtual discussions and a learning collaborative. Survey results indicated that nearly all clinicians (94%) found that learning from colleagues at other VAMCs during the Academy and attending the weekly live sessions (81%), were some of the most valuable aspects of the Academy.

The interviews corroborated the survey findings. Table 4 presents representative clinician quotes. Related to the Academy clinical training support, clinicians at about 75% of VAMCs viewed the clinical training content as helpful for preparing front-line clinicians to deliver COPD CARE. This was reiterated in the interviews with a clinician sharing that the clinical training content had far-reaching beneficial effects on increasing clinician comfort and motivation to treat COPD (Table 4). Clinicians shared how valuable it was to have the opportunity to problem-solve with clinicians from other VAMCs. However, clinicians also reported the Academy lacked sufficient content in certain areas (e.g., exploring spirometry in greater depth, additional resources to describe the COPD CARE referral process, and additional informatics support) (Table 4).

Table 4 Themes and representative quotes
Full size table

Appropriateness

Clinicians at nearly 70% of VAMCs perceived the Academy to be critically important and clinicians at 75% of VAMCs reported having the necessary tools and resources to implement COPD CARE after Academy participation. Clinicians found the external IF approach to be appropriate. Participants felt supported, motivated, and encouraged by the support from the national facilitator (Table 4). However, some clinicians reported challenges to Academy participation; describing ways in which the Academy may not have been perceived as suitable or practical as an implementation package for those VAMCs. For example, one clinician found that the weekly Academy topics were not always aligned with where their VAMC was in the implementation process (Table 4).

Maintenance

Clinicians from 92% of responding VAMCs reported long-term utilization of Academy resources and clinicians from 75% of VAMCs reported participating in the post-Academy meetings with other participants. The Academy had a lasting effect on sites, which was reflected in VAMC’s integration of the Academy into the site organizational structure. Several clinicians reported that their implementation teams continued to have regular communication after the Academy (Table 4). For many, the Academy served as a lasting resource. Clinicians reported long-term use of the training materials and resources months after the Academy, suggesting its lasting effect and value (Table 4).

Discussion

Guided by the RE-AIM framework we evaluated the Academy’s impact on building capacity for implementing COPD CARE, including clinicians’ perceived capability to implement COPD CARE. The use of IF as the overarching approach paired with additional strategies seemed to demonstrate positive outcomes across all RE-AIM domains. In this evaluation, we found that the fully-virtual, cohort-based IF approach was successful at building capacity for implementing COPD CARE at a large number of VAMCs simultaneously. This approach eliminates geographic and cost barriers to participation, increasing the reach of the Academy.

Interview findings suggested clinicians were committed to Academy participation and had a high degree of resource utilization, indicating successful adoption. A majority of clinicians were satisfied with the content and delivery approach and viewed the Academy as a useful and practical approach to implementing COPD CARE, which are indicative of successful implementation. These positive RE-AIM implementation outcomes likely contributed to the effectiveness of the Academy at increasing clinicians’ perceptions of their capability to implement COPD CARE. As clinicians embraced the Academy and participated in the virtual discussions with other VAMCs, they felt supported and their perceived capability to be successful at implementing COPD CARE increased. The linkage between the Academy’s strategies and increasing perceived capability or self-efficacy is supported by Bandura’s Social Cognitive Theory [31].

Findings from this evaluation add to the literature on the development of internal facilitators to build capacity for implementing best practices [23, 32, 33]. The Academy participants reported receiving the guidance and resources they needed to successfully implement, suggesting that the external facilitators provided the necessary knowledge and guidance for skill development [32]. The Academy supported relationship building and the creation of a supportive environment, which was reflected in clinicians’ reporting that they valued the opportunity to interact and problem solve with the external facilitators as well as with other internal facilitators from other VAMCs who were implementing simultaneously. The Academy also provided training in COPD management and materials and support for clinicians to train their colleagues, which is a necessary support for internal facilitators in healthcare settings [33].

Our findings suggest that the Academy was successful in enhancing clinician’s perceived capability to complete implementation tasks, which likely contributed to minimizing barriers to implementing COPD best practices [9, 10]. To promote practitioner engagement and address limited staffing, the Academy provided training and resources for clinicians to engage with colleagues in other disciplines (e.g., nursing, respiratory therapy) and obtain their buy-in to collaborate on the implementation of COPD CARE. This is reflected in our finding that clinicians’ reported increased capability to gain support from leadership for COPD CARE and to form collaborations with other services after participation in the Academy. The development of such collaborations promotes sharing of staffing responsibilities, addressing barriers related to limited staffing. The Academy also promoted the development of an informatics infrastructure through the provision of COPD CARE referral dashboards and clinical note templates. Clinicians were guided through the process of initially installing informatics tools and were provided training materials to support them in training their clinician colleagues in the use and application of the informatics tools.

Despite the positive outcomes in our evaluation, some clinicians identified challenges to participating in the Academy and highlighted content or resources they perceived to be lacking, such as additional training and informatics support to improve patient referrals. Opportunities exist to explore this barrier and enhance the COPD CARE referral process.

There were some potential limitations to this evaluation. First, clinicians assessed their pre-Academy capabilities after Academy participation, which introduces the potential for bias. This was mitigated in part by discussing their perceptions of the Academy on their capabilities during the interview. Second, some of the interviews involved multiple clinicians from a VAMC, which may have affected their willingness to openly share their experiences with the Academy. To the extent possible, interviewers made efforts to create a safe environment to facilitate honest feedback. Third, some clinicians did not participate in the interviews and important differences in their viewpoints may not have been captured in data collection. While there was variation in interview participants across VAMCs, the evaluation team ensured that at least one clinician who was designated as an implementation lead was involved in each interview. Future evaluations will more carefully consider sampling approaches. A limitation of this evaluation is its sole focus on the front-line clinician perspective and not other stakeholders’ (e.g., leadership, clinic managers) perspectives. Furthermore, VAMCs engaged in this evaluation elected to implement COPD CARE, which may have increased clinician motivation to complete the Academy. Notably, these efforts to disseminate COPD CARE were made during a global pandemic, clinicians were not provided with additional salary support or protected time for their implementation efforts, and national facilitation of the program relied heavily on part-time VA pharmacy interns.

Conclusions

Through this evaluation, we documented the impact the Academy had on enhancements to clinician perceptions of capability to implement successfully. We also identified potential areas of improvement for the Academy as an implementation package to support the scale-up of COPD CARE. These lessons learned are important to inform future Academy improvements as it is rolled out to additional VAMCs. This evaluation adds to the growing evidence base supporting the efforts to scale COPD CARE. It builds on a previous iteration of the implementation package focused solely on clinician training and paves the way for future evaluations to further examine the Academy’s impact and future iterations of post-Academy strategies.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

COPD:

Chronic obstructive pulmonary disease

COPD CARE:

Chronic Obstructive Pulmonary Disease Coordinated Access to Reduce Exacerbations

IF:

Implementation facilitation

PFTs:

Pulmonary function tests

VA:

Veterans’ Health Administration

VAMC:

Veterans’ Health Administration Medical Center

References

  1. Smith MC, Wrobel JP. Epidemiology and clinical impact of major comorbidities in patients with COPD. Int J Chron Obstruct Pulmon Dis. 2014;9:871–88.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Centers for Disease Control and Prevention. Basics about COPD - chronic obstructive pulmonary disease (COPD). Centers for Disease Control and Prevention; 2021 https://www.cdc.gov/copd/basics-about.html. Accessed 15 September 2023

    Google Scholar 

  3. Centers for Disease Control and Prevention. Disease or Condition of the Week - COPD. Centers for Disease Control and Prevetion; 2022. https://www.cdc.gov/dotw/copd/index.html. Accessed 15 September 2023.

    Google Scholar 

  4. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418–28.

    Article  CAS  PubMed  Google Scholar 

  5. Quaderi SA, Hurst JR. The unmet global burden of COPD. Glob Health Epidemiol Genom. 2018;3:e4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. McGhan R, Radcliff T, Fish R, Sutherland ER, Welsh C, Make B. Predictors of rehospitalization and death after a severe exacerbation of COPD. Chest. 2007;132(6):1748–55.

    Article  PubMed  Google Scholar 

  7. Global Initiative for Chronic Obstructive Lung Disease, Inc. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease (2021 Report). 2020. https://goldcopd.org/wp-content/uploads/2020/11/GOLD-REPORT-2021-v1.1-25Nov20_WMV.pdf. Accessed 15 Sept 2023.

  8. Cochrane B, Foster J, Boyd R, Atlantis E. Implementation challenges in delivering team-based care ('TEAMcare') for patients with chronic obstructive pulmonary disease in a public hospital setting: a mixed methods approach. BMC Health Serv Res. 2016;16(a):347.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Sadeghi-Bazargani H, Tabrizi JS, Azami-Aghdash S. Barriers to evidence-based medicine: a systematic review. J Eval Clin Pract. 2014;20(6):793–802.

    Article  PubMed  Google Scholar 

  10. Turner AM, Lim WS, Rodrigo C, Welham SA, Calvert JM. A care-bundles approach to improving standard of care in AECOPD admissions: results of a national project. Thorax. 2015;70(10):992–4.

    Article  CAS  PubMed  Google Scholar 

  11. Resar R, Griffin FA, Haraden C, Nolan TW. Using care bundles to improve health care quality. IHI Innovation Series white paper. Cambridge: Institute for Healthcare Improvement; 2012. https://www.ihi.org/resources/white-papers/using-care-bundles-improve-health-care-quality. Accessed 16 Nov 2023.

  12. Morton K, Sanderson E, Dixon P, King A, Jenkins S, MacNeill SJ, et al. Care bundles to reduce re-admissions for patients with chronic obstructive pulmonary disease: a mixed-methods study. Southampton: NIHR Journals Library; 2019. https://www.ncbi.nlm.nih.gov/books/NBK541984/. Accessed 15 Sept 2023.

  13. Gillis D, Demmons J, Rocker G. Expanding The INSPIRED COPD Outreach Program™ to the emergency department: a feasibility assessment. Int J Chron Obstruct Pulmon Dis. 2017;12:1597–604.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Lennox L, Green S, Howe C, Musgrave H, Bell D, Elkin S. Identifying the challenges and facilitators of implementing a COPD care bundle. BMJ Open Respir Res. 2014;1(1):e000035.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Press VG, Au DH, Bourbeau J, Dransfield MT, Gershon AS, Krishnan JA, et al. Reducing Chronic Obstructive Pulmonary Disease Hospital Readmissions. An Official American Thoracic Society Workshop Report. Ann Am Thorac Soc. 2019;16(2):161–70.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Portillo EC, Gruber S, Lehmann M, Kies K, Margolis A, Kreyer K, et al. Application of the replicating effective programs framework to design a COPD training program. J Am Pharm Assoc. 2003;61(2):e129–e35.

    Article  Google Scholar 

  17. Portillo EC, Lehman MR, Hagen TL, Maurer M, Kettner JT, Bhardwaj SD, et al. Evaluation of an implementation package to deliver the COPD CARE Service. BMJ Open Quality. 2023;12(1):e002074.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Portillo EC, Wilcox A, Seckel E, Margolis A, Montgomery J, Balasubramanian P, et al. Reducing COPD readmission rates: Using a COPD care service during care transitions. Fed Pract. 2018;35(11):30–6.

    PubMed  PubMed Central  Google Scholar 

  19. Kim J, Lin A, Absher R, Makhlouf T, Wells C. Comprehensive and collaborative pharmacist transitions of care service for underserved patients with chronic obstructive pulmonary disease. Chronic Obstr Pulm Dis. 2021;8(1):152–61. https://doi.org/10.15326/jcopdf.2019.0175.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Portillo EC, Lehman MR, Hagen TL, Costner MG, Kettner JT, Bhardwaj SD, et al. Integration of the patient-centered medical home to deliver a care bundle for chronic obstructive pulmonary disease management. JAPhA. 2023;63:212–9.

    Google Scholar 

  21. Holtrop JS, Rabin BA, Glasgow RE. Dissemination and Implementation Science in Primary Care Research and Practice: Contributions and Opportunities. J Am Board Fam Med. 2018;31(3):466–78.

    Article  PubMed  Google Scholar 

  22. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Ritchie MJ, Dollar KM, Miller CJ, Smith JL, Oliver KA, Kim B, et al. Using Implementation Facilitation to Improve Healthcare (Version 3). Veterans Health Administration, Behavioral Health Quality Enhancement Research Initiative (QUERI). 2020; https://www.queri.research.va.gov/tools/Facilitation-Manual.pdf. Accessed 11 Apr 2023.

  24. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review. Front Public Health. 2019;7:64. https://doi.org/10.3389/fpubh.2019.00064.

    Article  PubMed  PubMed Central  Google Scholar 

  26. U.S. Department of Veterans Affairs. VA Diffusion Marketplace. https://marketplace.va.gov/.  Accessed 7 Sept 2023.

  27. Corp IBM. IBM SPSS Statistics for Windows (Version 27.0). Armonk NY: IBM Corp.; 2020.

    Google Scholar 

  28. Lumivero. NVivo (Version 12); QSR International Pty. Ltd. 2018.

  29. Ogrinc G, Armstrong GE, Dolansky MA, Singh MK, Davies L. SQUIRE-EDU (Standards for QUality Improvement Reporting Excellence in Education): Publication Guidelines for Educational Improvement. Acad Med. 2019;94(10):1461–70.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

    Article  PubMed  Google Scholar 

  31. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191–215.

    Article  CAS  PubMed  Google Scholar 

  32. Ritchie MJ, Parker LE, Kirchner JE. From novice to expert: a qualitative study of implementation facilitation skills. Implement. Sci Commun. 2020;1(25). https://doi.org/10.1186/s43058-020-00006-8.

  33. Eldh AC, Halleberg-Nyman M, Joelsson-Alm E, Wallin L. Facilitating facilitators for facilitate – Some general comments on a strategy for knowledge implementation in health services. Front Health Serv. 2023;3 https://doi.org/10.3389/frhs.2023.1112936.

Download references

Acknowledgements

Not applicable.

Funding

This study was supported by the US Department of Veterans Affairs Office of Rural Health and the Clinical Pharmacy Practice Office of the Pharmacy Benefits Management Service, through the Office of Rural Health’s Enterprise-Wide Initiative, PROG-0000104, and the University of Wisconsin Institute for Clinical and Translational Research, which is supported by the Clinical and Translational Science Award (CTSA) program, the National Center for Advancing Translational Sciences (NCATS), grant UL1TR002373 - KL2TR002374.

Author information

Authors and Affiliations

  1. School of Pharmacy, University of Wisconsin – Madison, 77 Highland Avenue, Madison, WI, 53705, USA

    Edward C. Portillo, Martha A. Maurer, Jordyn T. Kettner, Sonia D. Bhardwaj, Ziting Zhang, Cassie Sedgwick, Aaron M. Gilson, Jamie A. Stone & Michelle A. Chui

  2. William S. Middleton Veterans Affairs Hospital, 2500 Overlook Terrace, Madison, WI, 53705, USA

    Edward C. Portillo, Jordyn T. Kettner, Sonia D. Bhardwaj, Ziting Zhang & Cassie Sedgwick

  3. Institute for Clinical and Translational Research and School of Nursing, University of Wisconsin – Madison, 4240 Health Sciences Learning Center, 750 Highland Avenue, Madison, WI, 53705, USA

    Nora Jacobson

  4. Zilber School of Public Health, University of Wisconsin – Milwaukee, 1240 N 10th St, Milwaukee, WI, 53205, USA

    Rose Hennessy-Garza

  5. Kansas City Veterans Affairs Medical Center, 4801 Linwood Blvd, Kansas City, MO, 64128, USA

    Sarah Will

  6. Department of Veterans Affairs Pharmacy Benefits Management, Clinical Pharmacy Practice Office, 810 Vermont Avenue NW, Washington, DC, 20571, USA

    Sarah Will, M. Shawn McFarland & Heather Ourth

Authors
  1. Edward C. Portillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martha A. Maurer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jordyn T. Kettner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sonia D. Bhardwaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ziting Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cassie Sedgwick
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Aaron M. Gilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jamie A. Stone
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Nora Jacobson
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Rose Hennessy-Garza
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Sarah Will
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. M. Shawn McFarland
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Heather Ourth
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Michelle A. Chui
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ECP supervised the project, data collection, and analysis, as well as drafted and revised the paper. MAM conducted the quantitative and qualitative analysis and was a major contributor in writing the manuscript. JTK, SDB, and ZZ collected quantitative and qualitative data and contributed to the writing and revisions of the paper. ECP, JTK, SDB, ZZ, SW, and MSM were involved in the implementation of the intervention. RHG provided dissemination and implementation science expertise to the evaluation and revised the paper. NJ provided expertise in qualitative analysis to the data analysis and presentation of findings and contributed to revisions of the paper. MAC, AMG, JAS, and CS provided critical feedback on the conceptualization of the analysis and presentation of the findings and contributed to revisions of the paper. SW, MSM, and HO provided guidance throughout the implementation and evaluation and contributed to revisions of the paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Edward C. Portillo.

Ethics declarations

Ethics approval and consent to participate

This evaluation was determined not to meet the federal definition of research and qualified for a quality improvement exemption. Institutional Review Board (IRB) review was not required per the University of Wisconsin-Madison IRB’s “QI/Program Evaluation Self-Certification Tool.”

Consent for publication

Not applicable.

Competing interests

ECP declares the following potential competing interest: He has completed consulting work with AstraZeneca. All other authors (MAM, JTK, SDB, ZZ, CS, AMG, JAS, NJ, RHG, SW, MSM, HO, MAC) declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Portillo, E.C., Maurer, M.A., Kettner, J.T. et al. Applying RE-AIM to examine the impact of an implementation facilitation package to scale up a program for Veterans with chronic obstructive pulmonary disease. Implement Sci Commun 4, 143 (2023). https://doi.org/10.1186/s43058-023-00520-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-023-00520-5

Keywords

Implementation Science Communications

ISSN: 2662-2211