Skip to main content
Download PDF
Download ePub

Implementation intensification to disseminate a skills-based caregiver training program: protocol for a type III effectiveness-implementation hybrid trial

Implementation Science Communications volume 4, Article number: 97 (2023) Cite this article

Abstract

Background

Family caregiver training decreases caregiver psychological burden and improves caregiver depressive symptoms and health-related quality of life. Caregivers FIRST is an evidence-based group skills training curriculum for family caregivers and was announced for national dissemination in partnership with the Veterans Health Administration (VHA) National Caregiver Support Program (CSP). Previous evaluations of Caregivers FIRST implementation highlighted that varying support was needed to successfully implement the program, ranging from minimal technical assistance to intensive assistance and support. However, we do not know the optimal level of support needed to inform cost-effective national scaling of the program. We describe a protocol for randomizing 24 non-adopting VA medical centers 1:1 to a tailored, high-touch implementation support or a standard, low-touch implementation support to test the primary hypothesis that high-touch support increases Caregivers FIRST penetration, fidelity, and adoption. Additionally, we describe the methods for evaluating the effect of Caregivers FIRST participation on Veteran outcomes using a quasi-experimental design and the methods for a business case analysis to examine cost of delivery differences among sites assigned to a low or high-touch implementation support.

Methods

We use a type III hybrid implementation-effectiveness study design enrolling VA medical centers that do not meet Caregivers FIRST adoption benchmarks following the announcement of the program as mandated within the CSP. Eligible medical centers will be randomized to receive a standard low-touch implementation support based on Replicating Effective Programs (REP) only or to an enhanced REP (high-touch) implementation support consisting of facilitation and tailored technical assistance. Implementation outcomes include penetration (primary), fidelity, and adoption at 12 months. Mixed methods will explore sites’ perceptions and experiences of the high-touch intensification strategy. Additional analyses will include a patient-level effectiveness outcome (Veteran days at home and not in an institution) and a business case analysis using staffing and labor cost data.

Discussion

This pragmatic trial will lead to the development and refinement of implementation tools to support VA in spreading and sustaining Caregivers FIRST in the most efficient means possible.

Trial registration

This study was registered on April 8, 2022, at ClinicalTrials.gov (identifier NCT05319535).

Background

Family and other unpaid caregivers are the invisible backbone of the US health care system, with approximately 53 million caregivers providing long-term care in the home, including an estimated 5.5 million military caregivers [1, 2]. Education and support to caregivers have been shown to decrease psychological burden, improve caregiver depressive symptoms, and enhance health-related quality of life for caregivers [3,4,5,6]. The 2022 National Strategy to Support Family Caregivers highlights the need to strengthen services and supports for family caregivers, where less than 10% of caregivers report that they receive the training they need to fulfill their caregiving role [7], and only 30% use supportive services [8]. One means of assisting individuals engaged in caregiving is to expand training for them [9]. The National Academies of Science, Engineering and Medicine published a landmark report recommending the spread of evidence-based trainings for caregivers as a means of expanding support [10].

The most comprehensive publicly funded caregiver support program in the country is led by the US Department of Veterans Affairs. The VA Caregiver Support Program (CSP) provides one-on-one coaching and support, group support, skills training, respite care, peer mentoring, counseling, and connection to resources for caregivers of Veterans across the continuum of care [11,12,13]. Over the past 10 years, CSP has promoted a culture of innovation by tapping input from caregivers and field staff to develop new supports including peer support groups, intimate partner violence trainings, and diagnosis-specific trainings for caregivers on dementia and post-traumatic stress disorder [11]. Caregivers FIRST is an evidence-based program embedded in the Optimizing Function and Independence Quality Enhancement Research Initiative (Function QUERI) which promotes function and independence in older Veterans and their family caregivers [14]. In 2018, CSP partnered with Function QUERI to implement Caregivers FIRST in eight VA medical centers, in part because CSP identified a gap in group caregiver trainings available and because results from a single-site randomized control trial of Caregivers FIRST improved caregiver and Veteran experiences of VA health care [15,16,17]. From 2018 to 2020, Caregivers FIRST was successfully implemented at those eight VA medical centers. Implementation support was guided by the Replicating Effective Programs (REP) framework and included five facilitated calls with each site, a two-day site visit, and technical assistance [15]. Based on the demand for caregiver training and high program satisfaction from caregivers [16], CSP announced Caregivers FIRST for national dissemination as a ‘strong practice’ program, meaning facilities could elect to begin offering in fiscal year 2021 [18] and then a mandated ‘minimum standard’ program, where all sites must begin offering in fiscal year 2022.

Few evidence-based caregiver support interventions have been scaled for widespread dissemination in the USA. One approach to addressing implementation barriers is to test the impact of implementation support strategies on the reach and integration of new practices into existing systems of care [10]. With Caregivers FIRST implementation, we observed barriers across multiple levels with the initial eight VA medical centers. For example, at the team level, Caregivers FIRST required novel practice patterns and coordination across service lines. At the provider level, successful implementation required time-intensive training and technical assistance (approximately 100–140 h per site). Furthermore, the level of support participating medical centers needed to successfully integrate Caregivers FIRST into routine practice significantly varied. Given competing clinical responsibilities for members of the implementation team and the costly nature of time-intensive approaches to enhance implementation delivered by research teams, it is essential to minimize the time and resources needed to implement. However, we do not know the optimal level of support needed to successfully offer Caregivers FIRST in diverse clinical contexts as part of routine care patterns, which is vital to inform cost-effective national scaling of the program. Supporting the adoption and implementation of Caregivers FIRST in routine care settings nationally requires the design and evaluation of promising and scalable implementation support strategies that leverage implementation facilitators and overcome identified barriers.

We describe a protocol for a type III effectiveness-implementation hybrid trial to evaluate the effectiveness of high-touch implementation support on Caregivers FIRST penetration, fidelity, and adoption implementation outcomes compared to low-touch implementation support in medical centers that did not adopt the program following the VA Health Care System announcement that Caregivers FIRST could be delivered to meet expected performance standards. The hybrid design was selected to evaluate both implementation and effectiveness outcomes and was informed by the goals of the nationwide VA Health Care System to require 142 medical centers to provide at least two caregiver group trainings in a fiscal year to meet Caregiver Support Program minimum standards. Partnered with the CSP, this project’s aims are three-fold. First, and the primary goal of this project is to evaluate the effectiveness of implementation support on implementation outcomes at 12 months in a cluster randomized trial (Aim 1). Second, in exploratory analysis, we will assess the effectiveness of Caregivers FIRST to increase Veteran patient’s time at home over 6 months, compared to similar patients whose caregivers did not participate in Caregivers FIRST (Aim 2). Third, we will conduct a business case analysis of the high- and low-intensity implementation support to identify cost-efficient strategies for facilitating the spread of Caregivers FIRST (Aim 3). Mixed methods best practices will be used to obtain qualitative data to better understand the sites’ experience with Caregivers FIRST and the implementation strategies offered. In addition to filling gaps in evidence surrounding implementation strategies needed to scale caregiver programs, this pragmatic trial will lead to the development and refinement of implementation tools to support VA in spreading and sustaining Caregivers FIRST in the most efficient means possible.

Methods

Overview

We hypothesize that high-touch implementation support will directly influence delivery teams’ capacity and skills to effectively self-organize and problem-solve and will lead to higher implementation penetration, fidelity, and adoption to Caregivers FIRST compared to foundational support (low-touch). We will use a cluster randomized trial to randomly assign medical centers 1:1 to either (i) foundational “low-touch” implementation support (active comparator) or (ii) enhanced “high-touch”  support (experimental) (Fig. 1). The Standards for Reporting Implementation Studies (StaRI) checklist is available as supplemental material.

Fig. 1
figure 1

Study schema by fiscal year quarter

Full size image

Randomization

A stratified block randomization will be used with two dichotomous stratification variables (1) site complexity (low complexity (2, 3 and no complexity) versus all others) [19] and (2) prior implementation of Caregivers FIRST (implementation in FY21 or FY22 vs. none). Clinical delivery staff will not be blinded to their randomization track nor will any Function QUERI study staff.

Caregivers FIRST intervention

Caregivers Finding Important Resources, Support, and Training (FIRST) is an evidence-based group skills training for friends or family members of Veterans and is designed to be delivered by existing clinical staff. Core and adaptable components of the Caregivers FIRST intervention were developed in prior work and refined over time using caregiver, provider, and operational partner input [15, 16, 20, 21], making its design user-centered. VA medical centers implementing Caregivers FIRST must deliver the core components which include four sessions that address caregiver coping, support seeking, hands-on, and health system navigation skills [15, 21]. Allowing for alignment with resources available at each VA medical center, adaptable components include the context or service line Caregivers FIRST is offered whether it be within the CSP or in partnership with multiple VA service lines (e.g., chaplaincy, mental health, primary care). Similarly, the approved adaptations allow flexibility with delivery pace (e.g., weekly or consolidated over fewer days) and modality (e.g., in-person, phone, video conference) [21]. Sites can decide on their own target caregiver population to recruit (e.g., based on needs in a given service line or caregivers of a specific diagnosis), but are told that Caregivers FIRST was developed for caregivers of Veterans with functional impairment and not based on a single diagnosis. Eligible caregivers who participate in Caregivers FIRST are enrolled in either the VA CSP’s Program of General Caregiver Support Services (PGCSS) or the Program of Comprehensive Assistance for Family Caregivers (PCAFC).

Site inclusion criteria

Site eligibility for this study will be VA medical centers that do not adopt Caregivers FIRST defined as sites that had no Caregiver FIRST activity or sites that implemented the program but had less than five unique caregivers trained in the first 6 months of the mandated period (October 1, 2021–March 31, 2022). The threshold of less than five caregivers trained was determined in partnership with CSP as “not adopting.” Adoption metrics will be tracked using a dashboard developed jointly with CSP to monitor site implementation, given individual sites are required to document the delivery of Caregivers FIRST and attendance. Ineligible sites will include the eight VA medical centers that had previously participated in Function QUERI, since in a previous trial, those sites were exposed to a high-touch implementation strategy [15]. Eligible sites will be identified by the dashboard and will be prioritized by VA service region, site complexity level, implementation activity (e.g., did not adopt versus implemented but with “low enrollment” defined as less than five caregivers), and CSP staffing capacity (CSP total staffing FY20). The goal will be to select from medical centers distributed across VA service regions (approximately two per region) with sites systematically selected to approach for enrollment based on complexity level and lowest implementation activity within a VA region. Additional inclusion criteria include (1) facility leadership willing to participate in the study via a signed participation agreement and (2) agreement to deliver Caregivers FIRST within 6 months.

Implementation framework

Understanding implementation as a dynamic and iterative process, we will use an overarching framework guided by the QUERI Implementation Roadmap and the Dynamic Sustainability Framework [14, 22, 23]. In this framework, implementation outcomes of new programs are influenced by the characteristics of the intervention and the interaction of factors within a site’s environmental context and processes to support program sustainment, as well as the delivery team’s capacity to self-organize for optimal problem-solving [14]. Because of variations in context and organizational characteristics, we propose that foundational (low-touch) implementation support will be sufficient for many VA medical centers to be able to successfully incorporate Caregivers FIRST into routine practice. For the enrolled sites with low adoption, we hypothesize that enhanced (high-touch) implementation support will influence readiness to change, organizational resilience and safety climate, and leadership support which will lead to higher Caregivers FIRST penetration, fidelity, and adoption (Fig. 2).

Fig. 2
figure 2

Function QUERI implementation intensification framework

Full size image

Aim 1: implementation strategies

Among the enrolled Caregivers FIRST sites, the goal is to test implementation intensification approaches, specifically foundational (low-touch) vs. enhanced (high-touch) implementation support, informed by Replicating Effective Programs (REP). REP is a framework ideal to support the delivery of interventions in new settings or to new populations [24, 25], and it has been pragmatically applied in implementation studies in the VA [15, 26, 27]. REP includes four phases: pre-conditions, pre-implementation, implementation, and maintenance/evolution, with emphasis on local customization and tailoring. We anticipate low-touch implementation support, which provides self-guided tools to be used by a variety of site settings with varying levels of resources and experience with quality improvement or innovation, will be sufficient to successfully incorporate Caregivers FIRST into routine practice.

Foundational implementation support (low-touch)

All enrolled sites will receive foundational support, a bundle of self-guided implementation strategies that include toolkits for implementation. The low-touch strategy includes four support components that were developed and tested in prior Function QUERI work [15]: toolkit; SharePoint access for clinical program training materials; data dashboard to assist sites with tracking their own data; and learning collaboratives to promote peer-to-peer sharing and implementation support (Table 1).

Table 1 Components of low-touch and high-touch implementation support
Full size table

Enhanced implementation support (high-touch)

Sites randomized to receive high-touch support will receive four facilitated phone calls over a period of approximately three months that address adoption barriers and elicit sharing of successful adoption strategies. The high-touch support will consist of facilitation, a process of interactive problem-solving and support that occurs in a context of a supportive interpersonal relationship. The study team plans to conduct four scheduled calls over three months in a group format with sites randomized to high-touch support, providing tailored facilitation around key barriers to Caregivers FIRST implementation and program sustainment (Table 1). Facilitation will be provided by Function QUERI team members, comprised of investigator and masters-level project staff who have received QUERI implementation facilitation training [28]. At the end of each call, study team facilitators will summarize action plans for the purpose of sites to have tangible steps to complete in between calls.

Aim 1: data collection and measures

Caregivers FIRST implementation will be evaluated using an explanatory sequential mixed method design where qualitative data will be collected in parallel with quantitative data to explore and understand contextual and experiential factors related to differences in site-level implementation outcomes [29]. Evaluation data sources will be collected through (1) CSP program data for implementation outcomes (Aim 1), (2) quantitative surveys (baseline, 6 months, and 12 months), (3) semi-structured interviews (sample of sites at baseline and 6 months), and (4) surveys of program adaptations. For additional context for Aim 1, other data sources include (5) electronic health record data for effectiveness outcomes/quality metrics and (6) staff and labor cost data to assess the business case delineated by program delivery and implementation strategy. Evaluation data sources, methods of data collection, source, and timepoint are summarized in Table 2. Surveys and qualitative interviews were approved as exempt research by the Institutional Review Board of the Durham VA Health Care System and received concurrence from the VA national labor unions.

Table 2 Study measures by aim
Full size table

Aim 1: implementation outcomes

Implementation outcomes informed by Proctor and colleagues [35] will be assessed through electronic health record (EHR) data pulls from the VA Corporate Data Warehouse. Prior to the announcement of Caregivers FIRST as a mandated CSP training, and in consultation with Function QUERI, CSP developed and launched a national EHR template for clinical staff to document attendance and details of Caregivers FIRST participation in individual caregiver’s VA electronic health records. Data for implementation outcomes will be extracted from health factors that are generated from the national EHR templated note. The primary outcomes are cluster-level PENETRATION defined as the (1) proportion of caregivers who received consults for VA caregiver education and training services who attended at least one class at a medical center over a 12-month period and (2) the number of classes delivered over a 12-month period. FIDELITY outcomes are cluster-level defined as (1) the proportion of the number of recommended classes delivered over a 12-month period, (2) the number of caregivers who attended at least one class over a 12-month period, and (3) the mean number of classes attended per caregiver over a 12-month period. ADOPTION is a cluster-level dichotomous outcome defined as meeting a threshold of four or more training classes delivered to a minimum of five caregivers over a cumulative 12-month period (yes) or not meeting this threshold is considered non-adoption (no). In addition, while 12 months is the primary endpoint, penetration and fidelity outcomes will also be assessed over 6- and 18-month time periods with descriptive results.

Quantitative staff surveys

Quantitative surveys will be administered using VA REDCap (Research Electronic Data Capture) [36] to all staff identified as delivery team members or relevant service line leaders by the site Point of Contact for each enrolled site. An initial invitation will be emailed, with two follow-up weekly reminders. Factors influencing implementation that were highlighted in our overarching framework (characteristics of the intervention, site’s environmental context, and delivery team’s capacity to self-organize) will be captured with baseline survey measures including organizational resilience [31], organizational readiness [32, 37], and implementation climate [30]. The 6- and 12-month survey measures include program sustainability [34], organizational resilience [31], and experience with implementation strategies. For both baseline and 12-month surveys, we will also collect information on challenging and helpful implementation factors with categories developed from previous Function QUERI implementation work.

Qualitative staff interviews

With a sample of half of the enrolled sites stratified by arm, we will conduct 30-min individual telephone interviews with Caregivers FIRST delivery staff and service line leaders (identified by each site’s Point of Contact) to elicit facilitators and barriers that affected implementation. A semi-structured interview guide will probe based on Proctor’s criteria for specifying and reporting strategies (e.g., actor, action, target, temporality, frequency) [38] as well as conditions relevant for implementing Caregivers FIRST. We will use maximum variation sampling to ensure diversity by implementation status (e.g., adopted vs. did not adopt or had “low enrollment”) and size of Caregivers FIRST delivery team. Participants will be invited via email, and we will aim for 5–10 interviews per sampled SITE. Baseline interviews will assess implementation climate, readiness, and available resources and 6-month interviews will assess experience with implementation strategies and tools, including those who received high-touch support, and any additional strategies developed during program implementation. All interviews will be audio recorded and transcribed.

Program adaptations

Points of Contact from all enrolled sites will also be asked to report Caregivers FIRST adaptations via the VA REDCap survey at baseline, 6 months, and 12 months. An initial invitation will be emailed, with two follow-up weekly reminders. Survey questions are derived from Wiltsey Stirman’s Framework for Reporting Adaptions and Modifications Expanded (FRAME), which provides a framework for standardized tracking of modifications and monitoring of Caregivers FIRST fit within each site [33]. Examples include when the modification was made, whether the adaptation was made proactively, and key decision-makers. Example modifications to probe upon are based upon prior work [21], including the use of optional components, delivery format, delivery team composition, and additions or removal of content. The survey will also ask about equipment and staff time spent in planning and delivery to assess cost.

Aim 2: effectiveness outcomes/quality metrics measures

Effectiveness outcomes will include Veteran days in the community (e.g., not in a hospital or nursing home) over a 6 month period after caregiver participation in Caregivers FIRST. We will use the inclusion of Caregivers FIRST in VA medical center performance plans as the quality metric, where sites report the status on meeting requirements within the CSP Program of General Caregiver Support Services (PGCSS). In addition, we will evaluate the number of caregivers enrolled with each site’s PGCSS and whether the quality metrics change based on the implementation approach.

Aim 3: analyses of business case/value proposition measures

The business case will include a budget impact analysis to help VA administrators consider the cost of implementation and delivery along with the effectiveness of the intervention in the context of the VA priorities to improve Veteran outcomes. The budget impact analysis will evaluate implementation costs for each strategy and intervention delivery costs per participant. The high-touch implementation support strategy is expected to use more resources (delivery time and attendance time) and potentially improve implementation outcomes, for example improved fidelity or increased penetration. To assess the value of the additional investment in implementation support, we will describe implementation costs per enrolled participant for each implementation strategy. While the overall resource use might be higher, a lower cost per participant would indicate a high value. We will use staff time and equipment data from the adaptations survey (baseline, 6 months, and 12 months) to assess costs using U.S. Veterans Health Administration salary and managerial cost accounting data. Drawing from methods in prior Function QUERI work [17], Business Case Analysis (BCA) will frame affordability to VA. For the cumulative 12-month study period, two types of costs will be collected for each site using a standardized method to track implementation activities 1) clinical delivery team costs incurred and 2) implementation strategy costs incurred. Because the 24 sites enrolled will represent a fraction of the sites nationally who will also implement (expect over 100 to implement), we can extrapolate to a national scenario in the BCA.

Analysis

Aim 1: implementation of quantitative analysis

As part of our type III effectiveness-implementation hybrid design framework, the primary research question compares differences in implementation outcomes (penetration, fidelity, and adoption) between implementation arms at 12 months. Implementation outcomes are continuous, count, and binary outcomes, and generalized linear models [39] will be used to examine the effect of high-touch support on implementation outcomes at 12 months. The main predictor of interest will be enhanced (high touch) implementation support vs. foundational (low touch) implementation support, and stratification variables for site complexity (low complexity (2, 3, and no complexity) versus all others) and prior implementation of Caregivers FIRST or not will be included; other facility level covariates for consideration are CSP program quality score and size of CSP program. In secondary analyses, implementation outcomes over a 6-month and 18-month period will be assessed and described. We will examine how implementation outcomes change over time using descriptive methods (e.g., plots, descriptive statistics, subgroups).

Descriptive statistics for survey measures such as organizational readiness for implementation change (see Table 2 for others) will be calculated overall and by randomization arm. We will use the same modeling approach described above to examine the effect of the implementation strategy on survey measures.

Aim 1: qualitative analysis and mixed methods

Interview transcripts will be coded and analyzed by at least two Function QUERI team members. Using Expert Recommendations for Implementing Change (ERIC) typology [40] and clusters [41], we will use directed content analysis including a priori labels to mark implementation strategies and activities and data-derived labels to reflect respondents’ description of their experience with barriers to implementation. A framework matrix will be developed to summarize coded data and compare implementation context, experience, and barriers across sites. Matrix rows will reflect coded implementation support types and the columns will reflect whether or not responses are from low-touch sites and implementation outcomes for each site. Summaries of coded data within each matrix cell will describe the implementation strategy.

Aim 2: effectiveness outcomes

We will assess the impact of Caregivers FIRST on Veteran days in the community (home time) over a 6-month period. Our sample of Veterans for this analysis will come from sites that adopt Caregivers FIRST nationally (that is, not limited to the 24 enrolled sites). The sample of Veterans will be from those whose caregiver received consults for VA caregiver education and training services, including to Caregivers FIRST over a year period after the program was announced as a minimum standard; some of those caregivers will attend Caregivers FIRST classes (i.e., treatment group) and some will not (i.e., comparison group). We will use inverse-probability of treatment weighted methods to adjust for confounding and estimate an average causal treatment effect for Caregivers FIRST on home time over the 6-month time period following the consult for a VA caregiver education and training. The study team will determine a priori which covariates are likely to be key determinants of attending Caregivers FIRST training. We will explore the distribution that best fits the days at home outcome: Poisson, Negative Binomial, or zero-inflated. Generalized linear models with inverse propensity score weights will be used. We will describe the quality metrics using descriptive statistics, but these will not be modeled.

Aim 3: business case analysis

The base-case BCA will use a decision tree to compare the expected value for costs (implementation and delivery) and implementation outcomes (number of participants) between the two implementation support arms. In addition, we will use one-way and probabilistic sensitivity analysis to simulate likely outcomes in the context of distributions informed by the trial data as well as prior evidence. By modeling plausible scenarios, the decision model will allow us to communicate a practical range of estimates to sites and operational partners rather than relying on statistical significance. We will work with CSP to establish thresholds.56 We will report costs by arm for the base case BCA and for different assumptions, along with what conditions would need to be met to be within VA thresholds for affordability/value.

Sample size and power

Sample size calculations were conducted for the penetration implementation outcomes at 12 months. Based on a two-sided t-test based on a sample size of 24 sites (12 receiving foundational low-touch implementation supports and 12 receiving enhanced high-touch support), a type-1 error rate of 5%, will have 80% power to detect an effect size difference of 1.2 and 90% power to detect an effect size difference of 1.4 between arms. We did not adjust alpha for co-primary outcomes as success will be assessed based on improvement in both measurements of penetration (Table 2).

Discussion

Health care systems need resource-efficient approaches to scale caregiver training interventions. This study is expected to illuminate the value of a high-touch implementation strategy to promote the penetration or integration of Caregivers FIRST within routine practice. The mixed methods design will help identify the right dose of implementation support for the efficient scaling of future caregiver training programs. Identifying the ‘right dose’ of implementation support to successfully implement is an important gap currently in the research literature because very few research-driven caregiver programs are disseminated after the research project concludes. In particular, we do not have benchmarks on expected rates of reach from most caregiver support and training studies, and as such, have little ability with which to frame their success as scaled pragmatic trials. In the first rendition of Caregivers FIRST at eight VA medical centers, we reached 29% of eligible caregivers with our training. And yet, we do not have other evidence to tell us whether that was a high or low rate of reach. The rates of penetration, fidelity, and adoption discovered in this study could contribute to developing a range of benchmarks as other teams scale their caregiver interventions in health systems.

In addition, given the VA Caregiver Support Program is in the practice of implementing new programs annually as a part of its mission, our findings could be instructive to them and to other operations decisions in health systems. For example, learning that low-touch implementation support works just as well for sites as a high-touch approach could reduce the barriers to health systems to initiate caregiver training supports. Alternatively, identifying which types of sites might benefit from a high-touch approach could help health systems target scarce resources to implement caregiver trainings. In general, this study should provide real-world evidence of the effort needed to implement caregiver trainings, which is timely as the National Caregiver Strategy to Support Family Caregivers recommended in 2022 spreading evidence-based trainings as a tenet of its recommendations.

Results from Aim 2 will help build the evidence of how pragmatic trials affect patient outcomes. Revisiting the effectiveness of Caregivers FIRST training in its current pragmatic trial form can inform the benefits to patients of caregiver skills training. Finally, assessing the business case of Caregivers FIRST in Aim 3 will inform the future scalability and sustainment of the program by informing the VA Health Care System about the costs of delivery using existing clinical teams.

Limitations

Our study is expected to have limitations. Assessment of implementation outcomes will rely on EHR data being entered appropriately in EHR templates by clinical staff. As such, we will rely on mixed methods to fully assess the primary aim of this work to assess fully the value of high-touch over low-touch implementation supports. Obtaining the perspectives from program delivery staff will add richness to our data and help us understand the types of sites who may benefit from low versus high-touch implementation supports. Missing data from sites would be an additional limitation. Regarding business case analysis, we will be using the VA perspective and considering implementation in an integrated health care system. Sites will report time spent implementing the program, which could also be a limitation if data are not completely reported. Additionally, costs of implementation will be primarily labor costs, but it is unknown how business case analysis findings would generalize to private or non-integrated health care systems.

We feel the strengths, described above, outweigh the anticipated limitations. Importantly, we use a partner-informed and user-centered design that incorporates feedback and the needs of both caregivers, clinical delivery teams implementing Caregivers FIRST, and our operational partners in the VA Caregiver Support Program. Since its inception as a single-site randomized control trial in 2014, the Caregivers FIRST curriculum has been improved collaboratively with sites, using input from multiple disciplines and assuring high-quality content and better cultural competence. Sites have also been very willing to engage in surveys and interviews with our team, leading to rich data collection and impressions of their experiences.

Availability of data and materials

Not applicable.

Abbreviations

BCA:

Business Case Analysis

Caregivers FIRST:

Caregivers Finding Important Resources, Support, and Training

CGF:

Caregivers FIRST

CSP:

Caregiver Support Program

ERIC:

Expert Recommendations for Implementing Change

EHR:

Electronic Health Record

FRAME:

Framework for Reporting Adaptions and Modifications Expanded

Function QUERI:

Function and Independence Quality Enhancement Research Initiative

PCAFC:

Program of Comprehensive Assistance for Family Caregivers

PGCSS:

Program of General Caregiver Support Services

QUERI:

Quality Enhancement Research Initiative

REP:

Replicating Effective Programs

VA:

Veterans Affairs

VHA:

Veteran Health Administration

VAMC:

VA medical center

References

  1. AARP NAfCa. Caregiving in the U.S. 2020.: National Alliance for Caregiving and AARP; 2020. Available from: https://www.caregiving.org/wp-content/uploads/2021/01/full-report-caregiving-in-the-united-states-01-21.pdf. Cited 20 June 2023.

  2. Ramchand R, Tanielian T, Fisher MP, Vaughan CA, Trail TE, Batka C, et al. Military Caregivers: Who are They? And Who Is Supporting Them? Santa Monica. CA: RAND Corporation; 2014.

    Book  Google Scholar 

  3. Berwig M, Heinrich S, Spahlholz J, Hallensleben N, Brähler E, Gertz HJ. Individualized support for informal caregivers of people with dementia - effectiveness of the German adaptation of REACH II. BMC Geriatr. 2017;17(1):286.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Barutcu CD, Mert H. Effect of support group intervention applied to the caregivers of individuals with heart failure on caregiver outcomes. Holist Nurs Pract. 2016;30(5):272–82.

    Article  PubMed  Google Scholar 

  5. Sherifali D, Ali MU, Ploeg J, Markle-Reid M, Valaitis R, Bartholomew A, et al. Impact of internet-based interventions on caregiver mental health: systematic review and meta-analysis. J Med Internet Res. 2018;20(7): e10668.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Terayama H, Sakurai H, Namioka N, Jaime R, Otakeguchi K, Fukasawa R, et al. Caregivers’ education decreases depression symptoms and burden in caregivers of patients with dementia. Psychogeriatrics. 2018;18(5):327–33.

    Article  PubMed  Google Scholar 

  7. The Recognize, Assist, Include, Support, and Engage (RAISE) Act Family Caregiving Advisory Council & The Advisory Council to Support Grandparents Raising Grandchildren 2022 National Strategy to Support Family Caregivers 2022. Available from: https://acl.gov/sites/default/files/RAISE_SGRG/NatlStrategyToSupportFamilyCaregivers.pdf. Cited 20 June 2023.

  8. Travers JL, Rosa WE, Shenoy S, Bergh M, Fabius CD. Characterizing caregiving supportive services use by caregiving relationship status. J Am Geriatr Soc. 2022;71:1566.

    Article  PubMed  Google Scholar 

  9. Burgdorf J, Roth DL, Riffin C, Wolff JL. Factors associated with receipt of training among caregivers of older adults. JAMA Intern Med. 2019;179(6):833–5.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Committee on Family Caregiving for Older Adults, Board on Health Care Services, Health, Medicine Division, National Academies of Sciences, Engineering, and Medicine. In: Schulz R, Eden J, editors. Families Caring for an Aging America. Washington, DC: National Academies Press (US) by the National Academy of Sciences; 2016.

  11. Bruening R, Sperber N, Miller K, Andrews S, Steinhauser K, Wieland GD, et al. Connecting caregivers to support: lessons learned from the VA caregiver support program. J Appl Gerontol. 2020;39(4):368–76.

    Article  PubMed  Google Scholar 

  12. Griffin JM, Malcolm C, Wright P, Hagel Campbell E, Kabat M, Bangerter AK, et al. U.S. Veteran health care utilization increases after caregivers’ use of national caregiver telephone support line. Health Soc Work. 2017;42(2):e111–9.

    Article  PubMed  Google Scholar 

  13. Sperber N, Van Houtven C, Andrews S, Miller K, Steinhauser K, Wieland GD, et al. Family caregiver use and value of support services in the VA program of comprehensive assistance for family caregivers. J Long Term Care. 2018;0(0):41–50.

  14. Hughes JM, Zullig LL, Choate AL, Decosimo KP, Wang V, Van Houtven CH, et al. Intensification of implementation strategies: Developing a model of foundational and enhanced implementation approaches to support national adoption and scale-up. Gerontologist. 2023;63(3):604–13.

    Article  PubMed  Google Scholar 

  15. Boucher NA, Zullig LL, Shepherd-Banigan M, Decosimo KP, Dadolf J, Choate A, et al. Replicating an effective VA program to train and support family caregivers: a hybrid type III effectiveness-implementation design. BMC Health Serv Res. 2021;21(1):430.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Sperber NR, Boucher N, Hughes JM, Bruening R, Zullig LL, Decosimo K, et al. Mandated caregiver training in the Veterans Health Administration: Caregiver inquiry informs national dissemination. Gerontologist. 2023;63(3):534–44.

    Article  PubMed  Google Scholar 

  17. Wang V, Allen K, Van Houtven CH, Coffman C, Sperber N, Mahanna EP, et al. Supporting teams to optimize function and independence in Veterans: a multi-study program and mixed methods protocol. Implement Sci. 2018;13(1):58.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Van Houtven CH, Drake C, Malo T, Decosimo K, Tucker M, Sullivan C, et al. Ready, Set, Go! The role of organizational readiness to predict adoption of a family caregiver training program using the Rogers’ diffusion of innovation theory. Implement Sci Commun. 2023;4(1):69.

    Article  Google Scholar 

  19. Facility Complexity Levels: Veterans Health Administration Office of Productivity, Efficiency and Staffing. Available from: http://opes.vssc.med.va.gov/FacilityComplexityLevels/Pages/default.aspx. Cited 7 Jan. 2022.

  20. Van Houtven CH, Smith VA, Lindquist JH, Chapman JG, Hendrix C, Hastings SN, et al. Family caregiver skills training to improve experiences of care: a randomized clinical trial. J Gen Intern Med. 2019;34(10):2114–22.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Shepherd-Banigan M, Kaufman BG, Decosimo K, Dadolf J, Boucher NA, Mahanna EP, et al. Adaptation and implementation of a family caregiver skills training program: from single site RCT to multisite pragmatic intervention. J Nurs Scholarsh. 2020;52(1):23–33.

    Article  PubMed  Google Scholar 

  22. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Kilbourne AM, Goodrich DE, Miake-Lye I, Braganza MZ, Bowersox NW. Quality enhancement research initiative implementation roadmap: toward sustainability of evidence-based practices in a learning health system. Med Care. 2019;57(10 Suppl 3):286–93.

    Article  Google Scholar 

  24. Neumann MS, Sogolow ED. Replicating effective programs: HIV/AIDS prevention technology transfer. AIDS Educ Prev. 2000;12(5 Suppl):35–48.

    CAS  PubMed  Google Scholar 

  25. Kilbourne AM, Neumann MS, Waxmonsky J, Bauer MS, Kim HM, Pincus HA, et al. Public-academic partnerships: evidence-based implementation: the role of sustained community-based practice and research partnerships. Psychiatr Serv. 2012;63(3):205–7.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Brunner J, Farmer MM, Bean-Mayberry B, Chanfreau-Coffinier C, Than CT, Hamilton AB, et al. Implementing clinical decision support for reducing women Veterans’ cardiovascular risk in VA: A mixed-method, longitudinal study of context, adaptation, and uptake. Front Health Serv. 2022;2: 946802.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci. 2007;2:42.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Ritchie MJ DK, Miller CJ, Smith JL, Oliver KA, Kim B, Connolly, SL, Woodward E, Ochoa-Olmos T, Day S, Lindsay JA, Kirchner JE. Using implementation facilitation to improve healthcare (Version 3). Veterans Health Administration, Behavioral Health Quality Enhancement Research Initiative (QUERI), 2020. Available from: https://www.queri.research.va.gov/tools/Facilitation-Manual.pdf. Cited 20 June 2023.

  29. Creswell JW, Clark L. Designing and conducting mixed methods research. Sage Publications; 2017.

    Google Scholar 

  30. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implement Sci. 2014;9:157.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Lee A, Vargo J, Seville E. Developng a tool to measure and compare organizations’ resilience. Nat Hazard Rev. 2013;14:29–41.

    Article  Google Scholar 

  32. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9:7.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. 2019;14(1):58.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Mancini JA, Marek LI. Sustaining community-based programs for families: conceptualization and measurement. Fam Relat. 2004;53(4):339–47.

    Article  Google Scholar 

  35. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

    Article  PubMed  Google Scholar 

  36. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

    Article  PubMed  Google Scholar 

  37. Helfrich CD, Li YF, Sharp ND, Sales AE. Organizational Readiness to Change Assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implement Sci. 2009;4:38.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci : IS. 2013;8:139.

    Article  PubMed  PubMed Central  Google Scholar 

  39. McCullagh P, Nelder John. Generalized linear models. 2nd ed. Boca Raton: Chapman and Hall/CRC; 1989.

    Book  Google Scholar 

  40. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, Proctor EK, Kirchner JE. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:109.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This work is funded by the US Department of Veterans Affairs Quality Enhancement Research Initiative (QUE 20–023), the VA Caregiver Support Program, and the Center of Innovation to Accelerate Discovery and Practice Transformation at the Durham VA Health Care System (CIN 13–410). Courtney Van Houtven is supported by the U.S. Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Research Career Scientist Program (RCS-21–137). The contents do not represent the views of the US Department of Veterans Affairs or the US Government.

Author information

Authors and Affiliations

  1. Center of Innovation to Accelerate Discovery and Practice Transformation, Durham VA Health Care System, Durham, NC, 27705, USA

    Kasey Decosimo, Connor Drake, Cynthia J. Coffman, Nina R. Sperber, Matthew Tucker, Jaime M. Hughes, Leah L. Zullig, Brystana Kaufman, Kelli D. Allen, S. Nicole Hastings & Courtney H. Van Houtven

  2. Department of Population Health Sciences, Duke University, Durham, NC, USA

    Connor Drake, Nina R. Sperber, Leah L. Zullig, Brystana Kaufman, S. Nicole Hastings & Courtney H. Van Houtven

  3. Department of Biostatistics and Bioinformatics, Duke University Medical Center, Durham, NC, USA

    Cynthia J. Coffman

  4. Department of Implementation Science, Wake Forest University School of Medicine, Winston-Salem, NC, USA

    Jaime M. Hughes

  5. Section On Gerontology and Geriatric Medicine, Division of Internal Medicine, Wake Forest University School of Medicine, Winston-Salem, NC, USA

    Jaime M. Hughes

  6. Veteran’s Health Administration Central Office, Washington, DC, USA

    Trisha Chadduck & Leah Christensen

  7. Duke-Margolis Center for Health Policy, Duke University, Durham, NC, USA

    Brystana Kaufman & Courtney H. Van Houtven

  8. Department of Medicine & Thurston Arthritis Research Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Kelli D. Allen

  9. Center for the Study of Aging and Human Development, Duke University School of Medicine, Durham, NC, USA

    S. Nicole Hastings

  10. Geriatric Research, Education, and Clinical Center, Durham VA Health Care System, Durham, NC, USA

    S. Nicole Hastings

  11. Division of Geriatrics, Department of Medicine, Duke University School of Medicine, Durham, NC, USA

    S. Nicole Hastings

Authors
  1. Kasey Decosimo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Connor Drake
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cynthia J. Coffman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nina R. Sperber
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Matthew Tucker
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jaime M. Hughes
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Leah L. Zullig
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Trisha Chadduck
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Leah Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Brystana Kaufman
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Kelli D. Allen
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. S. Nicole Hastings
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Courtney H. Van Houtven
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SNH, CHVH, and KDA designed this Function QUERI study and secured funding. LC and TC provide operational support. JMH and LLZ conceived the implementation framework design. KD, CHVH, CD, CJC, NS, BK, JMH, LLZ, and MT developed the study procedures and protocol. KD, CHVH, and CJC drafted the manuscript. All authors critically reviewed the manuscript for intellectual content, approved the final version, and have agreed to be accountable for all aspects of the work.

Corresponding author

Correspondence to Kasey Decosimo.

Ethics declarations

Ethics approval and consent to participate

This study was approved as exempt research by the Institutional Review Board of the Durham VA Health Care System (#2334).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Standards for Reporting Implementation Studies: the StaRI checklist for completion.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Decosimo, K., Drake, C., Coffman, C.J. et al. Implementation intensification to disseminate a skills-based caregiver training program: protocol for a type III effectiveness-implementation hybrid trial. Implement Sci Commun 4, 97 (2023). https://doi.org/10.1186/s43058-023-00475-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-023-00475-7

Keywords

Implementation Science Communications

ISSN: 2662-2211