Skip to main content
Download PDF
Download ePub

Dynamic interplay between available resources and implementation climate across phases of implementation: a qualitative study of a VA national population health tool

Implementation Science Communications volume 4, Article number: 74 (2023) Cite this article

Abstract

Background

Available resources within an organization can determine the implementation success of an intervention. However, few studies have investigated how the required resources change over the phases of implementation. Using stakeholder interviews, we examined the changes in and interactions between available resources and implementation climate in the implementation and sustainment phases of a national implementation effort for a population health tool.

Methods

We conducted a secondary analysis of the interviews with 20 anticoagulation professionals at 17 clinical sites in the Veterans Health Administration health system about their experiences with a population health dashboard for anticoagulant management. Interview transcripts were coded using constructs from the Consolidated Framework for Implementation Research (CFIR) and according to the phase of implementation (pre-implementation, implementation, and sustainment) as defined by the VA Quality Enhancement Research Initiative (QUERI) Roadmap. We analyzed the factors that may determine successful implementation by examining the co-occurrence patterns between available resources and implementation climate across different implementation phases. To illustrate the variations in these determinants across phases, we aggregated and scored coded statements using a previously published CFIR scoring system (− 2 to + 2). Key relationships between available resources and implementation climate were identified and summarized using thematic analysis.

Results

The resources necessary to support the successful implementation of an intervention are not static; both the quantity and types of resources shift based on the phases of the intervention. Furthermore, increased resource availability does not guarantee the sustainment of intervention success. Users need different types of support beyond the technical aspects of an intervention, and this support varies over time. Specifically, available resources in the form of technological support and social/emotional support help users establish trust in a new technological-based intervention during the implementation phase. Resources that foster and maintain collaboration between users and other stakeholders help them stay motivated during sustainment.

Conclusions

Our findings highlight the dynamic nature of available resources and their impacts on the implementation climate across different phases of implementation. A better understanding of the dynamics of available resources over time from the users’ perspectives will allow the adaptation of resources to better meet the needs of the intervention stakeholders.

Peer Review reports

Background

The availability of organizational resources, including funds, training, staffing, technological support, and time, dedicated to the implementation and ongoing operations of interventions has received long-standing attention in implementation science [1,2,3,4,5]. The level of available resources is often conceptualized as an independent variable that can influence the success or failure of the implementation outcomes (the dependent variable) in determinant frameworks, such as the Promoting Action on Research Implementation in Health Services (PARIHS) [6, 7], the conceptual model [4], and the Consolidated Framework for Implementation Research (CFIR) [8].

Available resources

While the existing literature predominantly views available resources as one factor that determines the success or failure of an intervention, most studies focus on the impact of available resources on implementation as a whole or focus primarily on the early stage of implementation [9,10,11,12]. Research that used determinant frameworks may not explicitly take a process perspective of implementation and fail to recognize the changing implementation needs over time [13]. In reality, the amount and type of resources needed to plan and execute implementation strategies and adapt the intervention to the clinical context are all expected to change over the course of an intervention [14]. This is particularly true for technology-based interventions in healthcare settings due to the dramatic alterations from technical to social problems during technological adoption. As the implementation process unfolds over time, users of interventions may encounter different kinds of implementation challenges that require different types of support and resources. These qualitative shifts within a resource type have not been adequately studied.

Implementation climate

Understanding the users’ perspectives is essential to the implementation success as the users hold first-hand knowledge of which available resources or facilitating factors they need to better integrate the intervention into their daily work environments [15, 16]. To assess the effectiveness of an intervention from the users’ perspectives, researchers in implementation science have turned their attention to defining and measuring implementation climate [17,18,19,20]. Implementation climate refers to the extent to which intended users perceive the use of the intervention as expected, supported, and rewarded within their organizations [21]. Over 25 years, implementation climate has been shown to be a vital component of implementation success [17, 22,23,24,25]. While there are a few studies investigating whether and how implementation climate changes over time, they have focused primarily on the effect of leadership on implementation climate [24, 25]. Just as understanding the dynamic changes in resource use requires more empirical data describing resource demands over time, research on implementation climate may benefit from more data regarding how the effect of different resource types on implementation climate changes over the course of the intervention.

DOAC Population Management Dashboard

This study examines the dynamic relationships between available resources and implementation climate using the DOAC Population Management Dashboard as an example. The dashboard is an innovative technological intervention that leverages clinical data from electronic health records (EHR) to improve the safe prescribing and management of direct oral anticoagulants (DOACs). It updates the traditional care model to a population-based management approach, allowing oversight of all patients prescribed DOACs and intervening only when clinically necessary [26]. Studies show that it improves patient safety and pharmacy efficiency, reduces adverse events, and increases clinic access [27, 28]. The DOAC Dashboard is an effective technological intervention for evidence-based anticoagulation medication management that can be implemented nationwide in various clinical contexts [29].

In our previous qualitative research on the implementation process of the DOAC Dashboard, we identified five key determinants that affect implementation success: (1) clinician authority and autonomy; (2) clinician self-identity and job satisfaction; (3) documentation, communication, and administrative needs; (4) staffing and work schedule; and (5) integration with existing information systems [30]. Our previous study implicitly recognized the significance of available resources and implementation climate. However, we did not specifically investigate how these factors may change over time, especially during the transition from the implementation phase to the sustainment phase. This knowledge gap is critical because the dynamics of resource availability and implementation climate can significantly impact the long-term success and sustainability of technological interventions in healthcare settings.

As the implementation of technological interventions in healthcare settings continues to increase, it is crucial to gain a deeper understanding of the interplay between available resources and implementation climate. The effective deployment of available resources requires a comprehensive understanding of their demand, allocation, and the specific contextual factors that influence their utilization. Therefore, there is a need for empirical insights to inform policy and practice in this domain. To address this need and advance our understanding of the role of available resources and implementation climate, as well as their dynamic interplay, this study utilized a secondary analysis approach. We analyzed interviews from our previous research and applied CFIR and a process perspective to a nationwide implementation effort for the DOAC Population Management Dashboard. Specifically, this study aims to answer the following questions: (a) How do available resources and implementation climate shift across different phases of implementation? (b) How do the changes in available resources affect implementation climate in various stages of implementation?

Methods

Research setting and participants

We used interview data derived from our previous research on nationwide implementation efforts of the DOAC Population Management Dashboard in the United States Veterans Health Administration (VA) [30]. The VA health system offers healthcare services to more than 9 million enrolled veterans each year [31]. The VA’s anticoagulation clinic services employ pharmacists who are supported by the DOAC Population Management Dashboard to help identify and reduce the potentially unsafe use of DOACs [29]. The DOAC Population Management Dashboard had been available to most VA pharmacists for up to 3 years at the time of the interviews, which were conducted between November 2019 and November 2020. We used a comparative multiple-case study design with the goal of analyzing the patterns across different VA clinical sites in order to produce more generalizable knowledge about whether and how available resources and implementation climate change across implementation phases specifically for technological interventions in healthcare settings. This study was approved by the institutional review board of the Ann Arbor VA Healthcare System. Interview participants were identified by clinic managers and invited to participate via e-mail communication. The semi-structured interview guide was created using predefined constructs from the Consolidated Framework for Implementation Research (CFIR) [8] and the Technology Acceptance Model (TAM) [32] (Additional file 1). The interviews and data analysis were conducted by the research staff with training in qualitative research. The average length of interviews was 37 min (range 25–57 min). Each transcript was de-identified prior to coding and analysis.

Qualitative data coding

We used the qualitative analysis software MAXQDA [33] to code and analyze the existing interview data. Data coding and analysis were guided by the VA Quality Enhancement Research Initiative (QUERI) Roadmap and the CFIR.

The interview transcripts were first coded by implementation phases adapted from the VA QUERI Roadmap, including the phases of pre-implementation, implementation, and sustainment [34]. We coded participants’ responses regarding their implementation efforts prior to the adoption of the DOAC Population Management Dashboard as the pre-implementation phase, including the process of identifying problems and potential solutions to optimize anticoagulation management, engaging stakeholders, and assessing organizational capacity prior to implementing the dashboard. Participants’ responses would be coded as the implementation phase if they discussed how the dashboard was first introduced to their clinics and the process of learning the new tool, activating the implementation teams, getting technical and adaptive support, and cleaning the backlog of notifications in the early phase of using the DOAC Population Management Dashboard. We coded participants’ statements as the sustainment phase if they were related to whether and how the dashboard has been embedded and become routine practice in their clinical settings as well as their ongoing evaluation and reflection regarding the impact of the dashboard on their anticoagulation practice.

When we coded interview data using the CFIR, we focused on two specific CFIR codes in the inner setting domain: available resources and implementation climate. In the initial stage of coding, we discovered that certain elements of available resources mentioned by participants were closely related to other CFIR constructs, such as champions and leadership engagement. Consequently, to capture this relationship, we expanded the definition of available resources to encompass the support provided by champions, leaders, and other stakeholders involved in the implementation process. When examining the CFIR code of implementation climate, we also included the six sub-constructs of implementation climate, including tension for change, compatibility, relative priority, organizational incentives and rewards, goals and feedback, and learning climate [8]. The expanded definition of available resources, along with the comprehensive exploration of the implementation climate and its six sub-constructs were applied when coding the transcripts and assigning ratings to the CFIR constructs (as described in the “Rating with CFIR constructs” section).

To achieve consistency during the coding process, three researchers (AR, FJS, and YL) independently coded the same transcripts and met to discuss the overlaps and discrepancies in coding after each transcript. After independent coding and discussion of five transcripts, coding was consistent across all coders. Upon achieving consensus, we divided the remaining transcripts among the team for individual coding. The team continued to meet and discuss any questions that came up during our individual coding process and used consensus to resolve any statements that were perceived as ambiguous or difficult to code clearly.

Rating with CFIR constructs

Once coding was complete, we examined the intersection between the three implementation phases (pre-implementation, implementation, and sustainment) and the CFIR codes. Using MAXQDA, all coded segments were aggregated, sorted by the interviewee’s site and study ID, and grouped by phase. We reviewed the coded segments from the interview data and created a document containing all summary statements, plus supporting quotes for each interview.

The coded segments in the document were rated using the previously published CFIR rating system [11, 35]. Three researchers (AR, FJS, and YL) first rated the same cases to ensure the consistency of the rating. After rating each case independently, we discussed the ratings until a consensus was reached. After two cases, the ratings were consistent across raters. We then divided the remaining cases and rated them independently. During the rating process, the three researchers continued to meet to resolve any potential discrepancies.

We created a table to place two CFIR constructs (available resources and implementation climate) in the columns and three implementation phases (pre-implementation, implementation, and sustainment) in the rows for each site to facilitate the rating of segments where implementation phases and CFIR constructs overlapped. In accord with the CFIR scoring system [11, 35], interview segments with the two CFIR codes were rated with a 5-point bipolar scale (− 2 to + 2) to reflect the valence (positive or negative) and magnitude of changes in available resources and implementation climate over the course of implementation. Interviews with the majority of statements that were positive toward a construct’s influence on the implementation process were assigned a score of + 2 or + 1 (facilitator) depending on the strength or magnitude of the construct discussed by the interviewee. Similarly, interviews with negative statements were assigned a score of − 2 or − 1 (barrier). An equal mixture of positive and negative statements received a 0 score (neutral) for that CFIR construct.

When there were various dimensions of CFIR constructs, such as different types of available resources or sub-constructs of implementation climate, we rated each item individually but also took into account the relative importance of each item. A weighted average was generated and assigned to the two CFIR constructs to reflect the relative importance of some dimensions of available resources or implementation climate at each site. We excluded the data of CFIR constructs during the pre-implementation phase from this analysis because too many participants either were not involved in the pre-implementation phase or were not knowledgeable about the available resources prior to the implementation.

Thematic analysis

After the rating process was complete, we used a graph to visualize the way that available resources and implementation climate change from implementation to sustainment phases for each site. We categorized sites based on the direction of change in the two CFIR constructs across phases. We also conducted a qualitative thematic analysis across three groups, as specified in the results, to identify the key resources that affected the implementation climate at different phases and examine how available resources and implementation climate interacted with each other during these phases.

Results

Interviewee characteristics

In this study, we focus on anticoagulation professionals’ experiences of using the DOAC Population Management Dashboard during both the implementation phase and sustainment phase at the VA clinical sites across the USA. We analyzed transcripts of 20 semi-structured phone interviews conducted with anticoagulation professionals at 17 VA clinical sites, which had reached the sustainment phase of implementation of the DOAC Population Management Dashboard. Interview participants included 13 pharmacists, one pharmacy technician, and six pharmacy clinic managers. Regarding staff involvement in the intervention, 18 out of 20 interviewees shared details about their duration of engagement with the intervention. The average length of their involvement, as reported at the time of the interview, was 2 years. Furthermore, 95% of the interviewees reported being the initial users of the DOAC Dashboard since its adoption by their clinics.

Changes in resource availability and implementation climate over time

In our previous research interviews [30], we explored the barriers encountered during the implementation of the DOAC Dashboard. By analyzing participants’ responses, we were able to gain valuable insights into the resources or support they used to overcome these barriers. During the initial analysis of the transcripts, we found that certain available resources mentioned by the participants were closely connected to other CFIR constructs, such as champions and leadership engagement. For example, several participants mentioned that champions played an important role in offering training and other needed resources; time and staffing dedicated to the intervention were closely linked to formal leadership support. We therefore expanded our definition of available resources to incorporate the support from stakeholders, such as champions, leaders, and other implementation partners to fit the context of DOAC Dashboard implementation.

To understand how available resources and implementation climate vary over time and across clinical sites, we graphed each site’s scores on these two parameters from implementation to sustainment phases (Fig. 1). This allows us to compare the “direction of movements” of the relative positions of the two CFIR constructs among various sites.

Fig. 1
figure 1

Changes in implementation climate and available resources from the implementation phase to the sustainment phase

Full size image

In the early phase of implementation, the sites in the study had a wide variety of available resources and implementation climate with scores ranging from − 2 to + 2 for both measures. Among the 17 sites, nine sites (green arrows and dots) had a more positive implementation climate in the sustainment phase. These sites generally had an increase in resource availability or maintained the same level of high resource availability (+ 2) over the trajectory of the implementation process. Four sites (black arrows and dots) had no change in implementation climate while their resource availability increased or maintained the same moderate levels (− 1 to 0). The sites with increased resources but no improvement in implementation climate (black arrows) are further examined in the following section (group 2). Four sites out of 17 had a decreased level of implementation climate (or remained the lowest (− 2)) on this dimension, and these sites all had the lowest level of resource availability in the sustainment phase.

The interplay between available resources and implementation climate

Based on the groupings in Fig. 1, we conducted a thematic analysis to examine what types of available resources influenced the implementation climate in the implementation phase and the sustainment phase.

Group 1: Sites with positive changes in both available resources and implementation climate

We first focused on the sites with positive changes in both resource availability and implementation climate across phases (Table 1). Champions, time, staffing, and leadership support during the implementation phase were identified as key resources that facilitate the adoption of the intervention at the clinical sites of those interviewed.

Table 1 Examples of resource-climate interactions at group 1 clinical sites
Full size table

Champions

Several participants from these sites mentioned that certain champions, such as intervention developers or experienced users, played an important role in providing support to help them trust and use the DOAC Population Management Dashboard. The participants stated that despite initial concerns about the reliability of the intervention, the support provided by champions during the implementation phase helped them develop trust and acceptance of this new tool. Notably, the champions’ support, according to the participants, not only included help resolving technical problems but also an in-depth explanation of the logic behind various actions on the dashboard; moreover, the participants found it valuable that the champions empathized with them and understood their pain points. Meanwhile, participants reported that no formal training was provided when they first started to use the dashboard at their VA clinical sites. As a result, some of the participants learned and navigated the intervention by themselves during the implementation phase. They then helped create training materials or step-by-step guidelines for other colleagues who implemented the intervention.

Time, staffing, and leadership

Whether there were sufficient time and staff for implementing the DOAC Population Management Dashboard greatly influenced the implementation climate during the implementation phase. At some sites, for example, the pharmacists continued using their previous system while implementing the new tool to verify that the new tool was providing accurate alerts and catching all potential errors for their patients. In such cases in which participants and their colleagues were able to experiment with the new tool, the resistance to the intervention tended to diminish over time as the users could verify the accuracy of the dashboard. Participants also highlighted the need for time and staff when faced with the challenge of processing an initial backlog of patient alerts in order to establish the ideal practice setting in which the alerts could be reviewed and managed daily by anticoagulation professionals. According to some participants, it took a significant amount of time or recruitment of extra help to review and process the initial backlog of alerts. They reported that to accomplish that, their leaders at the clinical sites often played an essential role in the deployment of necessary resources.

In the sustainment phase, time and staff remained key resources that affected the implementation climate. While some clinical sites were able to quickly activate existing staff or recruit extra help to learn and implement the intervention, many sites found it challenging to integrate the dashboard into their workflow during the implementation phase. According to the participants, as they already worked on various medication management tasks and patient care services, such as walk-in patient consultations, the dashboard often took a lower priority at their clinical sites. The lack of dedicated staff and time gradually surfaced as a major barrier to the sustainment of intervention over time. To overcome this barrier, these clinical sites ended up hiring staff that had dedicated time for dashboard management in the sustainment phase.

Support from partners

The support from implementation partners was also a critical factor that influenced the implementation climate during the sustainment phase. Since the DOAC Population Management Dashboard is designed for the use of anticoagulation professionals (pharmacists or nurses) to monitor safe outpatient anticoagulant prescribing by physicians and other clinicians, prescribers’ attitudes toward the recommendations made by the anticoagulation professionals greatly affected the implementation climate at the clinical site. According to one interviewee, despite feeling supported by the clinical site most of the time, the physicians’ preference for individual clinical judgment over the guidelines on the management of DOACs still caused frustration.

Group 2: Sites with increased resources but no improvement in implementation climate

We identified instances where available resources increased as the intervention moved from the implementation phase to the sustainment phase; however, the implementation climate did not improve (Table 2).

Table 2 Examples of resource-climate interactions at group 2 clinical sites
Full size table

In the early implementation phase, one interviewee, who was a program manager, was aware of the need for more staff to keep up with the DOAC Dashboard due to a significant increase in the number of patients switching from warfarin to DOACs. Over time, the participant’s clinical site had increased the number of pharmacists working on the intervention as a result of the VA’s policy change (the Diffusion of Excellence Initiative). Despite the increase in human resources in the sustainment phase, the interviewee and colleagues still had difficulties integrating the DOAC Dashboard into their existing workflow due to the lack of a dedicated team and time.

One interviewee from another site also discussed the issue of the lack of dedicated time. The staff needed to use their downtime at another clinic to manage the dashboard. Although the staff was still able to maintain the dashboard regularly, the use of the staff’s downtime did not significantly influence the implementation climate in this case.

Group 3: Sites with negative changes in both resources and implementation climate

Lastly, we examined the clinical sites where available resources and implementation climate changed in a negative way over time (Table 3). One interviewee wished there was technological support to streamline the dashboard and other tools into one system to make the tool more implementable. Although the interviewee was aware of the potential benefits of using the intervention, time constraints throughout the implementation process appeared to be a major barrier to integrating the tool into workflows and caused the staff to experience guilt and burnout at work.

Table 3 Examples of resource-climate interactions at group 3 clinical sites
Full size table

Another critical aspect of the interaction between available resources and implementation climate was the relationship between the support from implementation partners (physicians) and the intervention users’ motivation, which was also a theme that emerged from the interview data of group 1. One interviewee felt that the time they spent on the intervention was not valued as the physicians did not often sign their notes about medication renewals or changes. Even though the pharmacists identified potential medication errors from the dashboard, they still needed collaboration and support from physicians to make changes in patients’ medications. When the physicians chose to ignore the notes, the pharmacists and nurses could not see any difference they made to patient outcomes through the intervention use and would not feel motivated in the process.

Summary: the dynamics of available resources across phases

Our findings of the dynamics of key resources across implementation phases are summarized in Table 4. Technological resources and champions played critical roles in facilitating users’ adoption of the technology-based intervention. Sufficient time and staffing were key resources throughout the implementation process. Having time to experiment with the new tool and to establish the ideal practice setting laid a solid foundation for the sustainment of the intervention. Our interview data showed that having more staff and time was not necessarily sufficient to guarantee success; the organizations needed time and staff specifically dedicated to intervention use in the sustainment phase. The deployment of time and staff often required support from leadership at the organizations. Consistent support and cooperation provided by implementation partners were essential to maintain users’ motivation over time.

Table 4 The dynamics of available resources across phases
Full size table

Discussion

To our knowledge, this is the first empirical study which (1) examines the dynamics of available resources and their influence on the implementation climate of an intervention across the implementation and sustainment phases and (2) contributes to a better understanding of the changing needs for resources and support over the course of implementing technology-based interventions in healthcare settings.

The dynamic nature of available resources

Using the CFIR rating system, we assessed the levels of resource availability and the implementation climate across phases of the implementation program. We found that users’ resource needs were not static; instead, both the quantity and types of resources required for supporting the users of the intervention shifted throughout the phases of the implementation process. In the implementation phase, users required technological resources that streamlined the integration of the tool into their existing systems; in addition, they needed the support of champions to help them overcome apprehension and build trust in the new technological tool. In the sustainment phase, participants expressed the need for collaborative relationships and support from implementation partners (physicians) to implement the medication appropriateness recommendations. In some cases, the lack of support from the physicians diminished users’ motivation over time.

Most cases in this comparative study were in line with previous research that demonstrated a positive association between resource availability and implementation climate [36,37,38]. Nevertheless, we also identified instances where increased resources did not guarantee a more positive implementation climate over time. For example, one site with increased human resources involved in the intervention use did not result in a more positive implementation climate in the sustainment phase because the staff and time were diverted to competing priorities. Our findings echo those of previous studies that have shown the importance of dedicated resources in the sustainment of interventions [39, 40].

Support beyond the technology

To implement the technology-based intervention, the intervention users needed different types of support that went beyond the technical aspects of an intervention and varied over time. Specifically, interview participants pointed out that social and emotional support provided by the champions was critical during the adoption and implementation of the new technological intervention. This finding strongly resonates with recent research that highlights the anxiety and burnout experienced by healthcare professionals during the adoption and utilization of EHRs and health information technology and underscores the importance of providing appropriate support to address these challenges [41, 42].

For many of the pharmacists who first adopted the DOAC Dashboard, it was a huge transition from monitoring individual patients to using a technology-based population health management approach that relies on data analytics to identify patients prescribed DOAC medications and to screen for any potential prescription red flags (e.g., inappropriate dosing for a given renal function). Since the intervention entailed a very different mechanism to identify and manage potential medication errors, pharmacists needed an in-depth understanding of the logic of the dashboard, and it took time for the pharmacists to verify the accuracy and ensure the intervention provided patients with safe and effective medication use, which is the core of their professional identity [43]. The support from champions included helping users understand the rationale behind the dashboard codes, sharing successful experiences and benefits associated with using the intervention, empathizing with users, and relating to their frustrations during the initial implementation phase.

The collaboration between the intervention users and other implementation partners was an important theme in our interview data. The implementation partners (the primary care physicians) have played a critical role in making the changes recommended by the anticoagulation professionals. As one interviewee pointed out, increasing resources, such as enhancing provider education in these clinical sites, may encourage prescribers to adopt recommendations for safer DOAC monitoring. Additionally, establishing proactive measures to promote mutual respect and organizational norms for effective communication and collaboration between pharmacists/nurses and prescribers during the medication review process may further encourage prescribers to embrace safer DOAC prescribing recommendations. By fostering and maintaining collaboration among stakeholders, the pharmacists would feel more motivated to use and sustain the intervention, ultimately driving positive changes in patient care.

Study limitations

This study has important limitations that must be considered when interpreting the results. First, it relied on self-reported information about the implementation process and without on-site observations, which would have been impractical. We also acknowledge the unavailability of policy/practice documentation or other types of data for the purpose of triangulation. Second, applying the CFIR scoring approach to qualitative interviews may unavoidably result in the omission of certain nuanced aspects of the qualitative data. Third, we focused mainly on the implementation and sustainment phases, and especially on the front-line health professionals’ experiences with the intervention. We did not examine how available resources were planned and allocated during the pre-implementation phase, which often occurred at the higher level of the organization. Fourth, we discussed the role of leadership in the implementation from the perspectives of the participants. Additionally, the views of the intervention from patients were also beyond the scope of this study. Fifth, the findings of this study were drawn from a technology-based intervention and their applicability to a non-technology implementation intervention remains to be studied. Finally, this study was conducted in the United States VA health system, and results may not be generalizable to the settings outside of the VA or the USA.

Despite the limitations, this study highlights the dynamic nature of available resources and their impacts on the implementation climate across different phases of implementation. A more nuanced understanding of the dynamics of available resources over time from the users’ perspectives will allow adaptation to the resources to better meet the needs of the intervention stakeholders. Future studies can further explore the factors that affect physicians’ implementation of pharmacists’ recommendations and the strategies to facilitate collaboration between physicians and pharmacists.

Conclusions

This study demonstrates the dynamic interactions between available resources and implementation climate across implementation phases. Combining the determinant and process approaches, we examined the changes in and interactions between available resources and implementation using stakeholder interviews. Our findings showed that the resources necessary to support the successful implementation of an intervention are not static; instead, both the quantity and types of resources shift based on the phases of the intervention. This study highlights the importance of different types of support for intervention users from the implementation phase to the sustainment phase. In the implementation phase, users needed technological support that facilitates the integration of the intervention into users’ existing system and social/emotional support to help them overcome apprehension and develop trust in adopting the new technological intervention. In the sustainment phase, the support from other implementation partners, especially the physicians, greatly affected whether the intervention could produce a positive outcome in patient care. Fostering and maintaining the collaboration between users and other implementation partners could help the users stay motivated during the sustainment phase.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to the regulatory compliance of the Department of Veterans Affairs.

Abbreviations

CFIR:

Consolidated Framework for Implementation Research

DOAC:

Direct oral anticoagulant

EHR:

Electronic health record

PARIHS:

Promoting Action on Research Implementation in Health Services

QUERI:

Quality Enhancement Research Initiative

VA:

Veterans Health Administration

References

  1. Gustafson DH, Sainfort F, Eichler M, Adams L, Bisognano M, Steudel H. Developing and testing a model to predict outcomes of organizational change. Health Serv Res. 2003;38(2):751–76.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Edmondson AC, Bohmer RM, Pisano GP. Disrupted routines: team learning and new technology implementation in hospitals. Adm Sci Q. 2001;46(4):685–716.

    Article  Google Scholar 

  3. Fitzgerald L, Ferlie E, Wood M, Hawkins C. Interlocking Interactions, the Diffusion of Innovations in Health Care. Hum Relations. 2002;55(12):1429–1449.

  4. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Weiner BJ, Savitz LA, Bernard S, Pucci LG. How do integrated delivery systems adopt and implement clinical information systems? Health Care Manage Rev. 2004;29(1):51–66.

    Article  PubMed  Google Scholar 

  6. Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998;7(3):149–58.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Rycroft-Malone J. Promoting Action on Research Implementation in Health Services (PARIHS). In: Rycroft-Malone J, Bucknall T, editors. Models and frameworks for implementing evidence-based practice: linking evidence to action. Oxford: Wiley-Blackwell; 2010. p. 109–34.

    Google Scholar 

  8. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Bach-Mortensen AM, Lange BCL, Montgomery P. Barriers and facilitators to implementing evidence-based interventions among third sector organisations: a systematic review. Implement Sci. 2018;13(1):103.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Puchalski Ritchie LM, Khan S, Moore JE, Timmings C, van Lettow M, Vogel JP, et al. Low- and middle-income countries face many common barriers to implementation of maternal health evidence products. J Clin Epidemiol. 2016;1(76):229–37.

    Article  Google Scholar 

  11. Varsi C, Ekstedt M, Gammon D, Ruland CM. Using the Consolidated Framework for Implementation Research to identify barriers and facilitators for the implementation of an Internet-based patient-provider communication service in five settings: a qualitative study. J Med Internet Res. 2015;17(11):e5091.

    Article  Google Scholar 

  12. Geerligs L, Rankin NM, Shepherd HL, Butow P. Hospital-based interventions: a systematic review of staff-reported barriers and facilitators to implementation processes. Implement Sci. 2018;13(1):36.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):53.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res. 2020;1(283):112461.

    Article  Google Scholar 

  15. McGinn CA, Grenier S, Duplantie J, Shaw N, Sicotte C, Mathieu L, et al. Comparison of user groups’ perspectives of barriers and facilitators to implementing electronic health records: a systematic review. BMC Med. 2011;9(1):46.

    Article  PubMed  PubMed Central  Google Scholar 

  16. McGinn CA, Gagnon MP, Shaw N, Sicotte C, Mathieu L, Leduc Y, et al. Users’ perspectives of key factors to implementing electronic health records in Canada: a Delphi study. BMC Med Inform Decis Mak. 2012;12(1):105.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Weiner BJ, Belden CM, Bergmire DM, Johnston M. The meaning and measurement of implementation climate. Implement Sci. 2011;22(6):78.

    Article  Google Scholar 

  18. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implement Sci. 2014;9(1):157.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Jacobs SR, Weiner BJ, Bunger AC. Context matters: measuring implementation climate among individuals and groups. Implement Sci. 2014;17(9):46.

    Article  Google Scholar 

  20. Peters N, Borge RH, Skar AMS, Egeland KM. Measuring implementation climate: psychometric properties of the Implementation Climate Scale (ICS) in Norwegian mental health care services. BMC Health Serv Res. 2022;22(1):23.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Klein KJ, Sorra JS. The challenge of innovation implementation. AMR. 1996;21(4):1055–80.

    Article  Google Scholar 

  22. Ehrhart MG, Shuman CJ, Torres EM, Kath LM, Prentiss A, Butler E, et al. Validation of the implementation climate scale in nursing. Worldviews Evid Based Nurs. 2021;18(2):85–92.

    Article  PubMed  Google Scholar 

  23. Woodard GS, Triplett NS, Frank HE, Harrison JP, Robinson S, Dorsey S. The impact of implementation climate on community mental health clinicians’ attitudes toward exposure: an evaluation of the effects of training and consultation. Implement Res Pract. 2021;1(2):26334895211057884.

    Google Scholar 

  24. Williams NJ, Wolk CB, Becker-Haimes EM, Beidas RS. Testing a theory of strategic implementation leadership, implementation climate, and clinicians’ use of evidence-based practice: a 5-year panel analysis. Implement Sci. 2020;15(1):10.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Skar AMS, Braathu N, Peters N, Bækkelund H, Endsjø M, Babaii A, et al. A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate. BMC Health Serv Res. 2022;22(1):298.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Allen AL, Lucas J, Parra D, Spoutz P, Kibert JL, Ragheb B, et al. Shifting the paradigm: a population health approach to the management of direct oral anticoagulants. J Am Heart Assoc. 2021;10(24):e022758.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Rossier C, Spoutz P, Schaefer M, Allen A, Patterson ME. Working smarter, not harder: evaluating a population health approach to anticoagulation therapy management. J Thromb Thrombolysis. 2021;52(1):200–8.

    Article  CAS  PubMed  Google Scholar 

  28. Valencia D, Spoutz P, Stoppi J, Kibert JL, Allen A, Parra D, et al. Impact of a direct oral anticoagulant population management tool on anticoagulation therapy monitoring in clinical practice. Ann Pharmacother. 2019;53(8):806–11.

    Article  PubMed  Google Scholar 

  29. Barnes GD, Sippola E, Dorsch M, Errickson J, Lanham M, Allen A, et al. Applying population health approaches to improve safe anticoagulant use in the outpatient setting: the DOAC Dashboard multi-cohort implementation evaluation study protocol. Implement Sci. 2020;15(1):83.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Barnes GD, Sippola E, Ranusch A, Takamine L, Lanham M, Dorsch M, et al. Implementing an electronic health record dashboard for safe anticoagulant management: learning from qualitative interviews with existing and potential users to develop an implementation process. Implement Sci Commun. 2022;3(1):I re.

    Article  Google Scholar 

  31. Veterans Health Administration. About VHA. 2022. Available from: https://www.va.gov/health/aboutVHA.asp.

    Google Scholar 

  32. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13(3):319–40.

    Article  Google Scholar 

  33. VERBI Software. MAXQDA 2020 [computer software]. Berlin: VERBI Software; 2020. Available from: https://www.maxqda.com/.

    Google Scholar 

  34. Kilbourne AM, Goodrich DE, Miake-Lye I, Braganza MZ, Bowersox NW. Quality enhancement research initiative implementation roadmap: toward sustainability of evidence-based practices in a learning health system. Med Care. 2019;57 Suppl 10 Suppl 3:S286–93.

    Article  PubMed  Google Scholar 

  35. Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the Consolidated Framework for Implementation Research (CFIR). Implement Sci. 2013;8(1):51.

    Article  PubMed  PubMed Central  Google Scholar 

  36. McKay VR, Hoffer LD, Combs TB, Margaret DM. The dynamic influence of human resources on evidence-based intervention sustainability and population outcomes: an agent-based modeling approach. Implement Sci. 2018;13(1):77.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Denis JL, Hébert Y, Langley A, Lozeau D, Trottier LH. Explaining diffusion patterns for complex health care innovations. Health Care Manage Rev. 2002;27(3):60–73.

    Article  PubMed  Google Scholar 

  38. Perrin KM, Burke SG, O’Connor D, Walby G, Shippey C, Pitt S, et al. Factors contributing to intervention fidelity in a multi-site chronic disease self-management program. Implement Sci. 2006;1(1):26.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Batsis JA, McClure AC, Weintraub AB, Sette D, Rotenberg S, Stevens CJ, et al. Barriers and facilitators in implementing a pilot, pragmatic, telemedicine-delivered healthy lifestyle program for obesity management in a rural, academic obesity clinic. Implement Sci Commun. 2020;1(1):83.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Hassen N, Lofters A, Michael S, Mall A, Pinto AD, Rackal J. Implementing anti-racism interventions in healthcare settings: a scoping review. Int J Environ Res Public Health. 2021;18(6):2993.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Goldberg DG, Soylu TG, Grady VM, Kitsantas P, Grady JD, Nichols LM. Indicators of workplace burnout among physicians, advanced practice clinicians, and staff in small to medium-sized primary care practices. J Am Board Fam Med. 2020;33(3):378–85.

    Article  PubMed  Google Scholar 

  42. Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc. 2020;27(4):531–8.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Nelson NR, Armistead LT, Blanchard CM, Rhoney DH. The pharmacist’s professional identity: preventing, identifying, and managing medication therapy problems as the medication specialist. J Am Coll Clin Pharm. 2021;4(12):1564–71.

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the VA interview participants who made this study possible, as well as the former research staff (Emily Sippola) who conducted the interviews.

Funding

This project is funded by the Agency for Healthcare Research and Quality (R18HS026874). The funder had no role in the study design, data collection, analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Center for Bioethics and Social Sciences in Medicine, University of Michigan, Ann Arbor, MI, USA

    Ying-Jen Lin, F. Jacob Seagull & Geoffrey D. Barnes

  2. Center for Clinical Management Research, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI, USA

    Allison Ranusch & Jeremy B. Sussman

  3. Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MI, USA

    Jeremy B. Sussman & Geoffrey D. Barnes

  4. Division of General Medicine, Department of Internal Medicine, University of Michigan, Ann Arbor, MI, USA

    Jeremy B. Sussman

  5. Frankel Cardiovascular Center, Department of Internal Medicine, University of Michigan, 2800 Plymouth Rd, B14 G214, Ann Arbor, MI, 48109-2800, USA

    Geoffrey D. Barnes

Authors
  1. Ying-Jen Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allison Ranusch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. F. Jacob Seagull
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeremy B. Sussman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Geoffrey D. Barnes
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AR conducted the interviews. AR, FJS, and YL coded and analyzed the interview data. YL and AR drafted the manuscript. GDB, JS, and FJS provided critical revisions to the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Geoffrey D. Barnes.

Ethics declarations

Ethics approval and consent to participate

Regulatory approval was obtained from the institutional review board (IRB) at the VA Ann Arbor Healthcare System. Verbal informed consent was obtained from each interview participant.

Consent for publication

Not applicable.

Competing interests

GDB discloses consulting fees from Pfizer, Bristol-Myers Squibb, Janssen, Bayer, Sanofi, Abbot Vascular, and Boston Scientific and grant funding from Boston Scientific. The other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, YJ., Ranusch, A., Seagull, F.J. et al. Dynamic interplay between available resources and implementation climate across phases of implementation: a qualitative study of a VA national population health tool. Implement Sci Commun 4, 74 (2023). https://doi.org/10.1186/s43058-023-00460-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-023-00460-0

Keywords

Implementation Science Communications

ISSN: 2662-2211