Skip to main content
Download PDF

A stakeholder-driven method for selecting implementation strategies: a case example of pediatric hypertension clinical practice guideline implementation

Implementation Science Communications volume 3, Article number: 25 (2022) Cite this article

Abstract

Background

This article provides a generalizable method, rooted in co-design and stakeholder engagement, to identify, specify, and prioritize implementation strategies. To illustrate this method, we present a case example focused on identifying strategies to promote pediatric hypertension (pHTN) Clinical Practice Guideline (CPG) implementation in community health center-based primary care practices that involved meaningful engagement of pediatric clinicians, clinic staff, and patients/caregivers. This example was chosen based on the difficulty clinicians and organizations experience in implementing the pHTN CPG, as evidenced by low rates of guideline-adherent pHTN diagnosis and treatment.

Methods

We convened a Stakeholder Advisory Panel (SAP), comprising 6 pediatricians and 5 academic partners, for 8 meetings (~12 h total) to rigorously identify determinants of pHTN CPG adherence and to ultimately develop a testable multilevel, multicomponent implementation strategy. Our approach expanded upon the Expert Recommendations for Implementation Change (ERIC) protocol by incorporating a modified Delphi approach, user-centered design methods, and the Implementation Research Logic Model (IRLM). At the recommendation of our SAP, we gathered further input from youth with or at-risk for pHTN and their caregivers, as well as clinic staff who would be responsible for carrying out facets of the implementation strategy.

Results

First, the SAP identified 17 determinants, and 18 discrete strategies were prioritized for inclusion. The strategies primarily targeted determinants in the domains of intervention characteristics, inner setting, and characteristics of the implementers. Based on SAP ratings of strategy effectiveness, feasibility, and priority, three tiers of strategies emerged, with 7 strategies comprising the top tier implementation strategy package. Next, input from caregivers and clinic staff confirmed the feasibility and acceptability of the implementation strategies and provided further detail in the definition and specification of those strategies.

Conclusions

This method—an adaptation of the ERIC protocol—provided a pragmatic structure to work with stakeholders to efficiently identify implementation strategies, particularly when supplemented with user-centered design activities and the intuitive organizing framework of the IRLM. This generalizable method can help researchers identify and prioritize strategies that align with the implementation context with an increased likelihood of adoption and sustained use.

Peer Review reports

Background

A ubiquitous challenge for implementation researchers is selecting appropriate implementation strategies to improve the adoption, implementation, and sustainment of effective interventions. Oftentimes, implementation strategies are selected based on theory and prior research. For example, implementation researchers may use the Expert Recommendations for Implementing Change (ERIC) [1], a compilation of 73 discrete implementation strategies that was developed through a modified Delphi process with a wide range of stakeholders and is useful for identifying strategies and matching them to identified determinants [2]. Powell et al. [3] have also proposed a variety of methods for matching implementation strategies to identified barriers and facilitators, including concept mapping, group model building, conjoint analysis, and intervention mapping. However, following theoretical methods or relying solely on prior research does not always successfully translate to a new context and/or for a particular intervention as these methods do not account for on-the-ground stakeholder knowledge and preferences.

To overcome limitations of other methods for selecting implementation strategies, we propose a rigorous and generalizable stakeholder-driven method. Stakeholder engagement is a keystone of implementation research [4]. Stakeholders are often engaged throughout the research process; however, they are most likely to be engaged in data synthesis and dissemination (i.e., later in the implementation process) [5]. Engaging stakeholders in the identification, operationalization, and selection of implementation strategies is more likely to produce strategies that will be taken up and result in adoption and sustained implementation.

Herein, we illustrate a rigorous stakeholder-driven method for selecting implementation strategies using a case example with the goal of developing a multilevel, multicomponent strategy for the implementation of the clinical practice guidelines (CPG) for pediatric hypertension (pHTN) in safety-net community health centers [6].

Case example: pediatric hypertension clinical practice guideline implementation

Despite CPGs for pHTN being in place for decades [7, 8], evidence indicates poor adherence [7, 9,10,11]. Between 2 and 4% of children in the US general population have pHTN [12], and over 16% have elevated BP (previously called pre-hypertension) [13, 14]. In one electronic health record (EHR) review, nearly 85% of children who met the criteria for elevated blood pressure (BP) or pHTN were undiagnosed [15]. Guideline-adherent pHTN diagnoses are highly predictive of having HTN as an adult [16]. The consequences of untreated pHTN include left ventricular hypertrophy, neurocognitive deficits, and target organ damage in adolescence [17,18,19], as well as hypertension, metabolic syndrome, and left ventricular hypertrophy in adulthood [16, 20].

Numerous barriers to CPG adherence have been identified [11, 21]. One qualitative study found that primary care clinicians perceived significant barriers at both the system and patient levels. These included lack of systematic approach to measuring (e.g., children not sitting still; lack of proper equipment; not using manual BP readings) and reviewing BP values, difficulty interpreting BP readings and coordinating reassessment and necessary clinical actions within the workflow, and difficulty scheduling and completing follow-up appointments [22]. Parents of children with pHTN have also expressed uncertainty about diagnostic accuracy and treatment indication [23].

Identifying and testing implementation strategies to overcome existing barriers and improve pHTN CPG adherence is needed to prevent chronic illness. For example, clinical decision support (CDS) tools within the EHR have been shown to increase the detection and control of hypertension in adults [24, 25]. In contrast, pediatric clinicians using an EHR with such CDS failed to diagnose an alarming 95% of 3- to 17-year-olds whose BP measurements indicated meeting diagnostic criteria for pHTN per the 2017 CPG [26]. The fact that many children and adults with hypertension remain undetected [10] demonstrates that, although promising, health information technologies (HIT) used in isolation (i.e., without strategies to support their use and other aspects of the CPG) may be insufficient for guideline-adherent diagnosis and management [27].

The challenge of CPG adherence is not unique to pHTN. In a scoping review of barriers to CPG adherence and the strategies to overcome them, Fischer et al. [28] broadly grouped implementation strategies into workflow- or clinician-focused. Workflow-focused strategies included CDS tools, as well as standardized documentation and standing orders. Clinician-focused strategies largely focused on communication strategies (e.g., educational materials, ongoing trainings, social interactions between clinicians and opinion leaders). Fischer et al. further noted the importance of tailoring these strategies for the specific condition and setting. A similar process of identifying a multilevel, multicomponent strategy has not been undertaken specifically for pHTN CPG adherence. pHTN CPG adherence was chosen as an exemplar case given the low rates of guideline-adherent pHTN diagnosis and treatment, underscoring the difficulty of clinicians' and organizations' experience in implementing strategies to support adherence to pHTN CPG. The final implementation strategy will be multilevel, as patients, clincians, leadership, and policymakers influence pHTN CPG implementation.

Present study

This article presents a generalizable method focused on meaningful engagement with stakeholders in the identification, specification, and prioritization of implementation strategies. To identify strategies tailored for the condition and setting, as recommended by Fischer et al. [28], we expanded upon an adapted Expert Recommendations for Implementation Change (ERIC) protocol [1, 29, 30]. Specifically, we used a modified Delphi approach, user-centered design activities [31], and the Implementation Research Logic Model (IRLM) [32] in a series of iterative meetings with stakeholders. Through the case example, we illustrate the steps involved in this method for implementation strategy identification, specification, and prioritization and then discuss the advantages of using this process, alternative methodologic considerations, and implications for implementation science.

Methods

Participants

Academic-community partnership

This study is grounded in an academic-community partnership that is the result of longstanding collaborations between Northwestern University Feinberg School of Medicine, Lurie Children’s Hospital of Chicago, and AllianceChicago, an AHRQ-recognized Practice Based Research Network comprising 60 community health centers with more than 200 clinic sites in 19 states as of 2021.

Stakeholder Advisory Panel (SAP)

The SAP comprised pediatric healthcare clinicians and research team members. Pediatric clinicians (n=6) were pediatric or family medicine physicians that would represent the perspective of community health center-based primary care practices that intended to participate in a subsequent implementation trial. The research team members (n=5) who led and participated in the meetings had expertise in pHTN diagnosis and treatment, implementation science focused on chronic disease management, user-centered design, pediatric primary care, health disparities, and use of HIT to support CPG adherence. Pediatric clinicians were recruited from four community health center organizations in the Chicago area that routinely collaborate in practice transformation initiatives using AllianceChicago’s HIT and practice change infrastructure.

Procedures and case example

We used a pragmatic adaptation and expansion of the ERIC protocol (see Fig. 1 for alignment of study activities with the steps of the ERIC protocol) to engage stakeholders in identifying, specifying, and prioritizing implementation strategies [1, 29, 30]. ERIC involves an iterative, multi-method process of qualitative analysis of semi-structured stakeholder meetings, as detailed below. We expanded on the adapted ERIC process by (a) incorporating user-centered design methods [31] to understand determinants and identify strategies related to the assessment and management of pHTN and (b) using the IRLM [32] as a conceptual and organizing framework. SAP meetings were held monthly for 7 months (April–October 2020), and once in January 2021, and lasted 1–2 h each. SAP members spent an average of 12 h in SAP meetings and related activities (e.g., surveys). Meetings occurred via Zoom videoconferencing platform [33], recorded with panelists’ permission, and analyzed by the research team. SAP members were compensated $150 per hour. For replication and generalizability purposes, we now discuss the method by outlining the stakeholder-engaged activities that resulted in the multilevel, multicomponent implementation strategy.

Fig. 1
figure 1

Adapted ERIC protocol with user-centered design and IRLM augmentations

Full size image

Step 1: Identifying barriers to adhering to the CPG for pHTN

SAP members were introduced to the project, meeting logistics, and project specifics by reviewing the study protocol [6]. The SAP then engaged in a semi-structured discussion of current practices in their respective clinics for measuring, diagnosing, and managing BP in children and adolescents, as well as identifying the barriers to adhering to the 2017 CPG for pHTN.

Steps 2 and 3: Understanding context and generating implementation strategies

Stakeholders participated in two user-centered design workshops [31]. First, they were asked to diagram and discuss their workflows for BP measurement, including (a) the pre-encounter vitals, (b) the clinician-patient encounter, and (c) the end-of-visit and follow-up plan. The research team prompted for barriers; communication channels between clinicians, staff, and families about BP results and treatment plans; and recommendations for strategies to overcome named barriers. Although this method was informed by the user-centered design literature [31], assessment and redesign of the workflow was recently suggested as an additional ERIC strategy [34]. Second, stakeholders were introduced to an EHR-integrated population health tool via a brief video and demonstration. They were then asked about how this tool may be useful for CPG adherence, additional clinical characteristics (e.g., BMI) needed for the tool to be useful, and potential ways such a tool could be integrated into routine practice.

Step 4: Defining implementation strategies

Following the generation of candidate strategies to improve pHTN CPG implementation, the SAP operationally defined each discrete strategy. This step was necessary for step 5 activities that involved linking strategies to identified barrier(s), and the later strategy specification in step 6.

Between the sessions comprising steps 4 and 5, the research team created a matrix of barriers by potential strategies the SAP identified through the activities of the first four steps (see Additional file 1). The goal was to elucidate the concordance of strategies with barriers and inform where SAP input was still needed.

Step 5: Review and confirm matrix of barriers and potential strategies

The SAP defined each barrier and indicated which strategies addressed each barrier. They were also encouraged to identify new barriers or new strategies to fill any gaps in the matrix. Before the next meeting, the research team consolidated and optimized the list of identified barriers by collapsing and pruning as conceptually and practically applicable.

Step 6: Specify the strategies in the matrix

Next, the SAP was shown the consolidated list of strategies and asked to specify the actor(s) (who does the strategy), action(s) (what the actors do), temporality (when the strategy was used), and dosage (the frequency and time of each use), per Proctor et al. [35].

Rate strategies and determinants to inform prioritization and final selection

Next, panelists were invited to complete a survey (~30 min). First, they rated each determinant: –2 (strong, negative impact on implementation; i.e., strong barrier), –1, 0 (neutral impact), +1, +2 (strong, positive impact on implementation; i.e., strong facilitator) [36]. Second, panelists completed ratings of each strategy’s perceived effectiveness, feasibility, and priority for their community health center on a scale from 1 (low) to 4 (high) per the ERIC protocol [30].

Using the strategy ratings, the research team used a three-tier approach to prioritization, which largely reflected a natural division in the ratings (described below in the “Data analysis” section). To facilitate the process of prioritization with the SAP, the research team populated the determinants and strategies sections of the IRLM (Fig. 2) and used the matrix of determinants and strategies (created in accordance with ERIC steps 3 and 4 and following step 5) to indicate the relationships between them using superscripts (e.g., the population health tool strategy addresses the determinants of poor follow-up for elevated BP and coordination and consults for specialty care)—a recommended step in using the IRLM [29, 32]. This step helped the SAP assess the degree of coverage the proposed strategies provided for the prominent barriers (step 7).

Fig. 2
figure 2

Implementation Research Logic Model (IRLM) for OpTIMISe strategy selection and prioritization

Full size image

Step 7: Build consensus on the prioritization of strategy package using the IRLM

The IRLM was presented to the SAP with ratings, superscripts, and proposed prioritization of strategies as described above using the three-tier approach. The SAP was instructed to examine the coverage of the primary barriers with the proposed strategies in tier 1. Deficiencies in coverage of barriers in the tier 1 strategies resulted in elevating strategies from tier 2 to tier 1 and adding two new strategies that had not been previously discussed. We repeated the step of specifying these new strategies as done in step 6, but the survey ratings were not repeated as their prioritization (i.e., tier assignment) was clear from discussion during the session. Finally, because the identified determinants to this point were largely barriers, the SAP was asked to identify facilitators; seven were identified and rated through group consensus.

Step 8: Obtain stakeholder buy-in and feedback on project proposal

Approximately 3 months after the meeting to complete step 7, the SAP was convened to reflect on and review the strategies being proposed in a grant application to support an implementation trial (described in the “Discussion” section). The SAP was shown the final IRLM (including mechanisms and outcomes) and the supporting text describing the processes of the SAP, the study approach, and their proposed involvement in the project should it be awarded.

Complementary activities

In addition to the SAP meetings, the research team elicited input from caregivers of children with, or at risk for, pHTN and from clinic staff (i.e., nurses, medical assistants) based on the evolving strategy plan and identified barriers. Caregivers identified many similar determinants of pHTN diagnosis and treatment as the SAP (e.g., concern about elevated BP in their children). Clinic staff confirmed the feasibility and acceptability of all strategies presented to them and provided important details to increase the likelihood of implementation success (e.g., integrating follow-up or booster trainings into pre-existing staff activities, such as “lunch and learn” sessions and team huddles). See Additional file 2 for the full report of the methods and results of these complementary stakeholder activities.

Data analysis

The transcripts from steps 1–4 were analyzed using Rapid Turnaround Qualitative Analysis [37, 38]. Two members of the research team completed two 4-h trainings in Rapid Turnaround Qualitative Analysis for implementation research (conducted by ABH). The first two SAP sessions were double-coded and results were compared and discussed before sessions 3–4 were coded by a single rater. Coding was undertaken to identify determinants and corresponding strategies, in accordance with the five domains of the Consolidated Framework for Implementation Research (CFIR) [39]. Coding was also informed by the recommendations for implementing health information technology (HIT) tools [40].

Descriptive quantitative analyses of the survey, including means, ranges, and relative rankings, were used to rate determinants and prioritize strategies. First, the mean ratings of the determinants were rounded to the nearest whole integer (−2, −1, 0, +1, +2) and determinants were characterized as barriers (mean ratings <0) and facilitators (mean ratings >0) [36]. Second, the mean ratings of the strategies’ feasibility, effectiveness, and prioritization were compiled, and strategies were grouped into three tiers. Tier 1 included strategies that were rated to be highest priority, high effectiveness, and higher feasibility. Tier 2 included strategies that were rated to be moderate priority, moderate effectiveness, and moderate feasibility. Tier 3 included strategies that were rated to be lower priority, moderate effectiveness, and lowest feasibility. Determinations were made for each strategy relative to the others as no clear thresholds or cut points exist for such ratings.

Results

Selection of the implementation strategy

SAP meetings and ratings

The primary goal of the semi-structured SAP meetings was to engage stakeholders in identifying the primary determinants and strategies for implementing the 2017 CPG for pHTN into primary care practices in community health centers. Across activities, 14 determinants were identified. Stakeholder ratings of the strength of each determinant resulted in mean scores ranging from −1.71 (strong barrier) to +.14 (moderate facilitator), with a mean overall score of −.95 (see Table 1 for determinant and their ratings used in the IRLM). This overall mean score is consistent with the sessions being designed to elicit barriers more so than facilitators. Thus, the final SAP meeting included asking panelists to identify facilitators and rate their strengths. Six new facilitators were identified; scores ranged from +1 to +2, with an average score of +1.33 (moderately strong).

Table 1 Stakeholder ratings of determinants used in the logic model
Full size table

Selected implementation strategies

Across the SAP meetings and interviews with clinic staff, 18 discrete strategies were identified, defined, and specified (i.e., actors, actions, temporality, and doses) (see Table 2). While the identified strategies primarily targeted determinants in the CFIR domains of intervention characteristics (e.g., elements of the CPG), inner setting (e.g., equipment, leadership support), and implementer characteristics (e.g., staff awareness of BP value interpretation), there were also strategies addressing barriers in the outer setting and process domains. The strategies primarily involved the practice’s pediatric clinician or healthcare staff as the actors, and actions occurred most often during point-of-care interactions, with the exception of population health strategies.

Table 2 Discrete strategies and specifications identified by stakeholders across interviews and workshops
Full size table

Using the IRLM

The completed IRLM appears in Fig. 2. One of the final steps needed to convert the results of the adapted ERIC process and other activities to the IRLM was to determine the tier of each strategy based on stakeholder reporting priority, effectiveness, and feasibility, as well as alignment with peer-reviewed evidence of their effectiveness [40, 41].

Concerning the ratings, scores (Table 3) ranged from 1 to 4 for priority (M=2.97; SD=0.89), 2 to 4 for effectiveness (M=3.23; SD=0.68), and 1 to 4 for feasibility (M=2.82; SD=0.98). Determinant-strategy links are noted with capitalized superscript letters, and the tiers are reflected by text color (i.e., green: tier 1; yellow: tier 2, red: tier 3), grouped according to ERIC strategy category [30].

Table 3 Stakeholder ratings of strategy effectiveness, feasibility, and priority
Full size table

We then looked to the literature and verified that all of the implementation strategies selected by the SAP, particularly those in tier 1 (i.e., those with the highest priority, effectiveness, and feasibility), were supported as effective strategies to promote CPG adherence generally [28] and for hypertension in particular [42]. For example, training and education of pediatric clinicians and staff is commonly used through distribution of educational materials, group meetings and supervision, and formal training seminars [43, 44]. Our SAP members were also interested in using novel HIT strategies (e.g., EHR reminders, population health tools), which have proven successful in prior studies in this clinic network (e.g., [45]). To support these strategies, we closely followed the recommendations for implementing e-health strategies set forth by Ross et al. [40], which included considering compatibility with existing systems and practices, planning for implementation as well as ongoing monitoring and evaluation, training, and education for those involved, and amassing support from key stakeholders and leaders. Finally, for each of these strategies, we worked closely with the SAP to tailor the implementation strategies for the specific context of the primary care practices in community health clinics, as recommended by Fisher et al. [28].

After verifying the contents with stakeholders for accuracy and completeness, the IRLM was used to finalize the proposed multilevel, multicomponent implementation strategy for adherence to the CPG for pHTN. The two primary factors determining placement in the lower tiers, compared to tier 1, were cost and divergence from current practice, which reflects feasibility concerns. This consideration resulted in many strategies being understood to be minor-moderate adjustments to current practices rather than major changes. This aligns with the premise of this project that aims to improve adherence to a CPG that has many of its elements already as standard of care, but not in the precise manner specified by the CPG.

Discussion

Given the omnipresent research-to-practice gap, many models and frameworks have been proposed to narrow that gap and improve the speed of translation [46]. Several approaches (e.g., community-engaged research, user-centered design) emphasize the importance of stakeholders in the selection and tailoring of implementation strategies [3]. Stakeholders provide critical insights about implementation factors, such as intimate knowledge of workflows, organizational infrastructure, culture, available resources, and other challenges that could inhibit successful implementation. This article presents a generalizable method that expanded the adapted ERIC protocol to meaningfully engage stakeholders in the identification and prioritization of implementation strategies. This method was exemplified through a case example that worked closely with stakeholders to select implementation strategies to support adherence to the CPG for pHTN [47].

We augmented the adapted ERIC protocol used in previous research [29, 30] in two unique ways. First, two user-centered design workshops complemented the information elicited via the semi-structured SAP sessions. Specifically, when stakeholders diagrammed their workflow and interacted with the population health tool, existing determinants and implementation strategies were confirmed, and new ones were elicited. Other user-centered design activities could be used to complement the ERIC protocol. For example, given the restricted range in scores for determinant and strategy ratings obtained via our survey, as well as their conceptual overlap, card sorting may be a useful activity for both priority ranking strategies and categorizing similar determinants and strategies, similar to what was done in the concept mapping stage of ERIC [1]. Our approach has the potential to increase the likelihood of success of the strategies once enacted due to deep stakeholder engagement in the selection process.

A major strength of this method is the sequential, iterative meetings allowed us to first identify and then adapt the determinants and implementation strategies to focus on in subsequent stakeholder meetings. For example, in early meetings, we noticed most of the determinants identified by stakeholders were barriers (cf., facilitators). After analysis of early SAP meetings revealed this pattern, we were able to specifically ask stakeholders to identify facilitators in a subsequent meeting rather than relying on existing literature or other sources for this information, or it being absent. After the SAP meetings, we confirmed the acceptability and feasibility of selected strategies with additional stakeholders who would be involved or impacted by them (i.e., caregivers, clinic staff). This approach differs from typical data collection procedures that are often a fixed endeavor and, even if data collection is prospective, protocols and assessments are less adaptable even as relevant findings accrue.

Second, using the IRLM as an organizing framework and visual aid allowed us to connect determinants to strategies with desired implementation outcomes and prioritize strategies. Connecting determinants to strategies is not only recommended when specifying each discrete strategy [35], but it was also found to be useful in evaluating whether the proposed strategies appropriately addressed the barriers identified—a critical challenge when designing a multilevel, multicomponent implementation strategy for a complex problem like pHTN CPG adherence, and in the field more generally [48, 49]. Additionally, the IRLM enabled pruning and prioritizing strategies using the tiering process as shown in Fig. 2. One of our main goals of this project was to define feasible and effective implementation strategies that could be empirically optimized (e.g., improved over time with multiple iterations) in a subsequent implementation trial. Thus, rather than simply selecting a single multicomponent strategy, we placed discrete strategies in tiers as a way to facilitate the desired optimization approach (described next). Finally, using the IRLM as a final organizing tool led to the SAP generating two additional strategies after viewing the alignment of the strategies with the primary barriers.

Preparation for a subsequent implementation trial

The adapted and expanded ERIC process described here was partially designed to obtain preliminary data for a randomized optimization trial design, the Roll-Out Implementation Optimization (ROIO) design [50]. This design differs from traditional roll-out designs in that the explicit goal is an empirically driven optimization of implementation strategy effectiveness between clusters. The strategy tiering approach was instrumental to preparing for a ROIO trial as the tier 1 strategy package will serve as the first package tested in cluster 1, and the strategies from tier 2 (and tier 3 if needed) will serve as an already-vetted menu of options from which to choose in subsequent clusters. The IRLM furthered this goal because the conceptual connections between determinants, strategies, and outcomes are central to the deliberation process for choosing additional/new strategies, or removing ineffective strategies, in a way that is likely to lead to improvements in specific outcomes. While the adapted and expanded ERIC protocol used for this study was well aligned with the needs of the ROIO design, other optimization designs, adaptive designs, and comparative implementation trial designs [51] could all benefit from such a process in the planning phase. Regardless of the study design, the method described in this article can be used for identification, specification, and prioritization of strategies given that many implementation efforts will require multiple strategies that require buy-in from implementation partners and consideration of feasibility.

Limitations and other considerations

The current project should be considered in light of certain limitations. First, while some stakeholders additionally held clinical leadership and quality improvement positions, the case example did not include executive leadership members (e.g., Chief Medical Officer, Chief Operations Officers), policymakers, or payors, whose perspectives in determining the optimal strategy will be critical for uptake and sustainment of any organizational change. For projects considering using this method, it will be important to include stakeholders across organization levels as is both justifiable and feasible. Furthermore, the SAP members for the exemplar study were passionate about pediatric cardiovascular health and quality improvement, which may be an important factor to consider when forming stakeholder panels in future studies using this method. Second, while not a limitation specific to this project, the short-term mechanism that funded this work allowed for deep stakeholder engagement, a rigorous process spanning nearly a year. Such a timeline for implementation strategy development may not be feasible for all projects wanting to use this method, but would be possible using smaller funding mechanisms (e.g., R03, R21) [52]. Given how critical stakeholder engagement is to the success of implementation research, our field needs to rethink the small amount of time and financial support typically allocated for partnership development and stakeholder engagement prior to implementation trials. Additionally, it is worth noting that the case example’s project length was likely due to the complexity of the guidelines and multilevel, multicomponent strategy package, suggesting the method could be used on a smaller scale and in a shorter time period depending on project scope. Our team, led by JDS, AJK, and AJC, recently completed an abbreviated version comprising individual interviews (approximately 60 minutes) and three 2-hour group meetings of the method described in this paper to similar success for a project focused on implementation of an evidence-based screening and preventive intervention system for toddler social-emotional wellbeing in community-based pediatric practices.

Conclusion

This article presents a generalizable method to meaningfully engage stakeholders in the selection and specification of a multilevel, multicomponent implementation strategy. We illustrated the use of this method in the context of a project focused on pHTN CPG implementation. The procedures used for this project to select an implementation strategy, which involved an adapted ERIC protocol augmented with user-centered design methods and the IRLM, provide a generalizable approach that can be applied to other implementation challenges to improve the likelihood of strategy adoption, effectiveness, and sustainment.

Availability of data and materials

Data and materials are available upon request to the corresponding author.

Abbreviations

BP:

Blood pressure

CFIR:

Consolidated Framework for Implementation Research

CPG:

Clinical practice guideline

EHR:

Electronic health record

ERIC:

Expert Recommendations for Implementing Change

IRLM:

Implementation Research Logic Model

pHTN:

Pediatric hypertension

SAP:

Stakeholder Advisory Panel

HIT:

Health-information technology

References

  1. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21.

    PubMed  PubMed Central  Google Scholar 

  2. Waltz TJ, Powell BJ, Fernández ME, Abadie B, Damschroder LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci. 2019;14(1):42.

    PubMed  PubMed Central  Google Scholar 

  3. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44(2):177–94.

    PubMed  PubMed Central  Google Scholar 

  4. Goodman MS, Sanders Thompson VL. The science of stakeholder engagement in research: classification, implementation, and evaluation. Transl Behav Med. 2017;7(3):486–91.

    PubMed  PubMed Central  Google Scholar 

  5. Knoepke CE, Ingle MP, Matlock DD, Brownson RC, Glasgow RE. Dissemination and stakeholder engagement practices among dissemination & implementation scientists: Results from an online survey. PLoS One. 2019;14(11):e0216971.

    CAS  PubMed  PubMed Central  Google Scholar 

  6. Smith JD, Mohanty N, Davis MM, Knapp AA, Tedla YG, Carroll AJ, et al. Optimizing the implementation of a population panel management intervention in safety-net clinics for pediatric hypertension (The OpTIMISe–Pediatric Hypertension Study). Implement Sci Commun. 2020;1(1):57.

    PubMed  PubMed Central  Google Scholar 

  7. Flynn JT, Kaelber DC, Baker-Smith CM, Blowey D, Carroll AE, Daniels SR, et al. Clinical practice guideline for screening and management of high blood pressure in children and adolescents. Pediatrics. 2017;140(3):e20171904.

    PubMed  Google Scholar 

  8. Children NHBPEPWGoHBPi, Adolescents. The fourth report on the diagnosis, evaluation, and treatment of high blood pressure in children and adolescents. Pediatrics. 2004;114(Supplement_2):555–76.

    Google Scholar 

  9. Program NHBPE. The fourth report on the diagnosis, evaluation, and treatment of high blood pressure in children and adolescents: US Department of Health and Human Services, National Institutes of Health …; 2005.

    Google Scholar 

  10. Hansen ML, Gunn PW, Kaelber DC. Underdiagnosis of hypertension in children and adolescents. JAMA. 2007;298(8):874–9.

    CAS  PubMed  Google Scholar 

  11. Brady TM, Solomon BS, Neu AM, Siberry GK, Parekh RS. Patient-, provider-, and clinic-level predictors of unrecognized elevated blood pressure in children. Pediatrics. 2010;125(6):e1286–e93.

    PubMed  Google Scholar 

  12. Shatat IF, Brady TM. Editorial: Pediatric Hypertension: Update. Front Pediatr. 2018;6:209. https://doi.org/10.3389/fped.2018.00209.

  13. Bell CS, Samuel JP, Samuels JA. Prevalence of hypertension in children. Hypertension. 2019;73(1):148–52.

    CAS  PubMed  Google Scholar 

  14. Din-Dzietham R, Liu Y, Bielo M-V, Shamsa F. High blood pressure trends in children and adolescents in national surveys, 1963 to 2002. Circulation. 2007;116(13):1488–96.

    PubMed  Google Scholar 

  15. Moin A, Mohanty N, Tedla YG, Carroll AJ, Padilla R, Langman CB, et al. Under-recognition of pediatric hypertension diagnosis: examination of 1 year of visits to community health centers. J Clin Hypertens. 2021;23(2):257–64.

    Google Scholar 

  16. Du T, Fernandez C, Barshop R, Chen W, Urbina EM, Bazzano LA. 2017 Pediatric hypertension guidelines improve prediction of adult cardiovascular outcomes. Hypertension. 2019;73(6):1217–23.

    CAS  PubMed  Google Scholar 

  17. Urbina EM, Mendizábal B, Becker RC, Daniels SR, Falkner BE, Hamdani G, et al. Association of blood pressure level with left ventricular mass in adolescents. Hypertension. 2019;74(3):590–6.

    CAS  PubMed  Google Scholar 

  18. Lande MB, Batisky DL, Kupferman JC, Samuels J, Hooper SR, Falkner B, et al. Neurocognitive function in children with primary hypertension. J Pediatr. 2017;180:148–55.e1.

    PubMed  Google Scholar 

  19. Urbina EM, Khoury PR, McCoy C, Daniels SR, Kimball TR, Dolan LM. Cardiac and vascular consequences of pre-hypertension in youth. J Clin Hypertens (Greenwich, Conn). 2011;13(5):332–42.

    Google Scholar 

  20. Khoury M, Urbina EM. Cardiac and vascular target organ damage in pediatric hypertension. Front Pediatr. 2018;6:148.

    PubMed  PubMed Central  Google Scholar 

  21. Rao G. Diagnosis, epidemiology, and management of hypertension in children. Pediatrics. 2016;138(2):e20153616.

    PubMed  Google Scholar 

  22. Bello JK, Mohanty N, Bauer V, Rittner SS, Rao G. Pediatric hypertension: provider perspectives. Glob Pediatr. Health. 2017;4:2333794x17712637.

    Google Scholar 

  23. AlAbdulKader AM, Morse EF, Daley MF, Rao G. Pediatric hypertension: parent perspectives. Global. Pediatr Health. 2020;7:2333794X20981340.

    Google Scholar 

  24. Montgomery AA, Fahey T, Peters TJ, MacIntosh C, Sharp DJ. Evaluation of computer based clinical decision support system and risk chart for management of hypertension in primary care: randomised controlled trial. BMJ (Clinical research ed). 2000;320(7236):686–90.

    CAS  Google Scholar 

  25. Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38.

    CAS  PubMed  Google Scholar 

  26. Carroll AJ, Tedla YG, Mohanty N, Padilla R, Langman CB, Smith JD. Correlates of adherence to the 2017 clinical practice guidelines for pediatric hypertension in safety-net clinics: a two-year cross-sectional study. submitted for publication.

  27. Vuppala S, Turer CB. Clinical decision support for the diagnosis and management of adult and pediatric hypertension. Curr Hypertens Rep. 2020;22(9):67.

    PubMed  PubMed Central  Google Scholar 

  28. Fischer F, Lange K, Klose K, Greiner W, Kraemer A. Barriers and strategies in guideline implementation-a scoping review. Healthcare (Basel). 2016;4(3):36.

    Google Scholar 

  29. Go VF, Morales GJ, Mai NT, Brownson RC, Ha TV, Miller WC. Finding what works: identification of implementation strategies for the integration of methadone maintenance therapy and HIV services in Vietnam. Implement Sci. 2016;11(1):54.

    PubMed  PubMed Central  Google Scholar 

  30. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10(1):109.

    PubMed  PubMed Central  Google Scholar 

  31. Dopp AR, Parisi KE, Munson SA, Lyon AR. A glossary of user-centered design strategies for implementation experts. Transl Behav Med. 2018;9(6):1057–64.

    Google Scholar 

  32. Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15(1):84.

    PubMed  PubMed Central  Google Scholar 

  33. Zoom Video Communications Inc. Security guide: Zoom Video Communications Inc.; 2016.

    Google Scholar 

  34. Perry CK, Damschroder LJ, Hemler JR, Woodson TT, Ono SS, Cohen DJ. Specifying and comparing implementation strategies across seven large implementation interventions: a practical application of theory. Implement Sci. 2019;14(1):32.

    PubMed  PubMed Central  Google Scholar 

  35. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8(1):139.

    PubMed  PubMed Central  Google Scholar 

  36. Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013;8(1):51.

    PubMed  PubMed Central  Google Scholar 

  37. Neal JW, Neal ZP, VanDyke E, Kornbluh M. Expediting the analysis of qualitative data in evaluation: a procedure for the rapid identification of themes from audio recordings (RITA). Am J Eval. 2015;36(1):118–32.

    Google Scholar 

  38. Hamilton AB. Qualitative methods in rapid turn-around health services research. VA Women's Health Res Netw. 2013.

  39. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.

    PubMed  PubMed Central  Google Scholar 

  40. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016;11(1):146.

    PubMed  PubMed Central  Google Scholar 

  41. Miech EJ, Rattray NA, Flanagan ME, Damschroder L, Schmid AA, Damush TM. Inside help: an integrative review of champions in healthcare-related implementation. SAGE Open Med. 2018;6:2050312118773261.

    PubMed  PubMed Central  Google Scholar 

  42. Handler J, Lackland DT. Translation of hypertension treatment guidelines into practice: a review of implementation. J Am Soc Hypertens. 2011;5(4):197–207.

    PubMed  Google Scholar 

  43. Spoon D, Rietbergen T, Huis A, Heinen M, van Dijk M, van Bodegom-Vos L, et al. Implementation strategies used to implement nursing guidelines in daily practice: a systematic review. Int J Nurs Stud. 2020;111:103748.

    PubMed  Google Scholar 

  44. Ebben RHA, Siqeca F, Madsen UR, Vloet LCM, van Achterberg T. Effectiveness of implementation strategies for the improvement of guideline and protocol adherence in emergency care: a systematic review. BMJ Open. 2018;8(11):e017572.

    PubMed  PubMed Central  Google Scholar 

  45. Mohanty N, Goutham R, Naureckas S, Bello J, Padilla R, Tanni A, et al. Leveraging HIT for evidence-based pediatric blood pressure management. Pediatrics. 2018;142(1 MeetingAbstract):621.

    Google Scholar 

  46. Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50.

    PubMed  PubMed Central  Google Scholar 

  47. Thompson M, Dana T, Bougatsos C, Blazina I, Norris SL. Screening for hypertension in children and adolescents to prevent cardiovascular disease. Pediatrics. 2013;131(3):490–525.

    PubMed  Google Scholar 

  48. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1(1):1.

    PubMed Central  Google Scholar 

  49. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7(3).

  50. Smith JD, Brown CH. The roll-out implementation optimization (ROIO) design: rigorous testing of a data-driven implementation improvement aim. Washington, DC: AcademyHealth/NIH 13th Annual Conference on the Science of Dissemination and Implementation in Health; 2020. virtual

    Google Scholar 

  51. Hwang S, Birken SA, Melvin CL, Rohweder CL, Smith JD. Designs and methods for implementation research: advancing the mission of the CTSA program. J Clin Transl Sci. 2020;4(3):159–67.

    PubMed  PubMed Central  Google Scholar 

  52. Department of Health and Human Services. PAR-19-274: Dissemination and implementation research in health (R01 Clinical Trial Optional) Washington, DC; 2019. Available from: https://grants.nih.gov/grants/guide/pa-files/PAR-19-274.html

    Google Scholar 

Download references

Acknowledgements

The authors wish to thank our community health center partners and the coordinators of the Stakeholder-Academic Resource Panels (ShARPs) within the Center for Community Health at Northwestern University.

Funding

This study is supported by grant R56HL148192 from the National Heart, Lung, and Blood Institute of the National Institutes of Health, awarded to Justin D. Smith. Additional support was provided by grant P30DA027828 from the National Institute on Drug Abuse (awarded to C. Hendricks Brown), grant 1K01MH121854 from the National Institute of Mental Health (awarded to Ashley A. Knapp), grant UL1TR001422 from the National Center for Advancing Translational Sciences, grant QUE 20-072 from VA Quality Enhancement Research Initiative (awarded to Alison B. Hamilton), grant U01HL142109 from the National Heart, Lung, and Blood Institute (awarded to Alison B. Hamilton). The opinions expressed herein are the views of the authors and do not necessarily reflect the official policy or position of the National Institutes of Health or any other part of the US Department of Health and Human Services.

Author information

Author notes

  1. Ashley A. Knapp and Allison J. Carroll contributed equally as co-first authors.

Affiliations

  1. Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Ashley A. Knapp, Nivedita Mohanty, Emily Fu, Juan A. Villamar & Neil Jordan

  2. Department of Psychiatry and Behavioral Sciences and Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Allison J. Carroll

  3. Alliance Chicago, Chicago, IL, USA

    Nivedita Mohanty & Roxane Padilla

  4. Center for Mental Health Services Research, Brown School & School of Medicine, Washington University in St. Louis, St. Louis, MO, USA

    Byron J. Powell

  5. VA Center for the Study of Healthcare Innovation, Implementation & Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, USA

    Alison Hamilton

  6. Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA

    Alison Hamilton

  7. Near North Health, Chicago, IL, USA

    Nicole D. Burton

  8. Erie Family Health Centers, Chicago, IL, USA

    Elaine Coldren, Dhanya P. Limaye & Daniel Mendoza

  9. Heartland Health Centers, Chicago, IL, USA

    Tania Hossain

  10. Lawndale Christian Health Center, Chicago, IL, USA

    Michael Sethi

  11. Stanley Manne Children’s Research Institute, Ann & Robert H. Lurie Children’s Hospital of Chicago and Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Heather E. Price

  12. Center of Innovation for Complex Chronic Healthcare, Hines VA Hospital, Hines, IL, USA

    Neil Jordan

  13. Ann & Robert H. Lurie Children’s Hospital of Chicago and Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Craig B. Langman

  14. Department of Population Health Sciences, Spencer Fox Eccles School of Medicine at the University of Utah, Salt Lake City, UT, USA

    Justin D. Smith

Authors
  1. Ashley A. Knapp
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allison J. Carroll
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nivedita Mohanty
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emily Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Byron J. Powell
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alison Hamilton
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nicole D. Burton
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Elaine Coldren
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Tania Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Dhanya P. Limaye
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Daniel Mendoza
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Michael Sethi
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Roxane Padilla
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Heather E. Price
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Juan A. Villamar
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Neil Jordan
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Craig B. Langman
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Justin D. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JDS, NM, and CBL conceived of the overall project. AAK, AJC, and JDS conceived of this study. AAK, JDS, and BJP devised the adaptations to the Expert Recommendations for Implementing Change (ERIC) protocol to integrate the Implementation Research Logic Model and user-centered design activities. AAK, AJC, and JDS collaborated in drafting the manuscript. EF created the tables for the manuscript. AAK, AJC, JDS, NM, CBL, NDB, EC, TH, DPL, DM, MS, RP, HEP, JAV, AM, and NJ participated in all or part of the stakeholder advisory panel (SAP) meetings. AAK and AJC led the rapid qualitative analysis with initial training and guidance in the methods from AH. All authors reviewed the manuscript and approved the final version.

Corresponding author

Correspondence to Justin D. Smith.

Ethics declarations

Ethics approval and consent to participate

The study was designed in accordance with the basic ethical principles of autonomy, beneficence, justice, and non-maleficence and was conducted in accordance with the rules of Good Clinical Practice outlined in the most recent Declaration of Helsinki. Written informed consent of participants was obtained. Data confidentiality and anonymity were ensured according to the provisions of US law. The project was approved by the Institutional Review Board of Northwestern University on March 23, 2020 (Protocol STU00210809). All other institutions participating in this research provided signed reliance agreements ceding to the Institutional Review Board of Northwestern University.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Determinants—Strategies Matrix.

Additional file 2.

Complementary Activities Resulting from Additional Stakeholder Involvement.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Knapp, A.A., Carroll, A.J., Mohanty, N. et al. A stakeholder-driven method for selecting implementation strategies: a case example of pediatric hypertension clinical practice guideline implementation. Implement Sci Commun 3, 25 (2022). https://doi.org/10.1186/s43058-022-00276-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-022-00276-4

Keywords

  • Expert Recommendations for Implementing Change
  • Implementation Research Logic Model
  • User-centered design
  • Implementation strategy
  • Pediatric hypertension
  • Stakeholder engagement

Implementation Science Communications

ISSN: 2662-2211