Participants
Academic-community partnership
This study is grounded in an academic-community partnership that is the result of longstanding collaborations between Northwestern University Feinberg School of Medicine, Lurie Children’s Hospital of Chicago, and AllianceChicago, an AHRQ-recognized Practice Based Research Network comprising 60 community health centers with more than 200 clinic sites in 19 states as of 2021.
Stakeholder Advisory Panel (SAP)
The SAP comprised pediatric healthcare clinicians and research team members. Pediatric clinicians (n=6) were pediatric or family medicine physicians that would represent the perspective of community health center-based primary care practices that intended to participate in a subsequent implementation trial. The research team members (n=5) who led and participated in the meetings had expertise in pHTN diagnosis and treatment, implementation science focused on chronic disease management, user-centered design, pediatric primary care, health disparities, and use of HIT to support CPG adherence. Pediatric clinicians were recruited from four community health center organizations in the Chicago area that routinely collaborate in practice transformation initiatives using AllianceChicago’s HIT and practice change infrastructure.
Procedures and case example
We used a pragmatic adaptation and expansion of the ERIC protocol (see Fig. 1 for alignment of study activities with the steps of the ERIC protocol) to engage stakeholders in identifying, specifying, and prioritizing implementation strategies [1, 29, 30]. ERIC involves an iterative, multi-method process of qualitative analysis of semi-structured stakeholder meetings, as detailed below. We expanded on the adapted ERIC process by (a) incorporating user-centered design methods [31] to understand determinants and identify strategies related to the assessment and management of pHTN and (b) using the IRLM [32] as a conceptual and organizing framework. SAP meetings were held monthly for 7 months (April–October 2020), and once in January 2021, and lasted 1–2 h each. SAP members spent an average of 12 h in SAP meetings and related activities (e.g., surveys). Meetings occurred via Zoom videoconferencing platform [33], recorded with panelists’ permission, and analyzed by the research team. SAP members were compensated $150 per hour. For replication and generalizability purposes, we now discuss the method by outlining the stakeholder-engaged activities that resulted in the multilevel, multicomponent implementation strategy.
Step 1: Identifying barriers to adhering to the CPG for pHTN
SAP members were introduced to the project, meeting logistics, and project specifics by reviewing the study protocol [6]. The SAP then engaged in a semi-structured discussion of current practices in their respective clinics for measuring, diagnosing, and managing BP in children and adolescents, as well as identifying the barriers to adhering to the 2017 CPG for pHTN.
Steps 2 and 3: Understanding context and generating implementation strategies
Stakeholders participated in two user-centered design workshops [31]. First, they were asked to diagram and discuss their workflows for BP measurement, including (a) the pre-encounter vitals, (b) the clinician-patient encounter, and (c) the end-of-visit and follow-up plan. The research team prompted for barriers; communication channels between clinicians, staff, and families about BP results and treatment plans; and recommendations for strategies to overcome named barriers. Although this method was informed by the user-centered design literature [31], assessment and redesign of the workflow was recently suggested as an additional ERIC strategy [34]. Second, stakeholders were introduced to an EHR-integrated population health tool via a brief video and demonstration. They were then asked about how this tool may be useful for CPG adherence, additional clinical characteristics (e.g., BMI) needed for the tool to be useful, and potential ways such a tool could be integrated into routine practice.
Step 4: Defining implementation strategies
Following the generation of candidate strategies to improve pHTN CPG implementation, the SAP operationally defined each discrete strategy. This step was necessary for step 5 activities that involved linking strategies to identified barrier(s), and the later strategy specification in step 6.
Between the sessions comprising steps 4 and 5, the research team created a matrix of barriers by potential strategies the SAP identified through the activities of the first four steps (see Additional file 1). The goal was to elucidate the concordance of strategies with barriers and inform where SAP input was still needed.
Step 5: Review and confirm matrix of barriers and potential strategies
The SAP defined each barrier and indicated which strategies addressed each barrier. They were also encouraged to identify new barriers or new strategies to fill any gaps in the matrix. Before the next meeting, the research team consolidated and optimized the list of identified barriers by collapsing and pruning as conceptually and practically applicable.
Step 6: Specify the strategies in the matrix
Next, the SAP was shown the consolidated list of strategies and asked to specify the actor(s) (who does the strategy), action(s) (what the actors do), temporality (when the strategy was used), and dosage (the frequency and time of each use), per Proctor et al. [35].
Rate strategies and determinants to inform prioritization and final selection
Next, panelists were invited to complete a survey (~30 min). First, they rated each determinant: –2 (strong, negative impact on implementation; i.e., strong barrier), –1, 0 (neutral impact), +1, +2 (strong, positive impact on implementation; i.e., strong facilitator) [36]. Second, panelists completed ratings of each strategy’s perceived effectiveness, feasibility, and priority for their community health center on a scale from 1 (low) to 4 (high) per the ERIC protocol [30].
Using the strategy ratings, the research team used a three-tier approach to prioritization, which largely reflected a natural division in the ratings (described below in the “Data analysis” section). To facilitate the process of prioritization with the SAP, the research team populated the determinants and strategies sections of the IRLM (Fig. 2) and used the matrix of determinants and strategies (created in accordance with ERIC steps 3 and 4 and following step 5) to indicate the relationships between them using superscripts (e.g., the population health tool strategy addresses the determinants of poor follow-up for elevated BP and coordination and consults for specialty care)—a recommended step in using the IRLM [29, 32]. This step helped the SAP assess the degree of coverage the proposed strategies provided for the prominent barriers (step 7).
Step 7: Build consensus on the prioritization of strategy package using the IRLM
The IRLM was presented to the SAP with ratings, superscripts, and proposed prioritization of strategies as described above using the three-tier approach. The SAP was instructed to examine the coverage of the primary barriers with the proposed strategies in tier 1. Deficiencies in coverage of barriers in the tier 1 strategies resulted in elevating strategies from tier 2 to tier 1 and adding two new strategies that had not been previously discussed. We repeated the step of specifying these new strategies as done in step 6, but the survey ratings were not repeated as their prioritization (i.e., tier assignment) was clear from discussion during the session. Finally, because the identified determinants to this point were largely barriers, the SAP was asked to identify facilitators; seven were identified and rated through group consensus.
Step 8: Obtain stakeholder buy-in and feedback on project proposal
Approximately 3 months after the meeting to complete step 7, the SAP was convened to reflect on and review the strategies being proposed in a grant application to support an implementation trial (described in the “Discussion” section). The SAP was shown the final IRLM (including mechanisms and outcomes) and the supporting text describing the processes of the SAP, the study approach, and their proposed involvement in the project should it be awarded.
Complementary activities
In addition to the SAP meetings, the research team elicited input from caregivers of children with, or at risk for, pHTN and from clinic staff (i.e., nurses, medical assistants) based on the evolving strategy plan and identified barriers. Caregivers identified many similar determinants of pHTN diagnosis and treatment as the SAP (e.g., concern about elevated BP in their children). Clinic staff confirmed the feasibility and acceptability of all strategies presented to them and provided important details to increase the likelihood of implementation success (e.g., integrating follow-up or booster trainings into pre-existing staff activities, such as “lunch and learn” sessions and team huddles). See Additional file 2 for the full report of the methods and results of these complementary stakeholder activities.
Data analysis
The transcripts from steps 1–4 were analyzed using Rapid Turnaround Qualitative Analysis [37, 38]. Two members of the research team completed two 4-h trainings in Rapid Turnaround Qualitative Analysis for implementation research (conducted by ABH). The first two SAP sessions were double-coded and results were compared and discussed before sessions 3–4 were coded by a single rater. Coding was undertaken to identify determinants and corresponding strategies, in accordance with the five domains of the Consolidated Framework for Implementation Research (CFIR) [39]. Coding was also informed by the recommendations for implementing health information technology (HIT) tools [40].
Descriptive quantitative analyses of the survey, including means, ranges, and relative rankings, were used to rate determinants and prioritize strategies. First, the mean ratings of the determinants were rounded to the nearest whole integer (−2, −1, 0, +1, +2) and determinants were characterized as barriers (mean ratings <0) and facilitators (mean ratings >0) [36]. Second, the mean ratings of the strategies’ feasibility, effectiveness, and prioritization were compiled, and strategies were grouped into three tiers. Tier 1 included strategies that were rated to be highest priority, high effectiveness, and higher feasibility. Tier 2 included strategies that were rated to be moderate priority, moderate effectiveness, and moderate feasibility. Tier 3 included strategies that were rated to be lower priority, moderate effectiveness, and lowest feasibility. Determinations were made for each strategy relative to the others as no clear thresholds or cut points exist for such ratings.