Skip to main content
Download PDF
Download ePub

Interactive and Participatory Audit and Feedback (IPAF): theory-based development and multi-site implementation outcomes with specialty clinic staff

Implementation Science Communications volume 2, Article number: 58 (2021) Cite this article

Abstract

Background

Theory-based implementation strategies, such as audit and feedback (A&F), can improve the adoption of evidence-based practices. However, few strategies have been developed and tested to meet the needs of specialty clinics. In particular, frontline staff can execute cardiovascular disease (CVD) risk reduction protocols, but A&F strategies to support them are not well examined. Our objective was to develop and evaluate a theory-based approach to A&F, Interactive and Participatory A&F (IPAF).

Methods

We developed IPAF informed by two complementary theories, self-regulation theory (SRT) and self-determination theory (SDT). IPAF applies concepts from these theories to inform (1) what to address with staff to improve rates of best practices (SRT) and (2) how to interact with staff to improve behaviors aligned with best practices (SDT). We promoted IPAF fidelity by developing a semi-structured guide to facilitate staff discussion of target behaviors, perceived barriers, goals, and action plans. We evaluated IPAF in the context of eight quasi-experimental implementations in specialty clinics across two health systems. Following a hybrid type 2 effectiveness-implementation design, we reported intervention outcomes for CVD risk reduction elsewhere. This paper reports implementation outcomes associated with IPAF, focusing on feasibility, appropriateness, acceptability, fidelity, and adoption. We evaluated implementation outcomes using mixed-methods data including electronic health record (EHR) data, team records, and staff questionnaire responses.

Results

Eighteen staff participated in 99 monthly, individual, synchronous (face-to-face or phone) IPAF sessions during the first 6 months of implementation. Subsequently, we provided over 375 monthly feedback emails. Feasibility data revealed high staff attendance (90–93%) and engagement in IPAF sessions. Staff highly rated questionnaire items about IPAF acceptability. Team records and staff responses demonstrated fidelity of IPAF delivery and receipt. Adoption of target behaviors increased significantly (all P values < 0.05), and adoption or behaviors were maintained for over 24 months.

Conclusions

We developed and evaluated a theory-based approach to A&F with frontline staff in specialty clinics to improve the implementation of evidence-based interventions. The findings support feasibility, appropriateness, acceptability, and fidelity of IPAF, and staff adoption and maintenance of target behaviors. By evaluating multi-site implementation outcomes, we extended prior research on clinic protocols and A&F beyond primary care settings and providers.

Peer Review reports

Background

Audit and feedback (A&F) is an established implementation strategy [1] to improve uptake of evidence-based practices, but its effectiveness remains variable across studies, with effect sizes often stagnating since 2003 [2, 3]. Implementation science experts argue that applying theory to design, measure, and report implementation strategies can strengthen implementations [2, 4,5,6,7]. Yet, in the most recent Cochrane review of 140 A&F trials to improve professional practice and patient outcomes, only 14% of studies (n = 20) used theory to inform A&F and only 9% (n = 13) reported the details of how the theory was applied [5]. In addition, key characteristics of how A&F strategies were designed and used are often missing or inconsistently addressed [4, 5]. Using theories can address key characteristics previously identified to explain the varied effect sizes in prior studies, including the format and frequency of feedback, the role of persons giving feedback, the type of interpersonal interaction, transparency of desired clinical outcomes toward which implementation efforts are addressed, and whether instructions are provided about specific future goals and action plans [4, 5]. Implementation science could benefit from explicit descriptions of theories applied to implementation strategies in general and to A&F, in particular, to improve efficacy and replicability by leveraging existing knowledge on behavior change. Important details include the rationale for theory selection, details of how concepts are applied, and explication of how key characteristics of A&F correspond to the theory. With greater clarity and specificity about the application of theoretical concepts to A&F, knowledge in the field can be built more efficiently, with the intent to increase effect sizes.

A&F can promote guideline-concordant care by improving how health care professionals follow clinical practice protocols but has primarily been studied with physicians, nurses, and pharmacists [5]. Evidence-based protocols can be executed by frontline staff such as medical assistants. This is well illustrated in the area of cardiovascular disease (CVD) prevention, where clinic protocols executed by medical assistants have improved the proportion of primary care clinic patients whose blood pressure (BP) is controlled from 50 to > 80% [8]. In specialty clinics, however, clinicians discuss CVD risk factors such as BP or tobacco in only 10% of relevant specialty visits, despite routinely assessing them [9, 10]. Non-vascular specialty clinics have not implemented protocols to address high BP and tobacco use, which are the most prevalent risk factors for CVD in US adults. In particular, although many rheumatology populations face increased inflammatory CVD risks [11], rheumatologists typically consider addressing CVD risk factors to be outside their scope of practice. Because frontline staff routinely assess risk factors such as BP and tobacco, integrating evidence-based protocols could promote routine action to address immediate care and follow-up needs. If specialty clinics employed CVD risk reduction protocols, then they could reach many more of the two million specialty patients who have CVD events annually, recognizing that specialty visits are nearly equal to primary care visits in the USA [12]. The target behaviors for our specialty clinic protocol to align with CVD guidelines are for specialty frontline staff to Check a CVD risk factor (e.g., blood pressure verification or smoking and readiness to quit), Advise the patient on the risk factor’s relation to their condition (e.g., rheumatoid arthritis), and Connect the patient to risk reduction resources (e.g., primary care or quit line) [13, 14]. We reported elsewhere the intervention outcomes of Check, Advise, and Connect clinic protocols for timely patient follow-up in primary care after high BPs and referrals to the tobacco quit line after assessing readiness to quit [13,14,15,16].

Our objective was to develop, operationalize, and evaluate a theory-based A&F strategy. Specifically, we aimed to:

  1. 1)

    Develop a theory-based A&F strategy based on behavioral and motivational theories

  2. 2)

    Operationalize the proposed strategy with a tool to guide A&F session fidelity

  3. 3)

    Evaluate the proposed A&F strategy in the context of eight quasi-experimental implementations of evidence-based interventions to reduce CVD risk

We named the proposed strategy Interactive and Participatory A&F (IPAF). We describe the proposed strategy as interactive because rather than only offering unidirectional feedback, the facilitator both requests feedback from the staff about their barriers to engaging in the target behaviors and offers feedback to staff on their recent behaviors. Then, the facilitator and staff discuss and agree on short-term goals and action plans to improve target behaviors to align with the intervention goals (e.g., blood pressure control, smoking cessation). We further describe the proposed strategy as participatory because the facilitator invites the staff to make their own choices: choosing whether to give or receive feedback first, setting their own short-term goals to focus on, and deciding their own action plans for overcoming their barriers and achieving their goals.

We operationalized the proposed strategy with a tool consisting of a semi-structured guide that the facilitator used during A&F sessions to complete all the steps of IPAF. The tool had six sections: introduction, clarifying the purpose of the meeting, offering choice, obtaining feedback from staff, sharing feedback with staff and setting goals, and planning action steps.

Our evaluation of IPAF focused on the implementation outcomes of feasibility, appropriateness, acceptability, fidelity, and adoption [17]. By reporting on the application of IPAF in eight implementations as an exemplar, this paper describes the methodology to enable others to provide theory-based, interactive, and participatory feedback to promote target behaviors that health professionals need to perform to align with evidence-based recommendations. In this evaluation, the target behaviors were for specialty frontline staff to check a CVD risk factor, advise the patient on the risk factor’s relation to their specialty condition, and connect the patient to risk reduction resources.

Methods

Theory-based development of IPAF

Consistent with calls to apply theory to the development of interventions and implementation strategies [5, 6, 18], we applied theory to develop a theory-based A&F strategy and guide relevant evaluation measures. We selected the self-regulation theory (SRT) and self-determination theory (SDT) [19, 20], which are middle-range theories and are readily applied in practice [21]. Our choice of SRT was guided by evidence from, and recommendations in, a Cochrane review of A&F, and SDT has received empirical support from prior research with adult participants in a variety of worksites [19, 21]. SRT concepts guided us in identifying what components to address for behavior change (e.g., referent points for intervention outcomes). However, SRT concepts were not sufficient to inform us on how to create social environments in which staff would be likely to adopt. Because SDT can inform how to interact with staff to improve their behaviors aligned with new initiatives, we chose to apply SDT concepts also. Both theories offer clarity, utility, and parsimony for behavior change with adults [19, 20]. The following two sections describe how SRT and SDT theories guided our A&F development and evaluation in a complementary manner.

How self-regulation theory guided IPAF development

We applied SRT to the development of IPAF as an implementation strategy, consistent with recommendations by experts in SRT [20, 22] and A&F [4, 5]. SRT involves a process of how people regulate their behaviors toward an adopted future state or goal. The concept of an ideal state (e.g., behavior) functions as a referent point for those involved. Importantly, SRT includes a hierarchy of corresponding or aligned goals from more abstract to more specific [20, 22]. Organizational leaders and individual staff can compare current rates of behaviors against the same goal (e.g., to have 60% of the patient population vaccinated). The consequences of engaging in comparisons between real and ideal states result in conclusions of either congruence or discrepancy between states. Discrepancy typically is motivational for people to regulate their own behaviors so that they may observe more congruence alignment between their future behaviors and ideal states [6, 20, 22].

As an illustration, assume an intervention goal is to increase rates of BP re-measurements to 80% (an ideal) among patients whose initial BPs were high. This goal provides a reference point or standard against which clinicians can compare staff’s rates of repeat BPs in an audit (Fig. 1). In the context of our applications, if staff receive feedback that their behaviors are inconsistent with intervention goals that they agreed to support, then they likely would experience discrepancy. As a result, they would be motivated to improve their target behaviors, have future behaviors align with intervention goals, and experience less discrepancy in the future.

Fig. 1
figure 1

Steps based on the concepts in self-regulation theory and applied to IPAF in the context of CVD risk reduction. Starting at step 1 (top left), a person acquires new data. At step 2, compares new data to a pre-determined and desired reference point. Then decides (3) whether or not data are congruent with the reference point. If yes, record and stop (4a). If discrepant with the reference point (4b), take action (5) for improvement so future data would be congruent with the reference point. These steps apply for clinic staff comparing their performance to desired performance goals or patients targeting health goals

Full size image

Consistent with SRT, specific behavior change techniques have been useful for adopting new behaviors. Techniques include barrier identification, goal setting, and action planning [23], as shown in Table 1. These techniques have been associated with desired behavior change across different types of behaviors [23, 24] and proposed for application in A&F in particular [3, 5]. These behavior change techniques are relevant to our staff’s processes of monitoring and adopting target behaviors.

Table 1 Theoretical rationale for elements of IPAF and corresponding guide
Full size table

We also applied SRT broadly to the interventions and overall implementation approach, as shown in Fig. 1. Using SRT, researchers have described, explained, and predicted how people manage (i.e., regulate) themselves to reach their goals over time. At the overarching level of the clinic protocol, SRT applies to our choosing reference points to control CVD among the patient population, that is, high BP or readiness to quit tobacco. At the staff and patient levels, SRT applies to individuals; people need clear reference points against which to evaluate health risk factors and decide whether they are present or not. In summary, SRT involves what to address to improve behavior.

How self-determination theory guided IPAF development

We applied SDT, a theory regarding motivation and behavior [19, 25, 26], to guide the development and evaluation of IPAF, as shown in Table 1. According to SDT, all people have three inherent psychological needs relevant to behavior: relatedness, autonomy (i.e., choice), and perceived competence (i.e., self-efficacy) [19, 26]. When these needs are met, people are more motivated to engage in relevant behaviors [19, 26]. Randomized, controlled studies, based on SDT, have demonstrated improvements in work behaviors [26,27,28]. If the staff’s psychological needs were met, then the staff would be more motivated to adopt target behaviors than if their psychological needs were ignored or thwarted. An underlying proposition is that the staff are more likely to be motivated to align their behaviors toward altruistic goals (e.g., intervention goals of reducing CVD risk) when they experience autonomous rather than controlled environments. Accordingly, we designed IPAF to meet the staff’s psychological needs during individual feedback sessions. In summary, SDT involved how to interact with the staff to achieve both their target behaviors and intervention goals.

Application of IPAF as an implementation strategy

Design

We evaluated implementation outcomes associated with IPAF in the context of a broader pre-, post-, quasi-experimental evaluation of CVD risk reduction interventions [13]. That evaluation followed a hybrid type 2 effectiveness-implementation design [29], attending to both intervention and implementation outcomes. We reported elsewhere the intervention outcomes: (a) timely patient follow-up in primary care after high BPs and (b) referrals to the tobacco quit line after assessing readiness to quit [13,14,15,16]. In this paper, we report on evaluations of the implementation outcomes of feasibility, appropriateness, acceptability, fidelity, and adoption [17] focusing on measures relevant to IPAF. We delivered IPAF sessions to individuals synchronously (in-person or by phone) for 6 months after the beginning of implementation and, later, asynchronously by email for over 24 months. Table 2 shows the components of the CVD risk reduction interventions and the implementation package within which IPAF was used to provide feedback.

Table 2 Intervention and implementation components: context of IPAF application
Full size table

Setting and sample

We evaluated IPAF in eight implementations, representing two interventions (BP Connect and Quit Connect) [13, 16, 30,31,32] in four separate rheumatology clinics in two US health systems. Clinics A, B, and C were in a large, suburban, academic, multi-specialty practice; clinic D was a community clinic. Rheumatology clinics offer an ideal setting and specialty population to evaluate A&F with frontline staff as a strategy to implement CVD risk reduction [8, 33, 34]. Our IPAF participants were all medical assistants and nurses who performed pre-visit rooming (i.e., vital signs, patient history) at the clinics. We collected mixed-methods data including their responses to questionnaires, EHR data, and team records.

Context

Intervention components

The components of our interventions and implementation package are shown in Table 2. We sought to improve the staff’s target behaviors with two CVD risk reduction interventions: BP Connect for high BP [13, 30] and Quit Connect for tobacco use [16, 31]. With a Check-Advise-Connect structure for both interventions, the target behaviors were to check for addressable risk factors, confirming high BPs or readiness to quit tobacco; advise patients on CVD risk; and connect patients to relevant resources. Connecting consisted of offering follow-up arrangements for BP appointments with primary care or for quit line phone calls for tobacco cessation counseling.

Implementation components

Our multi-faceted implementation package consisted of four implementation components as described in Table 2: Engage, Educate, Remind, and Feedback. This paper focuses on the Feedback component, which is the application of IPAF that we describe in detail below (Fig. 2). We engaged the staff in tailoring focus groups and reminded the staff using EHR alerts to check/advise and connect via EHR referral orders as detailed in our toolkits https://www.hipxchange.org/AuditFeedback [35]. For the Educate component, we held 1-h educational sessions with the staff in small groups at the beginning of each implementation. We explained the rationale, principles, and components for the intervention and implementation. We shared relevant evidence to address BP and tobacco, encouraged interactive discussion, and provided scenarios for staff role-plays regarding the Check-Advise-Connect behaviors. These staff had not previously had responsibilities for confirming or addressing CVD risk factors with patients (i.e., BP level or readiness to quit tobacco) or for referring patients to resources (i.e., primary care or quit line). The interactive educational sessions concluded with staff demonstrating mastery of role-play dialog and navigation of the EHR (from the Remind component) and receiving information about the monthly, individual feedback they would receive, according to IPAF.

Fig. 2
figure 2

Logic model of the evaluation of IPAF as an implementation strategy

Full size image

Interactive and Participatory Audit and Feedback

IPAF sessions consisted of three theory-based components:

  1. 1.

    Providing feedback to individual staff about their actual rates of target behaviors, directing their attention to ideal target behaviors which would align with the intervention, and stimulating comparisons between their behavioral rate and the ideal, resulting in discrepancy or congruence based on SRT

  2. 2.

    Interactive, one-on-one discussions of the staff’s experiences, including barriers and goals for target behaviors, while simultaneously supporting the staff’s psychological needs, based on SDT and SRT

  3. 3.

    Eliciting action plans with the staff about how they could improve rates of target behaviors, based on SRT, SDT, and evidence for behavior change techniques [23]

Consistent with best practices from the most recent Cochrane review of A&F [5], we delivered feedback monthly, individually, and face-to-face when possible, by a respected colleague (not a supervisor), to improve the staff’s target behaviors. The source of feedback (i.e., A&F facilitator) in clinics A, B, and C was a physician known to the staff, a leader in the settings, not a direct supervisor of the staff, and the project’s principal investigator (CB). In clinic D, the facilitator was a nurse researcher from another organization, with expertise in supporting nurses, known to the staff only from engagement activities (AGB). The context for feedback with individual staff was synchronous for the first 6 months of implementation, in-person for clinics A, B, and C and by phone for clinic D. The IPAF facilitator met with individuals for up to 10 min, privately in a clinic room or by phone, at a mutually agreed time. After the first 6 months of each implementation, we shared feedback asynchronously by email, along with questions for the staff to share their barriers, goals, and action steps regarding target behaviors. The staff sent their responses and goals to the facilitator by email. The frequency of feedback was monthly amounting to at least four synchronous sessions per individual between months 1 and 6 of each implementation, and over 375 monthly emails for up to 4 years thereafter (2016–2019).

Facilitators and participants collaboratively used the IPAF tool during the synchronous IPAF sessions, shown in Fig. 3. The purpose of the tool was to serve as a guide, to support fidelity of delivery and receipt [17, 36]. Rather than being a rigid script, the tool consisted of a worksheet that was semi-structured to guide flexible and realistic discussions. It included what critical components to address with the staff to improve clinic rates of best practices (self-regulation theory) and how to interact with the staff to improve their behaviors aligned with best practices (self-determination theory) based on SDT. Table 1 presents the concepts, theories, and rationale for the IPAF components included in the tool.

Fig. 3
figure 3

Tool to guide the application of Interactive and Participatory Audit and Feedback (IPAF) with fidelity

Full size image

Feedback sessions addressed the SRT concepts depicted in Fig. 1. Guided by the IPAF tool (Fig. 3), the facilitator provided feedback on individuals’ rates of target behaviors and shared the range of peers’ rates of target behaviors, grouped by clinic. Feedback was designed to stimulate the recipients’ own comparison between their recent behaviors and individual target behaviors, as well as with intervention goals. Also, the IPAF facilitator explained the long-term goal of controlling CVD in the rheumatology population to highlight an altruistic and non-controlling motive to improve one’s behaviors. When the staff experienced a potential discrepancy between actual and desired behaviors, then we assumed this could be motivational for new goal setting and regulation.

Feedback sessions also addressed SDT concepts [25]. Guided by the IPAF tool (Fig. 3), the IPAF facilitator explained that the intent was to be collaborative, not judgmental. To respect individuals’ autonomy, the facilitator offered choices about the order of discussion topics, starting with either feedback on individual-level data or discussing how the individuals thought the intervention was going. The facilitator elicited from individuals their barriers to engaging in target behaviors, possible solutions, goals for target behaviors, and action steps for the upcoming month.

Evaluation of implementation outcomes associated with IPAF

Data collection

Feasibility

We evaluated feasibility by whether we were able to collect rates of individual staff’s target behaviors. The staff were to document these behaviors in new EHR data fields to enable monthly audits. The information technology staff were to report rates of these behaviors monthly to the implementation team. Additionally, we evaluated the degree of staff participation in IPAF sessions, based on team records including notes from facilitators and debriefing session facilitators had with other team members after each IPAF session. We assessed both staff’s attendance at feedback sessions and their engagement in planned steps for behavior change during these sessions.

Appropriateness

We assessed appropriateness qualitatively during meetings to engage the staff in planning (Engage implementation component) pre-implementation and at 3 months mid-implementation.

Acceptability

We evaluated the staff’s opinions with selected items from a 22-item written questionnaire. Questionnaires developed by the study team included items to evaluate acceptability and fidelity to theory components [19, 20, 22, 25, 26] and support for the work system [37, 38]. For example, consistent with SDT concepts, participants reported the degree to which the project team listened to them, responded to their ideas, flexibly tailored protocols, addressed barriers, and collaborated in planning. The staff answered anonymously at month 6 on a 5-point scale from 1 “not at all” to 5 “extremely”. We also asked the staff for comments at the end of the questionnaires. Finally, team records, including facilitator notes and post-session debriefings, documented the degree of participants’ engagement during the one-to-one feedback sessions.

Fidelity

We evaluated fidelity based on monthly copies of the IPAF tool and on staff questionnaire responses. Completion of all sections of the IPAF tool by the facilitator for each session indicated fidelity of delivery. We evaluated the fidelity of receipt with relevant questionnaire items, including feedback based on SDT and SRT. These included the extent to which participants felt respected by the staff, found problem solving helpful, and experienced goal setting as motivational during the feedback sessions.

Adoption

We measured staff’s rates of target behaviors, consistent with the Check-Advise-Connect structure and SDT/SRT reinforced A&F for the two interventions. We defined adoption of the Check behavior as the rate of confirming the CVD risk factor: re-measuring high BPs or asking about readiness to quit tobacco. We defined adoption of the Connect behavior as the rate of offering follow-up: appointments with primary care for high BP or electronic referrals for counseling calls from the tobacco quit line. Moreover, in addition to actual rates, our theories suggest that intentions and perceived confidence can predict behavior. On the monthly IPAF worksheets, we evaluated the staff’s goals for future rates of target behaviors to reflect intentions at the first and last interactive sessions. In questionnaires, we asked the staff to rate their perceived confidence in addressing CVD risk factors pre- and post-intervention, on a 5-point scale, retrospectively for clinics A, B, and C and prospectively for clinic D.

Analyses

We generated descriptive statistics, including frequencies and standard deviations, for responses to questionnaire items and target behaviors. Paired t tests were used to compare the staff’s pre- and post-questionnaire responses.

Protection of human participants

Our project received approval through the University of Wisconsin-Madison’s Health Sciences Institutional Review Board and the Gundersen Health System’s Institutional Review Board. According to the policies covering research activities at both institutions, our project met the exemption criteria for operational improvement activities, with permission to publish. We summarized participants’ data in aggregate and did not share individual audits with the supervisors of participants.

Results

Across our eight implementations, 18 different staff participated in IPAF sessions, and we received 30 responses from them on the month 6 questionnaires. We had 100% response rates among participants employed at the time the questionnaires were administered. All but one staff (94%) were female and most had no prior research or quality improvement experience. Table 3 shows the characteristics of the IPAF participants and questionnaire respondents.

Table 3 Characteristics of participants
Full size table

Feasibility

Both the audit and feedback components of IPAF demonstrated feasibility. The information technology staff were able to retrieve all monthly rates of target behaviors from the EHR, including new EHR fields documented by the staff. We were able to provide feedback in 99 total monthly in-person or phone sessions with 18 different staff and over 375 monthly emails thereafter. Staff attendance at the feedback sessions was high: 90% at 72 of 80 planned sessions in clinics A, B, and C and 93% at 25 of 27 planned sessions in clinic D. When participants missed a session, they were absent from work for illness or personal reasons. The sessions lasted 7 min on average (range 3–10 min).

According to team records, including facilitator notes and completed copies of the IPAF tool, all participants engaged in identifying barriers to target behaviors, setting goals, and making action plans. In notes and debriefings, IPAF facilitators observed that the staff were willing and able to set goals when prompted. A review of the copies of the IPAF tool showed that when the staff were unable to set a percentage goal for infrequent behaviors, the IPAF tool accommodated setting a numerical goal (e.g., n = 2 quit line referrals). Across team records, a common approach was to mutually determine suggested goals based on prior rates, achievable expectations, and the facilitator’s sense of an individual’s motivation in the project overall.

Appropriateness

The staff that we engaged in planning meetings pre- and mid-implementation qualitatively described IPAF as appropriate. They perceived it as fitting well with their workflows and described it as practicable and useful as part of the implementation plans.

Acceptability

Responses to questionnaire items relevant to IPAF revealed high acceptability. In clinics A, B, and C, respondents (n = 20) reported that the project team listened (mean = 4.47 ± 0.76 standard deviation), were responsive (4.46 ± 0.73), flexible to ideas (4.22 ± 0.97), and collaborative (4.44 ± 0.53). Table 4 shows further details from all implementations. In open-ended questionnaire responses, the staff reported appreciating seeing their data and practicing what they would say with patients in both training and individual IPAF sessions. Several staff reported eagerness to disseminate the protocols and implementation strategies, including IPAF, to other specialty clinics.

Table 4 Data on implementation outcomes: acceptability, fidelity, and adoption
Full size table

Fidelity

Data from copies of the IPAF tool documented the completion of >80% of all goal setting topics and other discussion sections, supporting fidelity of delivery. Detailed findings are in Table 4. To summarize, in clinics A, B, and C, respondents (n = 20) reported the following: the implementation team was respectful (4.90 ± 0.33), problem solving was helpful (4.79 ± 0.43), and goal setting was motivational (4.13 ± 0.78). When asked about perceived confidence addressing BP and tobacco, only 10–20% said they had been “very” or “extremely confident” at baseline while 90% reported they were “very” or “extremely confident” after 6 months (P = 0.001), as shown in Fig. 4.

Fig. 4
figure 4

Staff scores on perceived confidence to address high BP and tobacco before and after implementation. Results show that 90% were very or extremely confident after (dark bar) compared to 10–20% before (light bar) each intervention (n = 10 respondents per questionnaire)

Full size image

Table 4 shows further details from all eight implementations including our replication site, clinic D. Overall, responses from implementations in clinic D showed lower scores. That health system was understaffed and had 66% clinic staff turnover during implementation, thus new staff received abridged engagement and education.

Addressing barriers

Using the IPAF tool to guide fidelity, facilitators asked participants to identify barriers to the adoption of target behaviors when prompted. These barriers included having high patient volume, being understaffed, working with float staff, and working with some providers who rushed processes for rooming patients. Participants also generated solutions for barriers during IPAF sessions. For example, participants needed to address a barrier of time pressure to measure and re-measure BPs before visits with providers. Participants offered a solution using red laminated reminder cards on the desks in patients’ exam rooms to alert providers and staff that repeat BPs were needed after provider visits. Using their solution resulted in a subsequent increase in rates of BP re-measurement.

Building rapport

In addition to the structured discussion, the monthly IPAF sessions guided by the semi-structured IPAF tool also afforded opportunities for reciprocal flexible communication, consistent with SDT. It was not uncommon for participants to share unsolicited details on changes in the work system or staff perceptions of the intervention or implementation process. Based on facilitator debriefings and participant comments, by having a consistent point of contact, facilitators also formed relationships with participants, so additional communication was able to occur. This fostered rapport and made going through structured feedback items more comfortable for both facilitators and participants.

Adoption

As expected, adoption of target behaviors improved with monthly SRT- and SDT-based feedback. In the first IPAF sessions, the staff’s goals for the Check behavior varied from 50 to 100%. In contrast, in the last IPAF sessions, the staff’s goals for this behavior increased to 75–100%. Based on EHR data, the staff’s actual rates of the Check behavior improved from 2–3% to 98–100% pre- to post-implementation in clinics A, B, and C. Clinic D had a peak of 80% for BP Connect, though ranges varied while experiencing understaffing. The staff’s rates of the Connect behavior rose from 0–1% to 68–76% pre- to post-implementation, with the exception of Quit Connect at clinic D. Table 4 shows further details from all eight implementations.

Discussion

Advancing research on theory-based A&F as an implementation strategy

We have developed and evaluated a theory-based A&F strategy by applying a relatively novel combination of behavioral and motivational theories. Responding to the need for detailed reports of how theory guides A&F [4, 5, 39,40,41,42], we delineated the theoretical concepts informing our IPAF components and tool in Table 1 and Fig. 3, respectively. We have posted these resources as a toolkit at https://www.hipxchange.org/AuditFeedback [35]. SRT guided us in conceptualizing what to address, and SDT guided us in how to interact with the staff effectively. We applied concepts and evidence from SRT, supported by behavior change techniques [20, 22], which we incorporated into IPAF components, such as barrier identification and problem solving [20, 22, 23]. We applied SDT concepts to interpersonal processes during IPAF sessions to support the staff’s psychological needs and motivation for the adoption of target behaviors [26].

IPAF can be situated within the literature on behavior change through the Theoretical Domains Framework (TDF) which identifies influences on health professional behavior regarding implementation of evidence-based recommendations [43]. The TDF has classified SRT and SDT as action and motivation theories, respectively [44]. IPAF is consistent with most of the theoretical domains of the TDF. Specifically, our combination of SRT and SDT addresses 8 of 14 domains directly, three domains indirectly, and does not address three domains (optimism, emotion, intention). The strength of our approach is the parsimonious use of two middle-range theories as this combination provided concepts that were applicable and allows for testable relationships between the concepts to be examined in future research. We are also not aware of other detailed evaluations of implementation outcomes associated with theory-based or theory-informed A&F interventions.

More recently, Clinical Performance Feedback Intervention Theory (CP-FIT) [45] is an overarching theory of A&F, published after we completed the development and evaluation work reported here. The CP-FIT feedback cycle is consistent with IPAF, with an important distinction. CP-FIT conceptualizes A&F participants as recipients, which risks perpetuating a view of A&F as a passive, unidirectional delivery of information. IPAF demonstrates the benefits of an expanded view whereby A&F is a collaborative effort between facilitators and participants interacting to exchange feedback from participants about perceived barriers to target behaviors and from facilitators’ performance data and shared strategies, where participants can autonomously make choices about their goals and action plans. Our IPAF approach also adds to the Calgary Audit and Feedback Framework [46, 47] and the Relationship, Reactions, Content, Coach model (R2C2) [48]. Interactive and participatory processes may be particularly suited to A&F with health professionals with typically lower autonomy and perceived competence, including medical assistants versus physician-focused A&F approaches. The Calgary and R2C2 approaches were developed with a physician focus and use group A&F as a central component. Group A&F likewise requires further scrutiny in the context of frontline staff such as medical assistants who may feel intimidated in that setting, especially in groups including individuals with varying experience, tenure, and stature. The present IPAF study examined individual A&F, though we have begun comparing group vs. individual feedback with medical assistants in new studies.

Advancing research on implementing clinic protocols with A&F

We have extended prior research on clinic protocols in primary care [8] and on A&F with physicians, pharmacists, and nurses by primarily targeting frontline medical assistants [5, 6, 8]. We found that IPAF is feasible, appropriate, and acceptable to use with frontline staff, such as medical assistants in specialty clinics, which was important to demonstrate, especially since we asked staff to adopt new professional roles for CVD risk reduction. Fidelity was evidenced by the facilitators’ documentation on copies of the IPAF tool, by staff’s responses to questionnaire items, and by the fact that staff’s perceived competence was higher after the implementations than prior to them. Adoption of target behaviors improved at 6 months and 24 months, including checking risk factors (i.e., BP and tobacco) and connecting patients to CVD risk reduction follow-up [13,14,15,16]. Such positive findings are consistent with the theories guiding our project and indicate that frontline staff such as medical assistants can be partners in implementing clinic protocols in specialty clinics and that our interactive and participatory A&F strategy can support such improvements.

Limitations

Although this paper establishes strong multi-site implementation outcomes of a theory-based A&F strategy, we acknowledge limitations in our quasi-experimental design and the evaluation of IPAF in the context of other implementation components. Moreover, our reported sample size of staff receiving IPAF (n = 18) was modest. This limitation is offset by our multi-site findings representing eight implementations across two health systems, including 99 feedback sessions and more than 375 monthly feedback emails. Another limitation is that audit data for float staff were aggregated within staff data overall, yet float staff did not receive IPAF. Also, because of turnover, new staff received abridged engagement and education prior to IPAF, which may have diluted exposure to some SDT- and SRT-informed elements. Additionally, findings based on staff self-reports may be attributed to social desirability influences, given that the principal investigator was the IPAF facilitator in three of the four clinics. Consistent with Cochrane A&F recommendations for effective A&F [5], this facilitator was a respected colleague known to the staff within the three clinics. To reduce the risk of social desirability influence, the facilitator began each session anchoring the benefits of target behaviors for patient care and outcomes. In addition, the facilitator assured the staff that individual performance data remained confidential from supervisors. Similar staff target-behavior gains were confirmed with delivery by a nurse facilitator in a fourth clinic, suggesting efficacy and replicability beyond social desirability bias [14]. Finally, our pre- post-measurement of the staff’s perceived confidence was initially retrospective, which we addressed with prospective questionnaires in the fourth clinic.

Future research

Researchers and clinical improvement teams can apply and extend our theory-based IPAF approach to evaluate improvements in staff adoption of other target behaviors in other settings. Teams can download our IPAF toolkit, including its tool and theoretical details, at https://www.hipxchange.org/AuditFeedback [35]. We plan a future clinic randomized controlled study, with active A&F comparisons of individual versus group feedback, and larger sample sizes, to further test these methods.

Conclusions

We developed and evaluated IPAF, an interactive and participatory A&F strategy, to improve the implementation of evidence-based interventions, building on two complementary theories, SRT and SDT. Findings support feasibility, appropriateness, acceptability, and fidelity of IPAF, as well as staff adoption and maintenance of target behaviors. By evaluating implementation outcomes in multi-site specialty clinics with medical assistants and nurses performing pre-visit rooming, we have extended prior research on clinic protocols and A&F beyond primary care settings and providers. Our explication of how and why we applied the concepts from two complementary theories to IPAF can provide a strong foundation for future efforts. Researchers and clinical improvement teams can use IPAF to help improve clinical outcomes such as reducing CVD risk factors.

Reporting checklists: The STaRI and STROBE Checklists were used for reporting. (see Additional files).

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request per IRB approval.

Abbreviations

A&F:

Audit and feedback

CVD:

Cardiovascular disease

BP:

Blood pressure

IPAF:

Interactive and Participatory Audit and Feedback

EHR:

Electronic health records

SRT:

Self-regulation theory

SDT:

Self-determination theory

TDF:

Theoretical Domains Framework

References

  1. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21. https://doi.org/10.1186/s13012-015-0209-1.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Colquhoun HL, Carroll K, Eva KW, Grimshaw JM, Ivers N, Michie S, et al. Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses. Implement Sci. 2017;12(1):117. https://doi.org/10.1186/s13012-017-0646-0.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Ivers NM, Grimshaw JM, Jamtvedt G, Flottorp S, O’Brien MA, French SD, et al. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014;29(11):1534–41. https://doi.org/10.1007/s11606-014-2913-y.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8(1):66. https://doi.org/10.1186/1748-5908-8-66.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;(6):CD000259. Published 2012 Jun 13. https://doi.org/10.1002/14651858.CD000259.pub3.

  6. Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, et al. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9(1):14. https://doi.org/10.1186/1748-5908-9-14.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Sales A, Smith J, Curran G, Kochevar L. Models, strategies, and tools - theory in implementing evidence-based findings into health care practice. J Gen Intern Med. 2006;21(S2):S43–S9. https://doi.org/10.1111/j.1525-1497.2006.00362.x.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Frieden TR, King SM, Wright JS. Protocol-based treatment of hypertension: a critical step on the pathway to progress. JAMA. 2014;311(1):21–2. https://doi.org/10.1001/jama.2013.282615.

    Article  CAS  PubMed  Google Scholar 

  9. Vreede AP, Johnson HM, Piper M, Panyard DJ, Wong JC, Bartels CM. Rheumatologists modestly more likely to counsel smokers in visits without rheumatoid arthritis control: an observational study. J Clin Rheumatol. 2017;23(5):273–7. https://doi.org/10.1097/RHU.0000000000000559.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Bartels CM, Johnson H, Alcaraz Voelker K, Ogdie A, McBride P, Jacobs EA, et al. Frequency and predictors of communication about high blood pressure in rheumatoid arthritis visits. J Clin Rheumatol. 2018;24(4):210–7. https://doi.org/10.1097/RHU.0000000000000726.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Avina-Zubieta JA, Thomas J, Sadatsafavi M, Lehman AJ, Lacaille D. Risk of incident cardiovascular events in patients with rheumatoid arthritis: a meta-analysis of observational studies. Ann Rheum Dis. 2012;71(9):1524–9. https://doi.org/10.1136/annrheumdis-2011-200726.

    Article  PubMed  Google Scholar 

  12. Aitken M, Kleinrock M, Lyle J, Caskey L. Medicine use and shifting costs of healthcare: a review of the use of medicines in the United States in 2013. Parisppany: IMS Institute for Healthcare Informatics; 2014.

    Google Scholar 

  13. Bartels CM, Ramly E, Johnson HM, Lauver DR, Panyard DJ, Li Z, et al. Connecting rheumatology patients to primary care for high blood pressure: specialty clinic protocol improves follow-up and population blood pressures. Arthritis Care Res. 2019;71(4):461–70. https://doi.org/10.1002/acr.23612.

    Article  Google Scholar 

  14. Bartels CM, Johnson L, Ramly E, Panyard DJ, Gilmore-Bykovskyi A, Johnson HM, et al. Impact of a rheumatology clinic protocol on tobacco cessation quit line referrals. Arthritis Care Res. 2021. https://doi.org/10.1002/acr.24589.

  15. Ramly E, Panyard D, Lauver D, Sampene E, Li Z, Johnson H, et al. Sustained improvement in follow-up of hypertension in rheumatology patients: results of an intervention sustainability assessment [abstract]. Washington, DC: American College of Rheumatology; 2016. Arthritis Rheumatol

    Google Scholar 

  16. Bartels CM, Ramly E, Panyard D, et al. Rheumatology clinic smoking cessation protocol markedly increases quit line referrals [abstract]. Arthritis Rheumatol. 2017;69(suppl 10):1523-24.

  17. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38(2):65–76. https://doi.org/10.1007/s10488-010-0319-7.

    Article  Google Scholar 

  18. Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M. Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007;85(1):93–138. https://doi.org/10.1111/j.1468-0009.2007.00478.x.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Deci EL, Olafsen AH, Ryan RM. Self-determination theory in work organizations: the state of a science. Annu Rev Organ Psych. 2017;4(1):19–43. https://doi.org/10.1146/annurev-orgpsych-032516-113108.

    Article  Google Scholar 

  20. Scheier M, Carver C, Armstrong G. Behavioral self-regulation, health, and illness. In: Baum A, Revenson T, Singer J, editors. Handbook of health psychology. New York: Psychology Press; 2012. p. 79–98.

    Google Scholar 

  21. Gagné M, Deci EL, Ryan RM. Self-determination theory applied to work motivation and organizational behavior. In: The SAGE handbook of industrial, work & organizational psychology: organizational psychology, vol. 2. 2nd ed: Sage Reference; 2018. p. 97–121.

  22. Carver CS, Scheier MF. On the self-regulation of behavior. New York: Cambridge University Press; 1998. https://doi.org/10.1017/CBO9781139174794.

    Book  Google Scholar 

  23. Michie S, Abraham C, Whittington C, McAteer J, Gupta S. Effective techniques in healthy eating and physical activity interventions: a meta-regression. Health Psychol. 2009;28(6):690–701. https://doi.org/10.1037/a0016136.

    Article  PubMed  Google Scholar 

  24. Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci. 2009;4(1):40. https://doi.org/10.1186/1748-5908-4-40.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Ng JY, Ntoumanis N, Thogersen-Ntoumani C, Deci EL, Ryan RM, Duda JL, et al. Self-determination theory applied to health contexts: a meta-analysis. Perspect Psychol Sci. 2012;7(4):325–40. https://doi.org/10.1177/1745691612447309.

    Article  PubMed  Google Scholar 

  26. Ryan RM, Patrick H, Deci EL, Williams GC. Facilitating health behaviour change and its maintenance: interventions based on self-determination theory. Eur Health Psychol. 2008;10(1):2–5.

    Google Scholar 

  27. Fernet C, Austin S, Vallerand RJ. The effects of work motivation on employee exhaustion and commitment: an extension of the JD-R model. Work Stress. 2012;26(3):213–29. https://doi.org/10.1080/02678373.2012.713202.

    Article  Google Scholar 

  28. Moreau E, Mageau G. The importance of perceived autonomy support for the psychological health and work satisfaction of health professionals: not only supervisors count, colleagues too! Motiv Emot. 2012;36(3):268–86. https://doi.org/10.1007/s11031-011-9250-9.

    Article  Google Scholar 

  29. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26. https://doi.org/10.1097/MLR.0b013e3182408812.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Bartels CM, Ramly E, Panyard D, Lauver DR, Johnson HM, Lewicki K, et al. BP connect health toolkit. Madison: University of Wisconsin – Madison School of Medicine and Public Health; 2017. https://www.hipxchange.org/BPConnectHealth. Accessed 5 Feb 2018

    Google Scholar 

  31. Bartels CM, Ramly E, Panyard D, Lauver DR, Johnson HM, Lewicki K, et al. Quit connect health toolkit. Madison: University of Wisconsin – Madison School of Medicine and Public Health, UW Center for Tobacco Research and Intervention, and UW Health; 2017. https://www.hipxchange.org/QuitConnectHealth Accessed 5 Feb 2018

    Google Scholar 

  32. Bartels CM, Panyard D, Lauver D, et al. Feasibility of a rheumatology staff protocol for tobacco cessation counselling and Quit Line electronic referral [abstract]. Arthritis Rheumatol. 2016;68(suppl 10):548-49.

  33. Bartels CM, Johnson H, Voelker K, Thorpe C, McBride P, Jacobs EA, et al. Impact of rheumatoid arthritis on receiving a diagnosis of hypertension among patients with regular primary care. Arthritis Care Res. 2014;66(9):1281–8. https://doi.org/10.1002/acr.22302.

    Article  Google Scholar 

  34. Primdahl J, Clausen J, Horslev-Petersen K. Results from systematic screening for cardiovascular risk in outpatients with rheumatoid arthritis in accordance with the EULAR recommendations. Ann Rheum Dis. 2013;72(11):1771–6. https://doi.org/10.1136/annrheumdis-2013-203682.

    Article  CAS  PubMed  Google Scholar 

  35. Bartels CM, Ramly E, Lauver D. Guide for staff interactive participatory audit - feedback. Madison: University of Wisconsin School of Medicine and Public Health; 2018. https://www.hipxchange.org/AuditFeedback. Accessed 5 Nov 2018

    Google Scholar 

  36. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23(5):443–51. https://doi.org/10.1037/0278-6133.23.5.443.

    Article  PubMed  Google Scholar 

  37. Carayon P, Schoofs Hundt A, Karsh BT, Gurses AP, Alvarado CJ, Smith M, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care. 2006;15(Suppl 1):i50–8. https://doi.org/10.1136/qshc.2005.015842.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Holden RJ, Carayon P, Gurses AP, Hoonakker P, Hundt AS, Ozok AA, et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics. 2013;56(11):1669–86. https://doi.org/10.1080/00140139.2013.838643.

    Article  PubMed  Google Scholar 

  39. Smith GP, Williams TM. You can lead a horse to water … what self-determination theory can contribute to our understanding of clinical policy implementation. J Health Serv Res Policy. 2017;22(1):37–44. https://doi.org/10.1177/1355819616652922.

    Article  PubMed  Google Scholar 

  40. Breckenridge JP, Gray N, Toma M, Ashmore S, Glassborow R, Stark C, et al. Motivating change: a grounded theory of how to achieve large-scale, sustained change, co-created with improvement organisations across the UK. BMJ Open Qual. 2019;8(2):e000553. https://doi.org/10.1136/bmjoq-2018-000553.

    Article  PubMed  PubMed Central  Google Scholar 

  41. van Deen WK, Cho ES, Pustolski K, Wixon D, Lamb S, Valente TW, et al. Involving end-users in the design of an audit and feedback intervention in the emergency department setting - a mixed methods study. BMC Health Serv Res. 2019;19(1):270. https://doi.org/10.1186/s12913-019-4084-3.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Groen BAC, Wouters MJF, Wilderom CPM. Employee participation; performance metrics, and job performance: a survey study based on self-determination theory. Manag Account Res. 2017;36:51–66. https://doi.org/10.1016/j.mar.2016.10.001.

    Article  Google Scholar 

  43. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(1):37. https://doi.org/10.1186/1748-5908-7-37.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33. https://doi.org/10.1136/qshc.2004.011155.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Brown B, Gude WT, Blakeman T, van der Veer SN, Ivers N, Francis JJ, et al. Clinical Performance Feedback Intervention Theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci. 2019;14(1):40. https://doi.org/10.1186/s13012-019-0883-5.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Cooke LJ, Duncan D, Rivera L, Dowling SK, Symonds C, Armson H. The Calgary audit and feedback framework: a practical, evidence-informed approach for the design and implementation of socially constructed learning interventions using audit and group feedback. Implement Sci. 2018;13(1):136. https://doi.org/10.1186/s13012-018-0829-3.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Cooke LJ, Duncan D, Rivera L, Dowling SK, Symonds C, Armson H. How do physicians behave when they participate in audit and feedback activities in a group with their peers? Implement Sci. 2018;13(1):104. https://doi.org/10.1186/s13012-018-0796-8.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Sargeant J, Lockyer J, Mann K, Holmboe E, Silver I, Armson H, et al. Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Acad Med. 2015;90(12):1698–706. https://doi.org/10.1097/Acm.0000000000000809.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Courtney Maxcy, Sarah Loring, Amanda Perez, Monica Messina, and the Health Innovation Program Staff for the help with the data collection and manuscript preparation.

Funding

This work was supported by a peer-reviewed, independent investigator grant (PI-Bartels) titled “Systems-Based CVD Prevention Protocols for Rheumatology Teams: A low cost multidisciplinary approach” from Independent Grants for Learning and Change (Pfizer) and Stepping Up in Specialty Clinics to Reduce Blood Pressure (NIH-NCATS UL1TR000427, through a Clinical & Community Outcomes Research Pilot (PI-Bartels) Clinical and Translational Science Award (CTSA) program, through the NIH National Center for Advancing Translational Sciences (NCATS) grant as well as the UW School of Medicine and Public Health. Bartels also received support from the National Institutes of Health (NIH) National Institute of Arthritis, Musculoskeletal and Skin Diseases (NIAMS) (K23 #AR062381) for the baseline data. The authors have no other direct financial, consultant, or institutional conflict of interest pertaining to this article. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Funders had no role in the design, collection, analysis, or interpretation of the data; writing of the manuscript; or decision to submit for publication.

Author information

Author notes

  1. Edmond Ramly and Diane R Lauver are joint first authors.

Authors and Affiliations

  1. Department of Family and Community Medicine, School of Medicine & Public Health, University of Wisconsin-Madison, Madison, WI, USA

    Edmond Ramly

  2. Department of Industrial and Systems Engineering, College of Engineering, University of Wisconsin-Madison, Madison, WI, USA

    Edmond Ramly

  3. School of Nursing, University of Wisconsin-Madison, Madison, WI, USA

    Diane R. Lauver & Andrea Gilmore-Bykovskyi

  4. Department of Medicine, School of Medicine & Public Health, University of Wisconsin-Madison, 1685 Highland Ave, Rm 4132, Madison, WI, 53705-2281, USA

    Christie M. Bartels

Authors
  1. Edmond Ramly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Diane R. Lauver
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Gilmore-Bykovskyi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christie M. Bartels
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ER and DRL are joint first authors. ER provided conception of the effectiveness-implementation hybrid design, implementation outcomes, all implementation components, and two interventions; contributed to the acquisition, analysis, and interpretation of the data; and contributed to the drafting and substantive revisions of the article. DRL guided the conception of the theory-based audit and feedback strategy; contributed to the analysis and interpretation of the data; and contributed to the drafting and substantive revisions of the article. AGB provided acquisition, analysis, and interpretation of the data and contributed to the substantive revision of the article. CB obtained funding, provided conception of the two interventions, all four implementation components, and effectiveness-implementation hybrid design; contributed to the acquisition, analysis, and interpretation of the data; and contributed to the drafting and substantive revisions of the article. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Christie M. Bartels.

Ethics declarations

Ethics approval and consent to participate

We received approval for our project through the University of Wisconsin-Madison’s Health Sciences Institutional Review Board (IRB) and the Gundersen Health System’s IRB. According to the policies covering research activities at both institutions, our project met the exemption criteria for operational improvement activities, with permission to publish. We summarized participants’ data in aggregate and did not share individual audits with the supervisors of the participants.

Consent for publication

Not applicable.

Competing interests

All authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Results from the first three implementations were shared in a podium presentation at the American College of Rheumatology National Meeting, Washington DC, November 16, 2016, and a podium presentation at the Academy Health Annual Conference on the Science of Dissemination and Implementation, Washington DC, December 14, 2016. Theoretical foundations were shared in a podium presentation at the Sixth International Self-Determination Theory Conference June 2–5, 2016, Victoria, British Columbia, Canada.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramly, E., Lauver, D.R., Gilmore-Bykovskyi, A. et al. Interactive and Participatory Audit and Feedback (IPAF): theory-based development and multi-site implementation outcomes with specialty clinic staff. Implement Sci Commun 2, 58 (2021). https://doi.org/10.1186/s43058-021-00155-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-021-00155-4

Keywords

Implementation Science Communications

ISSN: 2662-2211