Skip to main content

Protocol for evaluating external facilitation as a strategy to nationally implement a novel stigma reduction training tool for healthcare providers



Identity Development Evolution and Sharing (IDEAS) is a theatre-based intervention for reducing healthcare provider stigma. IDEAS films are created by collecting narratives from people who have experienced discrimination and healthcare inequity, partnering with professional playwrights to create theatrical scripts that maintain the words of the narratives while arranging them into compelling storylines involving several interviews, and hiring professional actors to perform and record scenes. IDEAS implementation requires a moderator to establish a respectful learning environment, play the filmed performance, set ground rules for discussion, and moderate a discussion between healthcare providers who viewed the film and invited panelists who are members of the minoritized population being discussed. IDEAS’ impact on provider stigma is measured via pre/post Acceptance and Action Questionnaire – Stigma (AAQ-S) data collected from participating providers. The objectives of this manuscript are to provide narrative review of how provider stigma may lead to healthcare inequity and health disparities, describe the conceptual frameworks underpinning the IDEAS intervention, and outline methods for IDEAS implementation and implementation evaluation.


This manuscript describes a hybrid type 3 design study protocol that uses the Consolidated Framework for Implementation Research (CFIR) to evaluate external facilitation, used as an implementation strategy to expand the reach of IDEAS. CFIR is also used to assess the impact of characteristics of the intervention and implementation climate on implementation success. Implementation success is defined by intervention feasibility and acceptability as well as self-efficacy of internal facilitators. This manuscript details the protocol for collection and evaluation of implementation data alongside that of effectiveness data. The manuscript provides new information about the use of configurational analysis, which uses Boolean algebra to analyze pathways to implementation success considering each variable, within and across diverse clinical sites across the USA.


The significance of this protocol is that it outlines important information for future hybrid type 3 designs wishing to incorporate configurational analyses and/or studies using behavioral or atypical, complex, innovative interventions. The current lack of evidence supporting occupational justice-focused interventions and the strong evidence of stigma influencing health inequities underscore the necessity for the IDEAS intervention.

Peer Review reports


The field of occupational therapy is committed to fostering a culture of occupational justice and supporting the wellbeing of clients [1]. Occupational justice is “the right of every individual to be able to meet basic needs and to have equal opportunities and life chances to reach toward their potential…specific to the individual’s engagement in diverse and meaningful occupations” [2]. The field also emphasizes the crucial role of service, system, and policy structure at a community and society level to either promote or inhibit this access to justice [1]. A critical barrier to progress in occupational therapists’ ability to promote equity in health care is the limited availability of evidence-based approaches occupational therapists can implement to support occupational justice. To address this gap, external facilitation will be used to disseminate Identity Development Evolution and Sharing (IDEAS), a program for reducing healthcare provider stigma, with a growing body of supporting evidence [3, 4]. This manuscript describes (1) the IDEAS intervention, (2) the planned implementation and implementation evaluation; and (3) provides a narrative review of how provider stigma may lead to healthcare inequity and health disparities.

Intervention: Identity Development Evolution and Sharing (IDEAS)

IDEAS is a one-hour, client-centered, occupation-based, theatre intervention to reduce stigma beliefs among healthcare providers [3, 4]. IDEAS performances are created by conducting narrative interviews with people from a target population who have been harmed by provider stigma and translating those narratives into a theatrical performance to be performed by professional actors for healthcare providers. IDEAS implementation requires a moderator to establish a respectful learning environment, play the filmed performance, set ground rules for discussion, and moderate a discussion between healthcare providers who viewed the film and invited panelists who are members of the minoritized population being discussed. The three existing IDEAS videos to be used in this study are as follow: “The Rest of my Life”—an account of several peoples’ journeys with substance use disorder (SUD)—“Stories of Inequity”—ten Black women’s accounts of experiencing discrimination in healthcare—and “This Authentic Person”—the stories of several transgender and gender-diverse people’s journeys with gender and societal stigma.

IDEAS is rooted in a robust body of literature that suggests attending to stories through literature, art, theatre, and poetry improves clinical encounters and interpersonal relationships; this attention increases empathy and understanding [5,6,7,8]. Clinicians become better able to notice important nuances of people’s stories. Inter-professional communication improves [5, 8]. Fewer medical errors are made. Psychological flexibility increases. An IDEAS performance is an opportunity to attune to and witness others’ stories. Attending to stories through theatre provides viewers with an opportunity to privately witness and confront their own implicit biases [3, 4]. By becoming aware of and grappling with one’s own biases, viewers gain flexibility regarding how they respond to those biases in the moment with clients, co-workers, and people in general [9]. By contrast, being unaware of one’s own personal biases can result in problematic interpersonal interactions such as committing microaggressions, embodying a stigmatizing or alienating demeanor, and/or making inequitable, harmful decisions regarding client care [10].

Conceptual models

Conceptual framework for evaluating IDEAS effectiveness on stigma reduction and enhanced psychological flexibility

The primary quantitative outcome measure will be the AAQ-S, which measures enacted, stigma, i.e., stigma beliefs that produce actions such as discriminating against people or groups [9]. Levin and colleagues [9], drawing on the work of Akrami, Ekehammar, and Bergh [11], conceptualize stigma as “a more general tendency to evaluate and discriminate against others based on their group membership, rather than being specific to attitudes towards any one group in particular.”9(p21) The AAQ-S was created through the lens of Acceptance and Commitment Therapy (ACT) and is rooted in the notion that all people exist within shared contexts of racism and other forms of discrimination at the individual, group, and institutional level and are thus affected by stigma whether beliefs of personal biases exist or not [9, 12, 13]. Implicit biases often contradict personal values, which lends insight as to how medical providers who value equitable health care provision can make inequitable decisions that cause harm to those seeking care. Psychological flexibility is “the capacity to actively embrace one’s private experiences in the present moment and engage or disengage in patterns of behavior in the service of chosen values.” [9] (p21) The greater the psychological flexibility a person has regarding their own biases, the more choices they have in terms of how to respond to discriminatory thoughts or ideas [9]. The ability to recognize and distance one’s self from their own biases is critical to providing equitable health care. The AAQ-S is a reliable and valid measure of stigma and psychological flexibility [9]. The AAQ-S has good inter-rater reliability (Cronbach’s alpha = 0.84) and low to moderate correlations (r = 0.23—0.43) with measures of right-wing authoritarianism, social dominance, ethnocultural empathy, empathic concern, and perspective taking [5, 9].

Central characteristics of psychological flexibility measured by AAQ-S [9] (p22)

Flexible awareness of one’s private experiences in the present moment, including stigmatizing thoughts

De-fusion from stigmatizing thoughts (seeing thoughts as just thoughts rather than something literally true)

Willingness to have stigmatizing thoughts, rather than engaging in ineffective forms of avoidance (e.g., thought suppression, avoiding situations where stigmatizing thoughts occur)

Relating to oneself and others as distinct from thoughts and feelings about them

Clarifying valued patterns of activity in social interactions

Committing to patterns of valued activity with others, even when stigmatizing thoughts and feelings seem to stand in the way

Conceptual framework for evaluating implementation

The Consolidated Framework for Implementation Research (CFIR) [14] will be used; this tool is a widely-used framework with over 2,000 citations in PubMed and over 5500 Google Scholar citations designed to “identify factors that might influence intervention implementation and effectiveness.” [15] (p15) Several CFIR resources will be used, including the CFIR interview generator, and guidelines for qualitative coding and quantitative rating of each of the CFIR implementation constructs on which data will be collected (see Table 1).

Table 1 CFIR constructs



The overall approach will involve (1) evaluating external facilitation to expand IDEAS implementation across multiple occupational therapy clinical sites and (2) measuring the effect of IDEAS on stigma beliefs of practicing occupational therapists.


Convenience and chain sampling will be used to recruit up to ten implementing occupational therapists—internal facilitators—to lead the IDEAS intervention within their respective clinics. Participants will intentionally be recruited from a diverse range of sites with respect to geographic location and clinical setting type (e.g., pediatric, skilled nursing, inpatient rehabilitation). In addition to the ten implementing occupational therapists who will be internal facilitators, participants of this study will include the staff occupational therapists within the clinic who participate in the IDEAS training and clinical managers who participate in interviews to provide information about the implementation climate.


Partnering sites will include a variety of urban, suburban, and rural occupational therapy clinical settings that serve a wide range of patients across the lifespan. Confirmed sites include outpatient pediatric clinics, adult inpatient rehabilitation hospitals, and extended care facilities. Sites will vary from smaller occupational therapist-led clinics to occupational therapy departments within large hospital networks. IDEAS is implemented as a group training for staff occupational therapy practitioners via a combination of in-person and virtual interactions. The initial training of the internal facilitator occurs via zoom. The IDEAS implementation can occur during an in-person or Microsoft Teams staff meeting. If staff meeting is in-person, expert panel speakers join the meeting via zoom or teams on a screen in the meeting room. If virtual, the panel speakers join the staff on the virtual platform.

Study design

Using a hybrid type 3 design, a formative evaluation of IDEAS implementation will be conducted using external facilitation as the implementation strategy [16, 17]. Evaluative objectives are to (1) collect and analyze quantitative data on provider (occupational therapists IDEAS participants) pre/post stigma beliefs using the AAQ-S; and (2) collect qualitative pre/post data from IDEAS participants, implementors, and clinical managers regarding their experiences of implementation within their site. The latter will be analyzed using CFIR analysis guidelines (described below).

Implementation strategy—external facilitation

The external facilitator (SW) will support occupational therapists/internal facilitators from diverse practice settings throughout the USA in implementing a single IDEAS intervention within their local clinical sites via a one-time external facilitation virtual meeting. During this meeting, the external facilitator will provide and review with each implementing occupational therapist/internal facilitator an e-resource manual (see Appendices 1 and 2), three IDEAS performance video links (sites select which performance is the best fit with respect to their priorities), and a list of contact information for panelist speakers for each video. Following the initial external facilitation meetings, the external facilitator will provide support via phone, text, and email exchanges as needed. Dose of external facilitation will be tracked via an external facilitation tracking log in which the external facilitator notes type of communication (email, phone, zoom), frequency, date, and duration of each encounter.

Implementation success

Our primary measures of implementation success are self-efficacy of implementors, experiences of external facilitation, and feasibility and acceptability of IDEAS [11, 18, 19].

Data collection

The research team (excluding the external facilitator) will conduct 3 pre/post stakeholder interviews at each implementation site; one with the implementing occupational therapist/internal facilitator, one with a clinical manager, and one with an occupational therapy practitioner who intends to participate in the IDEAS training. These interviews are created using the CFIR interview generator and include the constructs listed in Table 1 [20].

Following the initial external facilitation meeting between the external and internal facilitators, the internal facilitator will complete an online survey containing the items of the Acceptability of Intervention Measure (AIM), the Intervention Appropriateness Measure (IAM), and the Feasibility of Intervention Measure (FIM) regarding their experience of external facilitation [21]. Following IDEAS implementation, the internal facilitator and the occupational therapist who participate in both the IDEAS training and stakeholder interview will complete the AIM and IAM questions regarding the intervention itself. The survey questions are listed in Table 2 and scored on a 5-point scale from (1) “completely disagree” to (4) “completely agree.”

Table 2 IDEAS post-implementation survey

Occupational therapists who attend the IDEAS training will complete pre/post AAQ-S surveys electronically; the pre-survey also contains demographic questions such as age, ethnicity, race, and whether the person identifies as a member of the minoritized population the IDEAS intervention is focused on (yes, no), as well as a brief description of the study and a space for providing written informed consent.

Analytical methods

Evaluating implementation

Members of the research team will use the CFIR codebook template, which contains CFIR definitions and coding guidelines, to code each stakeholder interview from 10 implementation sites. The team will highlight data corresponding with each of the constructs listed in Table 1, while remaining open to additional emerging codes as needed to highlight pertinent data related to implementation. The team will work together on coding until consistent agreement is reached on how to code the data. The team then will code remaining transcripts in pairs, returning to prior transcripts and recoding as needed if/when the codebook is revised.

Following qualitative coding, the team will independently review the data for a single site and meet to assign a quantitative rating to each code based on qualitative data from that site, rating from − 2 to +2 based on whether each construct has a positive or negative impact on implementation success. For example, if the packaging of the intervention is described as problematically interfering with successful implementation (e.g., the film does not work), that construct of “quality packaging” will receive a − 2 rating. If the packaging is problematic (the film quality is poor but still works), the construct of “quality packaging” will receive a − 1. If the packaging does not influence implementation, it will receive a 0, and if it has a positive or highly positive impact on implementation, it will receive a + 1 or + 2, respectively. The team will enter their ratings in a blinded vote and will discuss any discrepancies, re-voting until consensus is reached for each construct.

Implementation success will be rated as high or low, based on average scores from the AIM, IAM, and FIM items as well as consensus ratings on the CFIR constructs of self-efficacy and experiences of external facilitation [14, 21]. Configurational analysis will then be conducted to determine difference-making combinations of CFIR-related conditions uniquely distinguishing higher- from lower-performing sites. Configurational analysis will draw upon Boolean algebra and set theory to identify a “minimal theory” or a crucial set of difference-making combinations that remove redundancies and that uniquely distinguish one group of cases from another (i.e., those with versus those without implementation success) [22,23,24,25,26,27,28]. Major strengths of this approach include its ability to model equifinality, when multiple paths lead to the same outcome; its capacity to identify complex relationships, when several conditions work together jointly as a whole; and its versatility with small studies [29,30,31,32,33]. Configurational analysis in this project will be conducted using Coincidence Analysis (cna) and is supported by using the R package “cna” as well as the software applications R and R Studio.

IDEAS effectiveness on provider stigma and correlations with implementation success

IDEAS effectiveness will be measured regarding its impact on provider stigma using repeated measures ANOVA to compare average provider AAQ-S pre-post change scores within and between sites [9]. A priori power analysis will be performed to determine the required sample size needed to obtain adequate statistical power for the IDEAS intervention. Using the following parameters based on repeated measure ANOVA within- and between- factors of two measurement time points, it assumes effect size of 0.80, α at 0.05, β at 0.80, and the correlation between repeated measures of 0.5. The required total sample size is determined to be 80. The aim, therefore, is to have 8 occupational therapist complete IDEAS, including the pre/post AAQ-S, at 10 sites. Effectiveness outcomes will be included in configurational analyses described above to explore whether/how implementation success and associated factors are related to effectiveness. For example, it is hypothesized that observations of high implementor self-efficacy combined with organizational cultures that embrace change may correlate with greater intervention effectiveness at that site, demonstrated via larger AAQ-S changes scores [9].


Literature review of provider stigma, healthcare inequity, and health disparities

This project is supported by an overwhelming body of literature demonstrating (1) the degree to which provider biases detrimentally affect health outcomes for marginalized groups; (2) the limited evidence-based occupational justice interventions despite the profession’s commitment to client-centered practice, advocacy, and equity; and (3) the lack of adequate measures for examining provider bias on clients’ healthcare experiences [1, 6, 34, 35]. Existing reliable assessments for examining provider bias mainly rely on self-report measures with only a few contextualizing the practitioner within the workplace to include structural and systematic factors [36,37,38,39,40].

Health inequities and disparities resulting from stigma and bias

Numerous studies have illustrated how a longstanding history of systemic racism has influenced health disparities [41, 42]. The American Heart Association has linked poor cardiovascular health to self-reported experiences of racism, and literature has documented poor health outcomes for Black women compared to White women in many areas, including but not limited to pre/post-natal, mental health, and pain outcomes [43, 44]. This has been especially true for Black women with a history of substance misuse [45, 46]. Health disparities for gender minorities are alarming as well. The 2015 U.S. Transgender Survey report revealed that transgender and gender diverse (TGD) individuals (1) avoid needed medical services because they have been vastly mistreated in healthcare settings and (2) experience grave occupational injustices in society such as lacking access to restrooms and experiencing exorbitant rates of physical and sexual violence because of their gender identity and/or presentation [6]. Fear of discrimination has been linked to higher rates of anxiety, depression, suicide attempts, social isolation, and occupational marginalization [47]. Moreover, fear of discrimination during healthcare encounters reduces health behaviors to prevent and manage chronic conditions [6]. In addition, stigmatized health conditions such as substance use disorder (SUD) continue to plague the United States despite numerous accessible, evidence-based practices. Literature highlights the impact of stigma on substance users’ utilization of healthcare and recovery services, treatment engagement, and willingness to be honest with providers [34].

Novel use of an existing evidence-based approach: advantages over other methods

The importance and innovation of IDEAS as an occupational therapy intervention lie in its ability to evoke empathy, understanding, and awareness in audience members by connecting them not to abstract, dehumanized stories, but rather to the stories of people present during IDEAS as panel speakers. Literature suggests that explicit bias is largely absent in health care, but that implicit biases continue to drastically impede health outcomes for marginalized groups [42]. Meineke [4], through a cognitive science lens, describes how and why theatre-based approaches may succeed in reducing implicit biases of audiences. The team’s prior work lends further evidence to Meineke’s findings; in prior studies of IDEAS, a significant decrease in stigma beliefs of healthcare providers was found from baseline (M = 79.93, SD = 12.62) to follow-up (M = 69.74, SD = 12.77) measured by the Acceptance and Action Questionnaire—Stigma (AAQ-S); t(41) = 8.18, p < 0.0001, d = 0.80 [9]. The IDEAS intervention is advantageous compared to other stigma reduction approaches within health care because it can engage and affect its viewers more readily than standard cultural competence training programs. Its use of narrative theatre, which offers audiences an opportunity to immerse themselves in the stories of others and develop cultural humility, can cultivate open-mindedness and changed attitudes where more overtly persuasive messaging triggers cognitive resistance [8, 48, 49].

Lack of adequate measures of provider bias

This implementation research provides a critical outline for evaluating the effects of moderating variables on provider bias-targeted trainings, as measures detailed in existing literature examine provider bias within a limited scope by relying heavily on uncontextualized self-report assessments. Self-report and some client-report measures have been utilized within healthcare provider cultural competence education; however, these assessments lack recognition of the important role that environmental influences play into practitioner bias and stigma-influenced care provision [35]. Though “organizational support” has been evidenced as a distinguished and critical factor in the measurement of cultural competence, it is not widely validated in existing healthcare provider bias and stigma-related assessments [36,37,38,39,40].


The COVID-19 pandemic has affected site recruitment and implementation of the IDEAS intervention. Several larger hospital networks were established as initial partners; however, with the frequent spikes in COVID-19 patient numbers, these sites retracted from their original timeline for the program, citing the pandemic as higher priority for administration and management. As the goal is to include a diverse cohort of healthcare facilities from across the country, this has restricted or delayed some of the site recruitment options for the IDEAS intervention.

Future study

The resources shared with the internal facilitator provide a checklist for how to deliver IDEAS. We plan to track lessons learned from IDEAS implementation in this study as well as any adaptations or unanticipated changes/alterations that occur and use this information to determine which elements of IDEAS implementation are critical for implementation success. Data will consist of personal communications and observations as well as CFIR post-interview data [9]. These data will inform eventual development of a fidelity scale to allow for future fidelity assessment.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.



Identity Development Evolution and Sharing


Consolidated Framework for Implementation Research


Acceptance and Action Questionnaire – Stigma


Acceptability of Intervention Measure


Feasibility of Intervention Measure


Appropriateness of Intervention Measure


Transgender and gender diverse


Substance use disorder




Standard deviation


Acceptance and Commitment Therapy


Coincidence Analysis


Analysis of variance


  1. American Occupational Therapy Association. Occupational therapy practice framework: domain and process (4th ed.). Am J Occup Ther. 2020;74(2):7412410010.

    Article  Google Scholar 

  2. Wilcock A, Townsend E. Occupational justice. In: Crepeau E, Cohn E, Boyt Schell BA, editors. Willard and Spackman’s occupational therapy. 11th ed. Philadelphia: Lippincott Williams & Wilkins; 2009. p. 192–9.

    Google Scholar 

  3. Wasmuth S, Pritchard K, Milton C, Smith E. A mixed-method analysis of community-engaged theatre illuminates black women’s experiences of racism and addresses healthcare inequities by targeting provider bias. Inquiry. 2020;57:1–10.

    Article  Google Scholar 

  4. Wasmuth S, Leonhardt B, Pritchard K, Li CY, DeRolf A, Mahaffey L. Supporting occupational justice for transgender and gender-nonconforming people through narrative-informed theater: a mixed-methods feasibility study. Am J Occup Ther. 2021;75:1–12.

    Article  Google Scholar 

  5. Charon R, Hermann N, Devlin MJ. Close reading and creative writing in clinical education: teaching attention, representation, and affiliation. Acad Med. 2016;91:345–50.

    Article  PubMed  PubMed Central  Google Scholar 

  6. James SE, Herman JL, Rankin S, Keisling M, Mottet L, Anafi M. The report of the 2015 U.S. transgender survey. Washington, DC: National Center for Transgender Equality; 2016.

    Google Scholar 

  7. Brottman MR, Char DM, Hattori RA, Heeb R, Taff SD. Toward cultural competency in health care: a scoping review of the diversity and inclusion education literature. Acad Med. 2020;95:803–13.

    Article  PubMed  Google Scholar 

  8. Solomon DL. Ideological battlegrounds: entertainment to disarm divisive propaganda. Lanham: Rowman & Littlefield; 2018.

    Google Scholar 

  9. Levin ME, Luoma JB, Lillis J, Hayes SC, Vilardaga R. The Acceptance and Action Questionnaire–Stigma (AAQ-S): Developing a measure of psychological flexibility with stigmatizing thoughts. J Contextual Behav Sci. 2014;3:21–6.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Zestcott CA, Blair IV, Stone J. Examining the presence, consequences, and reduction of implicit bias in health care: a narrative review. Group Process Intergroup Relat. 2016;4:528–42.

    Article  Google Scholar 

  11. Akrami N, Ekehammar B, Bergh R. Generalized prejudice: common and specific components. Psychol Sci. 2011;22:57–9.

    Article  PubMed  Google Scholar 

  12. Hayes SC, Luoma JB, Bond FW, Masuda A, Lillis J. Acceptance and Commitment Therapy: model, processes and outcomes. Behav Res Ther. 2006;44(1):1–25.

    Article  PubMed  Google Scholar 

  13. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Mental Health. 2011;38:65–76.

    Article  Google Scholar 

  14. Damschroder L, Hall C, Gillon L, Reardon C, Kelley C, Sparks J, et al. The Consolidated Framework for Implementation Research (CFIR): progress to date, tools and resources, and plans for the future. Implement Sci. 2015;10(1):1–1.

    Article  Google Scholar 

  15. Landes SJ, McBain SA, Curran GM. Reprint of: an introduction to effectiveness-implementation hybrid designs. Psychiatry Res. 2020;283:112630.

    Article  PubMed  Google Scholar 

  16. Damschroder L, Hall C, Gillon L, Reardon C, Kelley C, Sparks J, et al. The Consolidated Framework for Implementation Research (CFIR): progress to date, tools and resources, and plans for the future. Implement Sci. 2015;10(1):A12.

    Article  PubMed Central  Google Scholar 

  17. Keith RE, Crosson JC, O’Malley AS, Cromp D, Taylor EF. Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. Implement Sci. 2017;12(1):1–2.

    Article  Google Scholar 

  18. Ritchie MJ, Parker LE, Kirchner JE. From novice to expert: a qualitative study of implementation facilitation skills. Implement Sci Commun. 2020;1(1):1–2.

    Article  Google Scholar 

  19. Creswell JW. Mapping the field of mixed methods research. Thousand Oaks: SAGE Publications; 2009.

    Google Scholar 

  20. Ho J. Anti-Asian racism, Black Lives Matter, and COVID-19. Japan Forum. 2021;33(1):148–59.

    Article  Google Scholar 

  21. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12:108.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Whitaker RG, Sperber N, Baumgartner M, Thiem A, Cragun D, Damschroder L, et al. Coincidence analysis: a new method for causal inference in implementation science. Implement Sci. 2020;15(1):1–0.

    Article  Google Scholar 

  23. Harris K, Kneale D, Lasserson TJ, McDonald VM, Grigg J, Thomas J. School-based self-management interventions for asthma in children and adolescents: a mixed methods systematic review. Cochrane Database Syst Rev. 2019;1(1):CD011651.

    Article  PubMed  Google Scholar 

  24. Baumgartner M. Parsimony and causality. Qual Quant. 2015;49(2):839–56.

    Article  Google Scholar 

  25. Cragun D, Pal T, Vadaparampil ST, Baldwin J, Hampel H, DeBate RD. Qualitative comparative analysis: a hybrid method for identifying factors associated with program effectiveness. J Mixed Methods Res. 2016;10(3):251–72.

    Article  Google Scholar 

  26. Thiem A. Conducting configurational comparative research with qualitative comparative analysis: a hands-on tutorial for applied evaluation scholars and practitioners. Am J Eval. 2017;38(3):420–33.

    Article  Google Scholar 

  27. Rohlfing I, Zuber CI. Check your truth conditions! Clarifying the relationship between theories of causation and social science methods for causal inference. Sociol Methods Res. 2021;50(4):1623–59.

    Article  Google Scholar 

  28. Rihoux B, Ragin CC. Configurational comparative methods: Qualitative Comparative Analysis (QCA) and related techniques. Thousand Oaks: SAGE Publications; 2008.

    Google Scholar 

  29. Cohen DJ, Sweeney SM, Miller WL, Hall JD, Miech EJ, Springer RJ, et al. Improving smoking and blood pressure outcomes: the interplay between operational changes and local context. Ann Family Med. 2021;19(3):240–8.

    Article  Google Scholar 

  30. Palinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health. 2019;1(40):423–42.

    Article  Google Scholar 

  31. Petrik AF, Green B, Schneider J, Miech EJ, Coury J, Retecki S, et al. Factors influencing implementation of a colorectal cancer screening improvement program in community health centers: an applied use of Configurational Comparative Methods. J Gen Intern Med. 2020;35(2):815–22.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Coury J, Miech EJ, Styer P, Petrik AF, Coates KE, Green BB, et al. What’s the “secret sauce”? How implementation variation affects the success of colorectal cancer screening outreach. Implement Sci Commun. 2021;2(1):1–1.

    Article  Google Scholar 

  33. Hickman SE, Miech EJ, Stump TE, Fowler NR, Unroe KT. Identifying the implementation conditions associated with positive outcomes in a successful nursing facility demonstration project. Gerontologist. 2020;60(8):1566–74.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Adams RS, Garnick DW, Harris AH, Merrick EL, Hofmann K, Funk W, et al. Assessing the postdeployment quality of treatment for substance use disorders among Army enlisted soldiers in the Military Health System. J Subst Abuse Treat. 2020;114:108026.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Chae D, Kim J, Kim S, Lee J, Park S. Effectiveness of cultural competence educational interventions on health professionals and patient outcomes: a systematic review. Jpn J Nurs Sci. 2020;17(3):e12326.

    Article  PubMed  Google Scholar 

  36. Gonzalez E, Sperandio KR, Mullen PR, Tuazon VE. Development and initial testing of the Multidimensional Cultural Humility Scale. Measure Eval Counsel Devel. 2021;54(1):56–70.

    Article  Google Scholar 

  37. Schim SM, Doorenbos AZ, Miller J, Benkert R. Development of a cultural competence assessment instrument. J Nurs Meas. 2003;11(1):29–40.

    Article  PubMed  Google Scholar 

  38. Lin MH, Chang TH, Lee YH, Wang PY, Lin LH, Hsu HC. Developing and validating the nursing cultural competence scale in Taiwan. PLoS One. 2019;14(8):e0220944.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Campinha-Bacote J. Inventory for assessing the process of cultural competence among healthcare professionals-revised (IAPCC-R). The Process of Cultural Competence in the Delivery of Healthcare Services. 2003;4.

  40. Suarez-Balcazar Y, Balcazar F, Taylor-Ritzler T, Portillo N, Rodakowsk J, Garcia-Ramirez M, et al. Development and validation of the cultural competence assessment instrument: A factorial analysis. J Rehabil. 2011;77(1):4–13.

    Google Scholar 

  41. Bailey ZD, Krieger N, Agénor M, Graves J, Linos N, Bassett MT. Structural racism and health inequities in the USA: evidence and interventions. Lancet. 2017;389(10077):1453–63.

    Article  PubMed  Google Scholar 

  42. Ferdinand KC, Yadav K, Nasser SA, Clayton-Jeter HD, Lewin J, Cryer DR, et al. Disparities in hypertension and cardiovascular disease in blacks: the critical role of medication adherence. J Clin Hypertens. 2017;19(10):1015–24.

    Article  Google Scholar 

  43. Havranek EP, Mujahid MS, Barr DA, Blair IV, Cohen MS, Cruz-Flores S, et al. Social determinants of risk and outcomes for cardiovascular disease: a scientific statement from the American Heart Association. Circulation. 2015;132(9):873–98.

    Article  PubMed  Google Scholar 

  44. Maina IW, Belton TD, Ginzberg S, Singh A, Johnson TJ. A decade of studying implicit racial/ethnic bias in healthcare providers using the implicit association test. Soc Sci Med. 2018;1(199):219–29.

    Article  Google Scholar 

  45. Sacks TK. Performing Black womanhood: a qualitative study of stereotypes and the healthcare encounter. Crit Public Health. 2018;28(1):59–69.

    Article  Google Scholar 

  46. Wallace ME, Green C, Richardson L, Theall K, Crear-Perry J. “Look at the whole me”: a mixed-methods examination of Black infant mortality in the US through women’s lived experiences and community context. Int J Environ Res Public Health. 2017;14(7):727.

    Article  PubMed Central  Google Scholar 

  47. Parr NJ, Howe BG. Heterogeneity of transgender identity nonaffirmation microaggressions and their association with depression symptoms and suicidality among transgender persons. Psychol Sex Orientat Gend Divers. 2019;6(4):461.

    Article  Google Scholar 

  48. Meineck P. Theatrocracy: Greek drama, cognition, and the imperative for theatre. New York: Taylor & Francis; 2017 Jul 14.

    Book  Google Scholar 

  49. Agner J. Moving from cultural competence to cultural humility in occupational therapy: a paradigm shift. Am J Occup Ther. 2020;74(4):1–7.

    Article  Google Scholar 

Download references


The EXTEND QUERI Mentorship Core provided invaluable guidance throughout the development of this study.


This study is funded by the American Occupational Therapy Foundation Implementation Research Grant; therefore, data collection and analysis are primarily focused on outcomes pertaining to occupational therapist participants. The funder had no role in the design of the study, data collection, analysis, or interpretation of results, nor in writing the manuscript.

Author information

Authors and Affiliations



SW, EM, and DB were major contributors in the overall design and implementation of the study. SW and EM designed data analyses methods and were responsible for analyses following data collection. JB, CH, AH, CS, and CA played a central role in refining study protocol, initiating recruitment efforts, and assisting in data analyses. SW and JB were major contributors in writing this manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sally Wasmuth.

Ethics declarations

Ethics approval and consent to participate

All participating sites receive a recruitment flyer, a study information sheet to distribute to staff occupational therapy practitioners, and an electronic survey item describing their participation with a box to check indicating consent to participate. All procedures in this study were approved by the first author’s university Institutional Review Board, number 11234.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


Appendix 1

Table 3

Table 3 IDEAS implementation E-resource manual for internal facilitator

Appendix 2 Shared assumptions

Adapted from

  • No one is exempt from the toxicity of institutional racism, ableism, classism, cis-genderism, heterosexism, colonialism, and ethnocentrism, and that have pervaded our culture

  • People have been discouraged from seeing the toxicity of our culture; we therefore must take actions to learn to see our culture and how grave inequities have come to be perceived as normal (but should not be!)

  • We can have good intentions and still have a hurtful or damaging impact

  • Unlearning oppressive attitudes will take a lifetime. Most of us have been struggling with these issues for years and years already. None of us are beginners and none of us have perfect clarity. This work is a journey without endpoint. This work is a lifelong process.

  • We operate in a culture that assumes justice, equity, and civil rights efforts are "for" non-majority communities and people and that the goal is to “help them”, which perpetuates the notion that the majority population is the default “normal” and doesn’t need to change. We believe this worldview perpetuates the problem(s).

  • We are not here to “fix” each other; rather our work is to love ourselves into who we are, knowing how conditioned we all are by white supremacy.

  • We have to believe in the possibilities of creating the world we want to see by walking our talk and learning from our mistakes.

  • None of this is easy and we have to do it anyway.

Shared agreements

Adapted from

Let us provide feedback on actions and opinions, not motivation or character: In this sort of environment, it is easy to lose focus on ideas and actions; instead, we start questioning people’s motivations and attacking their character. Instead of “I disagree with you on blah blah,” it becomes “You only say blah blah because you are blah blah and you want blah blah because all you blah blahs are so-and-so!” Let us check ourselves when we do that. It never does any good. People never respond well when their character or motivations are being questioned. It shuts down conversations, or worse, generates in-kind retaliation and perpetuates a vicious cycle.

Let us assume the best intentions, but also address impact: In trying to be helpful, many of us say and do things that may not end up very helpful. Let us assume everyone means well and let us be patient with one another. But that does not override the fact that our well-intentioned actions may be hurtful to others. When our actions are damaging, we must address them, whether or not they were well-intended.

Let us not assume we completely understand one another’s reality: Right now, many communities are hurting. Latino kids are getting “Build that wall!” chanted at them. Women wearing hijabs are attacked, or threatened to be set on fire. Black university students are getting pictures of lynchings texted to them. Many of people from marginalized communities do not feel safe anymore, and some of us are not sure we will ever be able to. If we do not come from these communities, let us not assume there is an equivalence with our own background. Let us not assume we know what another person is going through. I have seen too many conversations where people are dismissive of one another’s experiences. Let us agree not to do that.

Let us be gracious in accepting difficult feedback: When someone gives us feedback, especially those who come from backgrounds completely different from ours, it is a valuable gift. And it often comes with a price for the person giving the feedback, as they frequently must share vulnerable personal information in order to bring some “credibility” to their feedback.

Let us forgive ourselves and one another: We are all going to make mistakes; so many of us are terrified that making a mistake means we are a racist, or sexist, or ableist, or transphobe, or ageist. In this field, being called those things when many of us are actively fighting for social justice cuts at the core of our identities. Even the thought of anyone even suspecting us to be any of those things hurts deeply. When someone makes a mistake, it does not mean they are a bad person. We have to forgive ourselves when we make mistakes, and we must try to forgive others.

Let us not give up on one another: We cannot build a strong community if we cannot converse with one another, have hard conversations, disagree once a while, and learn from one another. Let us stay and let us talk through the hard stuff. And if we do need to take a break, that is understandable, but let us leave the door open to come back.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wasmuth, S., Belkiewitz, J., Bravata, D. et al. Protocol for evaluating external facilitation as a strategy to nationally implement a novel stigma reduction training tool for healthcare providers. Implement Sci Commun 3, 88 (2022).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: