Skip to main content

An exposure-based implementation strategy to decrease clinician anxiety about implementing suicide prevention evidence-based practices: protocol for development and pilot testing (Project CALMER)

Abstract

Background

Clinicians often report that their own anxiety and low self-efficacy inhibit their use of evidence-based suicide prevention practices, including gold-standard screening and brief interventions. Exposure therapy to reduce clinician maladaptive anxiety and bolster self-efficacy use is a compelling but untested approach to improving the implementation of suicide prevention evidence-based practices (EBPs). This project brings together an interdisciplinary team to leverage decades of research on behavior change from exposure theory to design and pilot test an exposure-based implementation strategy (EBIS) to target clinician anxiety to improve suicide prevention EBP implementation.

Methods

We will develop, iteratively refine, and pilot test an EBIS paired with implementation as usual (IAU; didactic training and consultation) in preparation for a larger study of the effect of this strategy on reducing clinician anxiety, improving self-efficacy, and increasing use of the Columbia Suicide Severity Rating Scale and the Safety Planning Intervention in outpatient mental health settings. Aim 1 of this study is to use participatory design methods to develop and refine the EBIS in collaboration with a stakeholder advisory board. Aim 2 is to iteratively refine the EBIS with up to 15 clinicians in a pilot field test using rapid cycle prototyping. Aim 3 is to test the refined EBIS in a pilot implementation trial. Forty community mental health clinicians will be randomized 1:1 to receive either IAU or IAU + EBIS for 12 weeks. Our primary outcomes are EBIS acceptability and feasibility, measured through questionnaires, interviews, and recruitment and retention statistics. Secondary outcomes are the engagement of target implementation mechanisms (clinician anxiety and self-efficacy related to implementation) and preliminary effectiveness of EBIS on implementation outcomes (adoption and fidelity) assessed via mixed methods (questionnaires, chart-stimulated recall, observer-coded role plays, and interviews).

Discussion

Outcomes from this study will yield insight into the feasibility and utility of directly targeting clinician anxiety and self-efficacy as mechanistic processes informing the implementation of suicide prevention EBPs. Results will inform a fully powered hybrid effectiveness-implementation trial to test EBIS’ effect on implementation and patient outcomes.

Trial registration

Clinical Trials Registration Number: NCT05172609. Registered on 12/29/2021.

Background

Designing and testing scalable implementation strategies to increase the use of evidence-based screening [1], assessment [2], and brief suicide prevention interventions [3, 4] are critical next steps for advancing suicide prevention efforts. There are about 800,000 deaths by suicide globally annually [5]. Numerous studies demonstrate the effectiveness of evidence-based practices (EBPs) for suicide screening, assessment, and intervention (or “SSAIs”) with individuals at risk for suicide [6]. Despite the evidence supporting SSAIs like the Columbia Suicide Severity Rating Scale [C-SSRS] for screening and assessment [1] and the Safety Planning Intervention [SPI] for brief intervention [3], they remain underused in routine clinical settings.

Leading implementation scientists have argued that implementation strategies that address specific facilitators and barriers to EBP delivery will be more successful than general strategies [7,8,9,10]. Choosing effective strategies requires identifying the specific mechanisms that affect implementation success [11]. To date, implementation strategies to improve the delivery of suicide prevention EBPs have not majorly attempted to target specific mechanisms [10].

One possible set of mechanisms that inhibit the uptake of suicide prevention EBPs is clinician anxiety and low self-efficacy related to practice use [12]. Clinicians can experience intense anxiety when patients endorse suicide risk [13] due to fear of liability, fear of miscalculating patient risk, uncertainty about how to intervene, and fear of having inadequate time to intervene [14]. Anticipatory anxiety about suicide risk paradoxically can lead to inadequate assessment or avoidance of screening entirely [15,16,17], consistent with maladaptive avoidance processes commonly associated with anxiety (see Fig. 1).

Fig. 1
figure 1

Conceptual model of maladaptive anxious avoidance applied to suicide prevention EBPs, adapted from Becker-Haimes et al. ([12]; permission given)

A robust qualitative literature delineates how clinicians’ fear and anxiety about their ability to support patients in managing suicide risk can lead to them avoiding EBPs [18], sometimes with detrimental consequences [17]. Emerging quantitative data support these findings. For example, a survey of 300 practicing mental health clinicians found that nearly half endorsed high anxiety about patient death by suicide, even though most of the sample reported high to very high knowledge of how to work with suicidal patients [19]. Other studies find that many providers fear, wrongly, that asking about suicide will increase patient suicide risk [20], which leads them to avoid using SSAIs or employ “shut down” language that minimizes the potential for actual suicidal risk to be disclosed [21]. Clinician discomfort working with patients at risk for suicide also negatively affects patient experience and their likelihood of disclosure, due to patient perceptions that their clinician would be concerned about liability and because they were worried about the clinician’s adverse emotional response to disclosures [22, 23].

High anxiety about SSAIs also can be driven by low self-efficacy or lack of confidence in one’s ability to effectively intervene to mitigate suicide risk [2, 19]. A recent review noted that self-efficacy potentially mediates the relationship between training and implementation outcomes [24]. Higher self-efficacy to intervene with patients at risk for suicide is an important independent predictor of SSAI use [25], perhaps because SSAI delivery is complex and can require intensive support to implement correctly [26]. For example, LoParo and colleagues [15] found a link between greater self-efficacy in SSAI skills and more use of EBPs for SSAIs in a sample of 137 community mental health clinicians.

Theoretical models of maladaptive anxiety posit that low self-efficacy can increase anticipatory anxiety and thus the likelihood of maladaptive avoidance [27]. For example, low clinician self-efficacy for SSAI delivery can prompt decisions to hospitalize patients out of fear that either a patient will attempt suicide or that there will be legal repercussions of inaction, rather than because of a patient’s true risk [28,29,30]. These unnecessary hospitalizations can have iatrogenic effects and are costly [31], yet likely reinforce clinicians’ beliefs that hospitalizing a client is the safest path forward, as the clinician does not experience sending a high-risk patient home with a safety plan in place where safety is not guaranteed. This perpetuates the likelihood that clinicians will continue to experience anxiety about working with individuals at risk for suicide and continue to over-refer to crisis centers.

To our knowledge, no efforts have targeted clinician anxiety about SSAI use other than by offering training [19], which many studies have established as insufficient to change behavior [32, 33]. Guided by a conceptual model delineating how clinician anxious avoidance drives suboptimal implementation [12] (see Fig. 1), we will develop and test an implementation strategy grounded in principles of learning theory and exposure therapy. Exposure therapy is based on the established theory that anxiety is maintained and worsened through avoidance of feared stimuli [34]. It involves guiding individuals to gradually confront and increase their tolerance of stimuli [35] in four phases: (1) psychoeducation about how maladaptive anxiety influences behavior; (2) assessing individuals’ feared outcomes and ranking ease of possible practice scenarios; (3) repeated facilitated practice in facing fears followed by cognitive debriefing; and (4) independent application to real-life (“relapse prevention”) [36]. It is the key cognitive-behavioral treatment ingredient leading to reduced maladaptive anxiety and increased self-efficacy to engage with feared scenarios [37,38,39,40].

We will design an exposure-based implementation strategy (EBIS) to augment traditional training approaches. The EBIS is intended to directly target and mitigate clinician anxiety related to working with individuals at risk for suicide and bolster clinician self-efficacy to implement SSAIs with their patient population. This approach has been used to mitigate clinician anxiety about delivering exposure therapy itself to anxious patients, with pilot work showing promise for improving exposure therapy use [36], but has not been used to support SSAI implementation, which may be more challenging for clinicians. Figure 2 illustrates how exposure therapy can address target mechanisms.

Fig. 2
figure 2

Facilitating a cycle of approach for SSAI use

This study (Clinician Affect Labeling and Management through Exposure Research; Project CALMER) is one of three exploratory projects through the NIMH-funded Penn Innovation in Suicide Prevention Implementation Research (INSPIRE) Center [41], which brings together experts in psychology, implementation science, health economics, machine learning, health information technology, and psychiatry to address suicide prevention. Project CALMER aims to increase clinicians’ high-fidelity implementation of SSAI by augmenting training and consultation with brief graduated exposure therapy that combines imaginal and in vivo exposures [42] with standardized patients. Specific aims are to:

  • Aim 1: Using participatory design methods, develop the EBIS to reduce clinician anxiety and bolster self-efficacy to deliver SSAI by leveraging exposure-based therapies to manage core clinician fears and anxiety related to suicide in high-risk patient encounters.

  • Aim 2: Iteratively refine the EBIS through pilot field-testing using rapid cycle prototyping with up to 20 clinicians. Participants will receive EBIS training and provide qualitative feedback to inform subsequent refinements.

  • Aim 3: Test acceptability, feasibility, and preliminary effectiveness of the EBIS compared with implementation as usual (IAU) to target SSAI implementation mechanisms and outcomes in a pilot randomized trial with 40 clinicians. Exploratory analyses will examine the impact of both conditions on putative implementation mechanisms (clinician anxiety, self-efficacy) and outcomes (SSAI adoption, fidelity). We hypothesize EBIS will be an acceptable and feasible implementation strategy and that it will engage target implementation mechanisms of clinician anxiety and self-efficacy better than IAU and improve rates of SSAI use relative to IAU.

Methods/design

Study setting

All research activities will take place in the City of Philadelphia. The City of Philadelphia Department of Behavioral Health and Intellectual disAbility Services has prioritized suicide prevention across behavioral health service settings. The City of Philadelphia has selected two suicide prevention EBPs that are of focus in this study: the Columbia Suicide Severity Rating Scale (C-SSRS) and the Safety Planning Intervention (SPI) [3, 43]. The C-SSRS is an assessment tool designed to guide clinicians in assessing individual suicide risk; it includes direct questions clinicians ask to assess past and current suicidal ideation, intent, plan, and behaviors, as well as non-suicidal-self injury. The C-SSRS is well-established and freely available, making it attractive for widespread implementation. The SPI is a brief, single-session intervention shown to reduce the risk of suicide attempts. Clinicians work together with at-risk patients to identify warning signs of future suicidal crises and identify a series of actionable steps patients can take to minimize suicide risk. The SPI has a growing evidence base in support of its effectiveness across settings [3, 44, 45]. Despite this, our pilot work demonstrated that there is variability in the extent to which these EBPs are routinely implemented as intended in the outpatient setting [13]. Specific outpatient sites for participation will be selected in collaboration with our advisory board.

Advisory board

We will conduct the study in collaboration with a diverse set of stakeholders from the Philadelphia region, including mental healthcare providers (e.g., practicing clinicians, medical directors), policy advisors (e.g., members of city government), and community organizations (e.g., leadership from organizations that support families who have experienced suicide loss). We will meet with stakeholders primarily in the first year of the project during the development stages and keep them closely apprised throughout the trial period. Stakeholders will be compensated $50 for each hour of their time.

Conceptual frameworks

First, our conceptual framework of clinician maladaptive anxious avoidance (Figs. 1 and 2) supports our selection of implementation mechanisms for clinician anxiety and self-efficacy, which we hope to target via the EBIS. Second, our measurement battery is guided by the conceptual model employed by the larger Penn INSPIRE Center that integrates principles of contextual determinants of implementation (e.g., organizational policies and procedures) [46] and behavior change theory (e.g., intentions and their determinants, including norms, attitudes, self-efficacy, knowledge, and beliefs) [47,48,49,50].

Aims 1 and 2: EBIS strategy development and refinement

Aims 1 and 2 will be completed concurrently. We will develop a prototype of EBIS based on several key components of exposure treatment. EBIS is informed by the latest science in exposure theory [35], borrowing heavily from brief exposure-based treatments that can be delivered in a single session [51,52,53]. We anticipate that EBIS will occur in four phases that map on to standard exposure therapy practice for patients with anxiety disorders: psychoeducation, assessment, practice, and relapse prevention; however, the final version will be based on Aim 1. Table 1 presents each anticipated component; each is also described more below.

Table 1 Anticipated EBIS components and rationale

Psychoeducation

In exposure therapy, psychoeducation is a critical step toward obtaining buy-in to engage in exposure practice. Typically, psychoeducation includes information about both adaptive and maladaptive anxiety. We will provide brief information about the role of anxiety in SSAI delivery, explain the “cycle of avoidance” that can occur when one experiences anxiety related to patient suicide risk [12], and discuss clinical implications of clinician avoidance of SSAI delivery. We will present common ways that anxiety can present in clinicians specifically as it relates to suicide prevention work and encourage clinicians to reflect on the ways in which anxiety may be likely to present for them.

Assessment/hierarchy-building

We anticipate using the “post-it method” in a group format for assessment and hierarchy-building. Clinicians will be presented with 8–12 flashcards that delineate various fears they may endorse about interacting with a suicidal patient that range in intensity (e.g., “patient endorses passive suicidal ideation” to “patient endorses a suicide plan”) and fears related to outcomes of intervention (e.g., “patient completes SPI but attempts suicide after leaving clinic,” “I am sued because my patient attempted suicide”). Clinicians will rank-order each fear with respect to how anxiety-provoking it is and rate their anticipated anxiety using the Subjective Units of Distress Scale (SUDS) for each scenario. We will select up to three of the most-feared scenarios for exposure practice at the conclusion of didactic training.

Guided practice: in vivo and imaginal exposure [54]

Clinicians will engage in either in vivo practice or imaginal exposure with a trained actor (research assistant; RA) based on feared scenarios selected in the step above. Consistent with standard practices [54], RAs or other members of the research team will guide clinicians to (1) identify core fears and anticipated anxiety via SUDS ratings, (2) engage in targeted practice to violate assumptions of core fears and track SUDS changes, and (3) engage in targeted cognitive debriefing to enhance coping self-efficacy. RAs will be guided by a previously developed “exposure checklist” that will be adapted for the EBIS to ensure adherence to the EBIS and will be closely supervised by licensed clinicians.

Relapse prevention/application to clinical practice

At the end of initial training, clinicians will be guided by the trainer(s) to summarize their experience and set intentions for managing anxiety they may experience in future clinical practice. Additional content in this EBIS domain will be embedded into follow-up consultation calls led by training staff and will include targeted discussion of clinician anxiety and avoidance alongside standard barriers in clinical practice to facilitate the maintenance of gains.

We will present the initial EBIS prototype to our Advisory Board for feedback and refinement, using a modified process of co-design [55, 56]. Meetings will be facilitated by project leads to encourage all stakeholders to provide feedback on general components of the strategy and how to structure each component for optimal results. Questions posed to the Board for EBIS components will be both general (e.g., “Tell us your thoughts on the psychoeducation component?”, “What exposure tasks do you think will be most important?”) and specific (e.g., “How can we make the psychoeducation component more engaging?”, “How can we optimize imaginal exposure to a feared outcome of being sued?”) with the goal of designing for acceptability and feasibility across diverse settings. We will integrate all feedback into a revised prototype that we will present to the Advisory Board in a second meeting.

After the second Board meeting, we will further refine EBIS in a pilot field test using rapid cycle prototyping with up to 20 mental health clinicians recruited from local outpatient community clinics.

Prototyping procedures

In each round of prototyping, we will recruit clinicians to participate in pilot EBIS versions. Clinicians will be recruited from local community mental health clinics; before recruiting, we will reach out to clinic leadership for approval to contact clinicians at each location. Clinicians will be eligible if they are practicing mental health clinicians in the City of Philadelphia who provide direct mental health services to a treatment-seeking population. Clinician participants will be excluded if they do not see any patients that are at risk for suicide (e.g., they screen out all high-risk patients for their individual practice). Participants will not be excluded based on sex, demographics, and/or experience. All participants will complete informed consent prior to study procedures via a scheduled phone or videoconference meeting with a trained research staff member.

We anticipate conducting 3–4 rounds of prototyping. In each round, consenting clinicians participating will receive some or all of the 4–6 h of training and EBIS content (exact length dependent on the EBIS design) either virtually or in person; full training content will include all gold-standard didactic and experiential practice typically included in trainings for the C-SSRS and the SPI, with the EBIS incorporated. Pilot trainings will be conducted by content experts on the research team and take place in small groups of 2–5 clinicians. Given the training length, it is anticipated training will occur up to over 2–3 days. We will video record all trainings to allow for quality checks. Video recordings will be reviewed by research staff after each training to identify “stuck points” (e.g., portions of the training that do not seem to flow smoothly) and areas where a clinician expresses confusion or exhibits other reactions to the EBIS components. These components will be flagged for review, discussion, and potential refinement between cycles.

Within approximately 1 week of the training, participants will complete a qualitative interview (45–60 min) with a member of the study team. Interviews will be structured around the clinician’s experience of and feedback on the strategy. Specific constructs include EBIS acceptability and feasibility and their perceived anxiety and self-efficacy about delivering the C-SSRS and SPI. We will query clinicians about the organizational policies and procedures they believe should be in place for the EBIS to be successful; information from this component of the qualitative interview will inform the development of an organizational policies checklist to be included in the Aim 3 trial. Clinicians will be provided with continuing education (CE) credits at no cost to them for participating and $75 for completing the follow-up feedback interview.

After each prototyping round (~ 2–5 per round), we will present information gathered, challenges identified, and suggested refinements to the EBIS to our Advisory Board to inform iterative changes. This process will repeat until we reach thematic saturation and have a version of the EBIS ready for testing in Aim 3.

Aim 3: Pilot randomized clinical trial: implementation as usual (IAU) vs. IAU + EBIS

Aim 3 will test the acceptability, feasibility, and preliminary effectiveness of IAU + EBIS to target implementation mechanisms and outcomes relative to gold-standard IAU in a randomized pilot trial.

Aim 3 procedures

We anticipate partnering with 5 outpatient clinics to enroll 40 clinicians in the pilot trial; additional clinics will be recruited to participate as needed. Inclusion and exclusion criteria are identical to Aim 2. Sample size was selected in anticipation of a small to medium effect size with respect to anxiety reduction and increases in self-efficacy (target mechanisms) between the two treatment arms, using established rules of thumb for designing pilot clinical trials [57]. Interested clinicians will complete informed consent prior to study procedures via a scheduled phone, in-person, or videoconference meeting with a trained research staff member. Clinicians who agree to participate will be randomized 1:1 by senior study staff into one of two arms: IAU or the EBIS condition (IAU + EBIS). Participants will be aware of group assignment and informed that participation is voluntary and that they can discontinue participation at any time.

IAU

Gold-standard IAU typically comprises pre-implementation preparation, didactic training, knowledge tests, experiential role plays, ongoing expert consultation, and providing completion certificates to clinicians who complete all training and consultation activities. Pre-implementation preparation will include providing materials (e.g., SPI manual [3], instructions [58], and forms). Didactic training will occur in two parts: (1) C-SSRS screening and assessment and (2) SPI use. Part one will consist of materials we previously developed based on community clinician feedback. Part two will follow established SPI guidelines, including didactic training about SPI rationale and evidence base and experiential practice. After training, clinicians will receive 6 biweekly expert consultation sessions to discuss implementation barriers and practice through role plays.

EBIS

Clinicians randomized to EBIS will receive all IAU elements outlined above plus the EBIS refined in Aims 1 and 2.

Clinical trial data will include clinician responses to self-report questionnaires, chart-stimulated recall, semi-structured interviews, and fidelity role-plays with research staff. Recruitment and retention statistics will capture feasibility metrics. No identifiable data about patients will be collected. All data will be used exclusively for research purposes and administered at the following time points: T1 = baseline/pretraining; T2 = post-training; T3 = 2-week follow-up; T4 = 12-week follow-up. Between T1 and T2, clinicians will complete 4–6 h of suicide prevention training aligned with their assigned training condition for CE credit. Between T2 and T4, clinicians in each condition will receive a biweekly consultation protocol specific to their condition for 12 weeks. Clinicians will be compensated $50 for T1 measures, $25 for T2 measures, $100 for T3, and $125 for T4. This compensation protocol was selected to promote participant retention based on past experiences conducting community-based implementation trials.

Table 2 shows the assessment battery and timeline for Aim 3. Primary trial outcomes include EBIS acceptability and feasibility, measured both quantitatively and qualitatively. Secondary outcomes include target implementation mechanisms of clinician anxiety and self-efficacy, as well as exploratory implementation outcomes. Specific measures are described below. All measures will be administered to all clinicians enrolled in the trial.

Table 2 Study timeline for Aim 3 participant procedures

Acceptability: At T2 and T4, we will administer the Acceptability of Intervention Measure (AIM), a 4-item, psychometrically validated measure that measures the extent to which stakeholders believe an implementation strategy (in this case, IAU or EBIS) is acceptable [59]. Qualitative questions (see below) also will assess the perceived acceptability of assigned condition.

Feasibility: At T2 and T4, we will administer the Feasibility of Intervention Measure (FIM), which is a 4-item, psychometrically validated measure that measures the extent to which stakeholders perceive an implementation strategy (in this case, IAU or EBIS) is feasible [59]. We will rigorously track study attrition and note retention statistics at each timepoint. Qualitative questions (see below) will also assess the perceived feasibility of the assigned condition.

Clinician anxiety: We will measure anxiety specific to SSAI use and broad clinician anxiety. Our primary anxiety outcome will rely on the Subjective Units of Distress Scale (SUDS). The SUDS is a one-item, 10-point rating scale of perceived distress, commonly used to guide exposure therapy. This will be administered at T1–T4 separately for clinician anxiety about screening, assessment, and intervention. Clinicians also will complete the Suicidal Patient Comfort Survey, which is a brief, 5-item measure assessing clinician anxiety about interacting with and treating patients with suicidality. This will be administered at T1-T4. Finally, we will index broad clinician anxiety at baseline using the Anxiety Sensitivity Index (ASI-3). The ASI-3 is an 18-item measure that assesses an individual’s physical, cognitive, or social concerns about anxiety and its manifestations. This will be administered at T1.

Clinician self-efficacy: At all timepoints, we will measure clinician self-efficacy using validated question stems from behavioral science designed to measure a clinician’s own perceptions of whether they believe they have the skills and abilities to perform a task on a 7-point scale; these question stems are intended to be adapted to any behavior of interest [60]. Clinicians will report separately on their self-efficacy with respect to suicide screening, assessment, and intervention. Questions about screening apply to all patients; questions about assessment and intervention specifically reference clinician self-efficacy related to patients who screen positive for suicide risk.

Clinician SSAI use: We will use Chart Stimulated Recall (CSR), an established technique for examining clinician decision-making and clinical processes beyond what can be determined from chart review or self-report alone, to index SSAI use [61, 62]. We previously adapted CSR methods to measure SSAI use [13, 63]. A trained research team member will review the clinician’s caseload with them for the past clinic week and ask brief questions (no more than 5 min) related to the clinician’s suicide-related practices (e.g., Did you conduct a screen for suicide risk? How did you screen for risk? If risk was present, was a full C-SSRS/SPI administered?). This will be administered at T1, T3, and T4. No identifiable patient information is collected during the CSR. Clinicians are instructed to refrain from sharing any identifiable patient details.

Clinician SSAI fidelity: We will use a standardized role-play paradigm to index clinician SSAI fidelity at T3 and T4. Participants will receive a vignette and prepare for a 45–60-min role play, during which they will be asked to complete an SPI with a patient who was determined to be at risk for suicide following C-SSRS administration with a trained actor. This paradigm is a training method that provides a standardized opportunity for assessing skills and competencies [64]. The intent is to make the information in the clinical scenarios equally challenging in conducting the SPI among all standardized patients [65]. Role plays will be audio-recorded and coded for competence with the Safety Planning Intervention Rating Scale (SPIRS) [66]. Coders will be masked to the clinician study condition.

Additional mechanisms of interest: We will measure SSAI knowledge, beliefs, attitudes, norms, and intentions, and organizational policies and procedures that may facilitate implementation success. Knowledge will be indexed via a subscale of the Short Survey on Knowledge, Self-Confidence, and Attitudes Towards Suicidal Behavior to assess a clinician’s perceived knowledge about suicide and how to intervene. Beliefs will be assessed using an established 12-item questionnaire designed to assess clinician beliefs about the use of safety planning interventions in clinical practice with suicidal patients [67]. Attitudes, norms, and intentions will be measured via validated questions stemming from behavioral science using items on a 7-point scale [60]. For each of these constructs, we will ask clinicians to report separately on perceptions related to screening, assessment, and intervention, as appropriate. For attitudes, clinicians will rate items asking about the perceived clinical utility of each SSAI component from “strongly disagree” to “strongly agree”. For norms, clinicians will rate their perception of how other clinicians like them engage in SSAI use and the extent to which they feel their clinical supervisor will approve of them using each SSAI component from “strongly disagree” to “strongly agree”. We will measure intentions separately for each SSAI component. These all will be assessed at T1–T4. Organizational policies and procedures will be measured via a brief checklist developed following Aim 2 activities as described above at T1; additional contextual factors influencing implementation success will be measured qualitatively.

Qualitative interview: All clinicians will complete a three-section qualitative interview at T4. Section 1 will query about the acceptability of the implementation condition to which they were assigned (EBIS or IAU) and any barriers that arose in engaging with any study component. Section 2 will focus on how the implementation condition engaged target implementation mechanisms. The final section will be guided by the Consolidated Framework for Implementation Research [46] to query about barriers to SSAI use to identify contextual determinants not adequately addressed by EBIS.

Clinician demographics: At T1, we will administer a brief questionnaire about clinician demographic (e.g., age, sex) and professional characteristics (e.g., years of experience, theoretical orientation), and a question about the clinician’s personal experience with suicide (e.g., “Have you ever had a client die by suicide while under your care?”).

Data analysis

Data screening and missing data analysis will be conducted in accordance with best practices. We will examine the psychometric properties of all scales used to assess constructs of interest (e.g., coefficient alpha) to confirm adequate psychometric performance. We will conduct analyses of baseline variables as a randomization check to ensure comparable baseline characteristics for clinicians in EBIS and IAU. Any differences will be controlled for in subsequent analyses.

Quantitative analyses

This pilot feasibility trial is not adequately powered to detect significant effects by design. Findings will provide key preliminary data to support the feasibility of EBIS and study procedures for a fully powered type 3 hybrid effectiveness-implementation trial. Qualitative and mixed methods analysis will supplement quantitative analyses to answer key questions of interest related to EBIS acceptability, feasibility, and target mechanism engagement.

Hypothesis 1: EBIS will be an acceptable and feasible implementation strategy. We will calculate descriptive statistics on the AIM and FIM measures and compare scores between EBIS and IAU. We will also evaluate EBIS feasibility by calculating the proportion of clinicians that are randomized to EBIS and complete all exposure tasks; we will compare study dropout rates and measure completion rates between conditions.

Exploratory hypotheses 1 and 2: EBIS will engage target implementation mechanisms of clinician anxiety and self-efficacy better than IAU; clinicians randomized to EBIS will show improved SSAI adoption and fidelity relative to IAU. We will examine the effect of condition over time on target mechanisms (anxiety and self-efficacy) and implementation outcomes (mean C-SSRS screening frequency across encounters, proportions of appropriate encounters in which clinicians conduct follow-up assessment with the C-SSRS and SPI use, as measured by CSR, and average SPI fidelity scores on role plays) using repeated measures analysis of covariance (ANCOVA), controlling for organization (given the small sample size and that we will recruit from only a small number of organizations for this pilot, we will not use multilevel analysis).

Qualitative and mixed-methods analyses

Qualitative analysis of interviews will complement quantitative data to examine: (1) the efficacy of EBIS to reduce clinician anxiety and bolster self-efficacy, (2) processes by which EBIS does or does not facilitate SSAI use, and (3) contextual factors in clinicians’ settings that are not addressed by EBIS. Interviews will be transcribed and analyzed via qualitative software, guided by an integrated approach which uses an inductive process of iterative coding to identify recurrent themes, categories, and relationships. We will develop a structured codebook and code for a priori attributes of interest (i.e., the extent to which EBIS engages our target mechanisms of anxiety and self-efficacy) and use modified grounded theory, which provides a systematic and rigorous approach to identifying codes and themes (e.g., to identify additional barriers that arise to SSAI use). Members of the research team will separately code three transcripts to develop an initial coding scheme, which will then be applied to subsequent transcripts. Code reliability will be rigorously monitored; if coder reliability drops below 0.85, each transcript will be coded by two separate raters, using a consensus procedure to reconcile discrepancies for all transcripts. We will employ mixed methods analyses in accordance with best practice recommendations [68,69,70].

Progression criteria

We will examine outcomes of acceptability and feasibility, and target engagement, to determine whether pilot trial results support the notion of progressing to a larger, confirmatory trial. For acceptability and feasibility, we will descriptively compare mean AIM and FIM scores between the two conditions; comparable acceptability and feasibility scores will be suggestive of the potential utility of a full trial. If EBIS scores are significantly lower than those within the IAU condition, this would raise concerns about the acceptability and feasibility of a larger trial. With respect to target engagement, we will be looking to see (1) greater pre- to post-reductions in clinician SUDS and greater pre- to post-increases in self-efficacy scores in the EBIS condition relative to IAU, and/or (2) anxiety reduction and self-efficacy to emerge as a theme in qualitative interviews more commonly and strongly in the EBIS condition relative to IAU.

Data safety and monitoring

The larger NIMH-funded P50 Penn INSPIRE Center has convened a three-member DSMB that consists of an independent group of experts. This DSMB is charged with reviewing study data for data quality and integrity, adherence to the protocol, participant safety, study conduct and progress, and making determinations regarding study continuations, modifications, and suspensions/terminations for all INSPIRE projects, including this exploratory project. DSMB members are independent from any professional or financial conflict of interest with the research project or study investigators. The DSMB will meet at least once during the project.

Discussion

This exploratory project will develop and evaluate the initial acceptability and feasibility of an EBIS specifically designed to reduce clinician anxiety and increase clinician self-efficacy related to the implementation of SSAIs. To our knowledge, this is the first implementation strategy to target these often-cited target mechanisms in suicide prevention implementation. This work is also responsive to the growing recognition of the importance of attending to clinician emotional reactivity within the context of implementation efforts [71] and explicitly identifying how clinician emotional responses may affect implementation processes [12]. Should data prove promising, we will test the refined EBIS in a large-scale confirmatory RCT and formally determine whether the EBIS reduces clinician anxiety and increases clinician self-efficacy and subsequent fidelity to SSAIs. In addition, SSAIs are just one example of evidence-based clinical practices that can be aversive to clinicians; future work will explore the extent to which an EBIS may support the implementation of other anxiety-provoking practices, such as time-out for disruptive behavior disorders [12].

Design considerations

First, we considered utilizing a pilot type 3 hybrid-effectiveness implementation trial in Aim 3; we decided that collecting patient data in this small clinician sample was unlikely to be meaningful. However, we will explore how to collect patient data for future trials. Second, we considered directly observing clinicians delivering the SPI, rather than relying on standardized role plays. Role plays may lead to inflated fidelity, given that clinicians are not bound by the time constraints of real-world clinical practice; however, our prior work has demonstrated that role plays can be representative of direct observation [72]. We will address this concern by having clinicians rate how representative their role play felt of their clinical practice. Third, we considered working with a software company to develop exposure simulations with virtual or augmented reality. However, we thought it important to test the preliminary effectiveness of the exposure-based model before investing in expensive technological features.

Future directions

Future directions include confirmatory trials to test EBIS effectiveness to engage target mechanisms and improve implementation and develop strategies for EBIS scale-up. Future work will include cost-effectiveness analysis of EBIS to support its adoption widely in community mental health and other clinical settings where providers are likely to encounter individuals at risk for suicide (e.g., schools, primary care) and other provider types who are asked to work with individuals at-risk for suicide (e.g., nurse practitioners). Future work also will include an examination of the timing of an EBIS in a clinician’s career trajectory to determine if there are optimal developmental stages (e.g., during graduate training, immediately post-graduation) that optimize response. We will also explore how an EBIS can support implementation of other EBPs that can elicit strong emotional reactions in clinicians that lead to underutilization of best practices (e.g., the use of time-out for disruptive behavior disorders).

Conclusions

Outcomes will yield insight into the feasibility and importance of directly targeting clinician anxiety and self-efficacy as mechanistic processes informing suicide prevention EBP implementation. Should this study produce promising results, this work will lead to large-scale testing of an EBIS to promote the use of gold-standard suicide prevention EBPs across diverse contexts.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study. The final dataset will be shared to the NIMH Data Archive, consistent with the terms of the grant award.

Abbreviations

AIM:

Acceptability of Intervention Measure

ANCOVA:

Repeated measures analysis of covariance

ASI-3:

Anxiety Sensitivity Index

CALMER:

Clinician Affect Labeling and Management through Exposure Research

C-SSRS:

Columbia Suicide Severity Rating Scale

CSR:

Chart Stimulated Recall

DSMB:

Data and Safety Monitoring Board

EBIS:

Exposure-based implementation strategy

EBPs:

Evidence-based practices

FIM:

Feasibility of Implementation Measure

IAU:

Implementation as usual

INSPIRE:

Innovation in Suicide Prevention Implementation Research

RA:

Research assistant

RCT:

Randomized controlled trial

SPI:

Safety Planning Intervention

SPIRS:

Safety Planning Intervention Rating Scale

SSAIs:

Suicide screening, assessment, and intervention

SUDS:

Subjective Units of Distress Scale

References

  1. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011. https://doi.org/10.1176/appi.ajp.2011.10111704.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Fowler JC. Suicide risk assessment in clinical practice: pragmatic guidelines for imperfect assessments. Psychotherapy. 2012. https://doi.org/10.1037/a0026148.

    Article  PubMed  Google Scholar 

  3. Stanley B, Brown GK. Safety planning intervention: a brief intervention to mitigate suicide risk. Cogn Behav Pract. 2012. https://doi.org/10.1016/j.cbpra.2011.01.001.

    Article  Google Scholar 

  4. Turecki G, Brent DA, Gunnell D, O’Connor RC, Oquendo MA, Pirkis J, et al. Suicide and suicide risk. Nat Rev Dis Primer. 2019. https://doi.org/10.1038/s41572-019-0121-0.

    Article  Google Scholar 

  5. Ritchie H, Roser M, Ortiz-Ospina E. Suicide. Our world data. 2015. http://ourworldindata.org/suicide. Accessed 27 Oct 2022.

  6. Gordon JA, Avenevoli S, Pearson JL. Suicide prevention research priorities in health care. JAMA Psychiat. 2020. https://doi.org/10.1001/jamapsychiatry.2020.1042.

    Article  Google Scholar 

  7. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019. https://doi.org/10.3389/fpubh.2019.00003.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017. https://doi.org/10.1007/s11414-015-9475-6.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Williams NJ, Beidas RS. Annual research review: the state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field. J Child Psychol Psychiatry. 2019. https://doi.org/10.1111/jcpp.12960.

    Article  PubMed  Google Scholar 

  10. Labouliere CD, Vasan P, Kramer A, Brown G, Green K, Kammer J, et al. “Zero suicide” – a model for reducing suicide in United States behavioral healthcare. Suicidologi. 2018;23(1):22–30.

  11. Frank HE, Kemp J, Benito KG, Freeman JB. Precision implementation: an approach to mechanism testing in implementation research. Adm Policy Ment Health Ment Health Serv Res. 2022. https://doi.org/10.1007/s10488-022-01218-x.

    Article  Google Scholar 

  12. Becker-Haimes EM, Klein CC, Frank HE, Oquendo MA, Jager-Hyman S, Brown GK, et al. Clinician maladaptive anxious avoidance in the context of implementation of evidence-based interventions: a commentary. Front Health Serv. 2022. https://doi.org/10.3389/frhs.2022.833314.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Davis M, Siegel J, Becker-Haimes EM, Jager-Hyman S, Beidas RS, Young JF, et al. Identifying common and unique barriers and facilitators to implementing evidence-based practices for suicide prevention across primary care and specialty mental health settings. Arch Suicide Res. 2021. https://doi.org/10.1080/13811118.2021.1982094.

    Article  PubMed  Google Scholar 

  14. Roush JF, Brown SL, Jahn DR, Mitchell SM, Taylor NJ, Quinnett P, et al. Mental health professionals’ suicide risk assessment and management practices: the impact of fear of suicide-related outcomes and comfort working with suicidal individuals. Crisis. 2018. https://doi.org/10.1027/0227-5910/a000478.

    Article  PubMed  Google Scholar 

  15. LoParo D, Florez IA, Valentine N, Lamis DA. Associations of suicide prevention trainings with practices and confidence among clinicians at community mental health centers. Suicide Life Threat Behav. 2019. https://doi.org/10.1111/sltb.12498.

    Article  PubMed  Google Scholar 

  16. LeCloux M, Aguinaldo LD, Lanzillo EC, Horowitz LM. PCP opinions of universal suicide risk screening in rural primary care: current challenges and strategies for successful implementation. J Rural Health. 2021. https://doi.org/10.1111/jrh.12508.

  17. Hendin H, Haas AP, Maltsberger JT, Koestner B, Szanto K. Problems in psychotherapy with suicidal patients. Am J Psychiatry. 2006. https://doi.org/10.1176/appi.ajp.163.1.67.

    Article  PubMed  Google Scholar 

  18. Wilson JL, Uthman C, Nichols-Hadeed C, Kruchten R, Thompson Stone J, Cerulli C. Mental health therapists’ perceived barriers to addressing intimate partner violence and suicide. Fam Syst Health J Collab Fam Healthc. 2021. https://doi.org/10.1037/fsh0000581.

    Article  Google Scholar 

  19. Jahn DR, Quinnett P, Ries R. The influence of training and experience on mental health practitioners’ comfort working with suicidal individuals. Prof Psychol Res Pract. 2016. https://doi.org/10.1037/pro0000070.

    Article  Google Scholar 

  20. Quinnett P. The role of clinician fear in interviewing suicidal patients. Crisis. 2019. https://doi.org/10.1027/0227-5910/a000555.

    Article  PubMed  Google Scholar 

  21. Vannoy SD, Robins LS. Suicide-related discussions with depressed primary care patients in the USA: gender and quality gaps. A mixed methods analysis. BMJ Open. 2011. https://doi.org/10.1136/bmjopen-2011-000198.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Blanchard M, Farber BA. “It is never okay to talk about suicide”: patients’ reasons for concealing suicidal ideation in psychotherapy. Psychother Res. 2020. https://doi.org/10.1080/10503307.2018.1543977.

    Article  PubMed  Google Scholar 

  23. Love HA, Morgan PC. You can tell me anything: disclosure of suicidal thoughts and behaviors in psychotherapy. Psychotherapy. 2021. https://doi.org/10.1037/pst0000335.

    Article  PubMed  Google Scholar 

  24. Shapiro CJ, Watson MacDonell K, Moran M. Provider self-efficacy in delivering evidence-based psychosocial interventions: a scoping review. Implement Res Pract. 2021. https://doi.org/10.1177/2633489520988258.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Lau AS, Lind T, Crawley M, Rodriguez A, Smith A, Brookman-Frazee L. When do therapists stop using evidence-based practices? Findings from a mixed method study on system-driven implementation of multiple EBPs for children. Adm Policy Ment Health Ment Health Serv Res. 2020. https://doi.org/10.1007/s10488-019-00987-2.

    Article  Google Scholar 

  26. Brodsky BS, Spruch-Feiner A, Stanley B. The Zero Suicide model: applying evidence-based suicide prevention practices to clinical care. Front Psychiatry. 2018. https://doi.org/10.3389/fpsyt.2019.00033.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Bouchard S, Gauthier J, Nouwen A, Ivers H, Vallières A, Simard S, et al. Temporal relationship between dysfunctional beliefs, self-efficacy and panic apprehension in the treatment of panic disorder with agoraphobia. J Behav Ther Exp Psychiatry. 2007. https://doi.org/10.1016/j.jbtep.2006.08.002.

    Article  PubMed  Google Scholar 

  28. Shore JH, Hilty DM, Yellowlees P. Emergency management guidelines for telepsychiatry. Gen Hosp Psychiatry. 2007. https://doi.org/10.1016/j.genhosppsych.2007.01.013.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Lang M, Uttaro T, Caine E, Carpinello S, Felton C. Implementing routine suicide risk screening for psychiatric outpatients with serious mental disorders: II. qualitative results. Arch Suicide Res. 2009; https://doi.org/10.1080/13811110902835106.

  30. Simpson S, Stacy M. Avoiding the malpractice snare: documenting suicide risk assessment. J Psychiatr Pract. 2004. https://doi.org/10.1097/00131746-200405000-00008.

    Article  PubMed  Google Scholar 

  31. Ward-Ciesielski EF, Rizvi SL. The potential iatrogenic effects of psychiatric hospitalization for suicidal behavior: a critical review and recommendations for research. Clin Psychol: Sci Pract. 2021. https://doi.org/10.1111/cpsp.12332.

    Article  Google Scholar 

  32. Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol: Sci Pract. 2010. https://doi.org/10.1111/j.1468-2850.2009.01187.x.

    Article  Google Scholar 

  33. Frank HE, Becker-Haimes EM, Kendall PC. Therapist training in evidence-based interventions for mental health: a systematic review of training approaches and outcomes. Clin Psychol Sci Pract. 2020. https://doi.org/10.1111/cpsp.12330.

    Article  Google Scholar 

  34. Abramowitz JS. The practice of exposure therapy: relevance of cognitive-behavioral theory and extinction theory. Behav Ther. 2013. https://doi.org/10.1016/j.beth.2013.03.003.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Craske MG, Kircanski K, Zelikowsky M, Mystkowski J, Chowdhury N, Baker A. Optimizing inhibitory learning during exposure therapy. Behav Res Ther. 2008. https://doi.org/10.1016/j.brat.2007.10.003.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Frank HE, Becker-Haimes EM, Rifkin LS, Norris LA, Ollendick TH, Olino TM, et al. Training with tarantulas: a randomized feasibility and acceptability study using experiential learning to enhance exposure therapy training. J Anxiety Disord. 2020. https://doi.org/10.1016/j.janxdis.2020.102308.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Higa-McMillan CK, Francis SE, Rith-Najarian L, Chorpita BF. Evidence base update: 50 years of research on treatment for child and adolescent anxiety. J Clin Child Adolesc Psychol. 2016. https://doi.org/10.1080/15374416.2015.1046177.

    Article  PubMed  Google Scholar 

  38. Peris TS, Compton SN, Kendall PC, Birmaher B, Sherrill J, March J, et al. Trajectories of change in youth anxiety during cognitive-behavior therapy. J Consult Clin Psychol. 2015. https://doi.org/10.1037/a0038403.

    Article  PubMed  Google Scholar 

  39. Kendall PC, Robin JA, Hedtke KA, Suveg C, Flannery-Schroeder E, Gosch E. Considering CBT with anxious youth? Think exposures Cogn Behav Pract. 2005. https://doi.org/10.1016/S1077-7229(05)80048-3.

    Article  Google Scholar 

  40. Breuninger C, Tuschen-Caffier B, Svaldi J. Dysfunctional cognition and self-efficacy as mediators of symptom change in exposure therapy for agoraphobia - systematic review and meta-analysis. Behav Res Ther. 2019. https://doi.org/10.1016/j.brat.2019.103443.

    Article  PubMed  Google Scholar 

  41. Penn innovation in suicide prevention for implementation research (INSPIRE). https://www.med.upenn.edu/INSPIRE/. Accessed 27 Oct 2022.

  42. Kaplan JS, Tolin DF. Exposure therapy for anxiety disorders: theoretical mechanisms of exposure and treatment strategies. Psychiatr Times. 2011;28(9):33–7.

    Google Scholar 

  43. Posner K, Brent D, Lucas C, Gould M, Stanley B, Brown G, et al. Columbia-Suicide Severity Rating Scale (C-SSRS). 2008. https://cssrs.columbia.edu/wp-content/uploads/C-SSRS_Pediatric-SLC_11.14.16.pdf. Accessed Dec 2022.

  44. Ferguson M, Rhodes K, Loughhead M, McIntyre H, Procter N. The effectiveness of the safety planning intervention for adults experiencing suicide-related distress: a systematic review. Arch Suicide Res. 2022. https://doi.org/10.1080/13811118.2021.1915217.

    Article  PubMed  Google Scholar 

  45. Stanley B, Brown GK, Brenner LA, Galfalvy HC, Currier GW, Knox KL, et al. Comparison of the safety planning intervention with follow-up vs usual care of suicidal patients treated in the emergency department. JAMA Psychiat. 2018. https://doi.org/10.1001/jamapsychiatry.2018.1776.

    Article  Google Scholar 

  46. The Consolidated Framework for Implementation Research – technical assistance for users of the CFIR framework 2009. https://cfirguide.org. Accessed 27 Oct 2022.

  47. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009. https://doi.org/10.1186/1748-5908-4-50.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Patey AM, Hurt CS, Grimshaw JM, Francis JJ. Changing behaviour ‘more or less’—do theories of behaviour inform strategies for implementation and de-implementation? A critical interpretive synthesis. Implement Sci. 2018. https://doi.org/10.1186/s13012-018-0826.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Becker-Haimes EM, Mandell DS, Fishman J, Williams NJ, Wolk CB, Wislocki K, et al. Assessing causal pathways and targets of implementation variability for EBP use (Project ACTIVE): a study protocol. Implement Sci Commun. 2021. https://doi.org/10.1186/s43058-021-00245-3.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Fishbein M, Ajzen I. Predicting and changing behavior. 1st ed. New York: Psychology Press; 2009. https://doi.org/10.4324/978020383820.

  51. Zlomke K, Davis TE. One-session treatment of specific phobias: a detailed description and review of treatment efficacy. Behav Ther. 2008. https://doi.org/10.1080/16506073.2013.773062.

    Article  PubMed  Google Scholar 

  52. Ollendick TH, Davis TE. One-session treatment for specific phobias: a review of Öst’s single-session exposure with children and adolescents. Cogn Behav Ther. 2013. https://doi.org/10.1080/16506073.2013.773062.

    Article  PubMed  Google Scholar 

  53. Davis TE, Ollendick TH, Öst LG. Intensive one-session treatment of specific phobias. 1st ed. New York: Springer Science + Business Media; 2012. https://doi.org/10.1007/978-1-4614-3253-1.

  54. Abramowitz J, Deacon B, Whiteside S. Exposure therapy for anxiety: principles and practice. 2nd ed. New York: Guilford Publications; 2019.

    Google Scholar 

  55. Langley J, Wolstenholme D, Cooke J. ‘Collective making’ as knowledge mobilisation: the contribution of participatory design in the co-creation of knowledge in healthcare. BMC Health Serv Res. 2018. https://doi.org/10.1186/s12913-018-3397-y.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Steen M, Manschot M, De Koning N. Benefits of co-design in service design projects. Int J Des. 2011;5(2).

  57. Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res. 2016. https://doi.org/10.1177/0962280215588241.

    Article  PubMed  Google Scholar 

  58. Stanley B, Brown GK. Safety planning guide: a quick guide for clinicians. Boulder: Suicide Prevention Resource Center and Western Interstate Commission for Higher Education. 2020. https://www.sprc.org/sites/default/files/SafetyPlanningGuide%20Quick%20Guide%20for%20Clinicians.pdf. Accessed Dec 2022.

  59. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017. https://doi.org/10.1186/s13013-017-0635-3.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Ajzen I, Joyce N, Sheikh S, Cote NG. Knowledge and the prediction of behavior: the role of information accuracy in the theory of planned behavior. Basic Appl Soc Psychol. 2011. https://doi.org/10.1080/01973533.2011.568834.

    Article  Google Scholar 

  61. Goulet F, Jacques A, Gagnon R, Racette P, Sieber W. Assessment of family physicians’ performance using patient charts: interrater reliability and concordance with chart-stimulated recall interview. Eval Health Prof. 2007. https://doi.org/10.1177/016327870707924.

    Article  PubMed  Google Scholar 

  62. Sinnott C, Kelly MA, Bradley CP. A scoping review of the potential for chart stimulated recall as a clinical research method. BMC Health Serv Res. 2017. https://doi.org/10.1186/s12913-017-2539-y.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Davis M, Wolk CB, Jager-Hyman S, Beidas RS, Young JF, Mautone JA, et al. Implementing nudges for suicide prevention in real-world environments: Project INSPIRE study protocol. Pilot Feasibility Stud. 2020. https://doi.org/10.1186/s40814-020-00686-y.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Ainsworth MA. Standardized patient encounters: a method for teaching and evaluation. JAMA. 1991. https://doi.org/10.1001/jama.266.10.1390.

    Article  PubMed  Google Scholar 

  65. Beidas RS, Cross W, Dorsey S. Show me, don’t tell me: behavioral rehearsal as a training and analogue fidelity tool. Cogn Behav Pract. 2014. https://doi.org/10.1016/j.cbpra.2013.04.002.

    Article  PubMed  Google Scholar 

  66. Brown G, Stanley B. The Safety Planning Intervention Rating Scale (SPIRS). Philadelphia: Unpublished Document; 2020.

    Google Scholar 

  67. Reyes-Portillo JA, McGlinchey EL, Toso-Salman J, Chin EM, Fisher PW, Mufson L. Clinician experience and attitudes toward safety planning with adolescents at risk for suicide. Arch Suicide Res. 2019. https://doi.org/10.1080/13811118.2018.1456382.

    Article  PubMed  Google Scholar 

  68. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007. https://doi.org/10.1111/j.1475-6773.2006.00684.x.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. 5th ed. New Brunswick: Aldine Transaction; 2010.

    Google Scholar 

  70. Creswell JW, Klassen AC, Plano Clark VL, Smith KC. Best practices for mixed methods research in the health sciences. Bethesda: National Institutes of Health; 2011. https://obssr.od.nih.gov/sites/obssr/files/Best_Practices_for_Mixed_Methods_Research.pdf.

  71. Beidas RS, Buttenheim AM, Mandell DS. Transforming mental health care delivery through implementation science and behavioral economics. JAMA Psychiat. 2021. https://doi.org/10.1001/jamapsychiatryu.2021.1120.

    Article  Google Scholar 

  72. Becker-Haimes EM, Marcus SC, Klein MR, Schoenwald SK, Fugo PB, McLeod BD, et al. A randomized trial to identify accurate measurement methods for adherence to cognitive-behavioral therapy. Behav Ther. 2022. https://doi.org/10.1016/j.beth.2022.06.001.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the other investigators within the Penn INSPIRE Center for their support of this work. We are especially grateful to the members of our Stakeholder Advisory Committee: Dr. Kathleen Kline, Ms. Natalie Dallard, Dr. Sean Snyder, Mr. Ihsan Hines, Mr. Eric Marsh, and Ms. Anna Kiesnowski. Finally, we would like to thank Ms. Michal Weiss and Ms. Sophia Young for their assistance in preparing this manuscript for submission.

Funding

Support for this research project is provided by the NIMH (P50 MH127511; MPIs: Brown, Oquendo).

Author information

Authors and Affiliations

Authors

Contributions

EBH, MO, SJH, and GB conceptualized the study. JJ, MB, EP, DSM, and MR provided meaningful contributions to the study design. EBH and MB wrote the initial draft of this manuscript. All others reviewed the manuscript, provided input, and approved the version submitted for publication. Funders were not involved in the writing of this report.

Corresponding author

Correspondence to Emily M. Becker-Haimes.

Ethics declarations

Ethics approval and consent to participate

All study procedures were approved by the City of Philadelphia Institutional Review Board (IRB) and the University of Pennsylvania (IRB) in accordance with ethical research guidelines for human subjects. Any modifications to the strategy and outreach materials will be submitted to the IRBs per institution procedures. Any deviations from the protocol or adverse events will be documented and reported. All participants will be provided a copy of documentation of their consent before engaging in any study activities. This report was prepared using the Standard Protocol Items: Recommendations for Interventional Trials 2013 Checklist.

Consent for publication

N/a.

Competing interests

Dr. Oquendo and Dr. Brown receive royalties from the Research Foundation for Mental Hygiene for the commercial use of the Columbia Suicide Severity Rating Scale. Dr. Oquendo serves as an advisor to Alkermes, St. George’s University, and Fundación Jimenez Diaz; her family owns stock in Bristol Myers Squibb. Dr. Brown serves as an advisor for Oui Therapeutics, LLC, and Psych Hub, LLC. The other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Becker-Haimes, E.M., Brady, M., Jamison, J. et al. An exposure-based implementation strategy to decrease clinician anxiety about implementing suicide prevention evidence-based practices: protocol for development and pilot testing (Project CALMER). Implement Sci Commun 4, 148 (2023). https://doi.org/10.1186/s43058-023-00530-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-023-00530-3

Keywords