Skip to main content
Download PDF
Download ePub

Formative evaluation prior to implementation of a brief treatment for posttraumatic stress disorder in primary care

Implementation Science Communications volume 4, Article number: 48 (2023) Cite this article

Abstract

Background

Successful implementation of evidence-based treatments (EBT) for posttraumatic stress disorder (PTSD) in primary care may address treatment access and quality gaps by providing care in novel and less stigmatized settings. Yet, PTSD treatments are largely unavailable in safety net primary care. We aimed to collect clinician stakeholder data on organizational, attitudinal, and contextual factors relevant to EBT implementation.

Methods

Our developmental formative evaluation was guided by the Consolidated Framework for Implementation Research (CFIR), including (a) surveys assessing implementation climate and attitudes towards EBTs and behavioral health integration and (b) semi-structured interviews to identify barriers and facilitators to implementation and need for augmentation. Participants were hospital employees (N = 22), including primary care physicians (n = 6), integrated behavioral health clinicians (n = 8), community wellness advocates (n = 3), and clinic leadership (n = 5). We report frequency and descriptives of survey data and findings from directed content analysis of interviews. We used a concurrent mixed-methods approach, integrating survey and interview data collected simultaneously using a joint display approach. A primary care community advisory board (CAB) helped to refine interview guides and interpret findings.

Results

Stakeholders described implementation determinants of the EBT related to the CFIR domains of intervention characteristics (relative advantage, adaptability), outer setting (patient needs and resources), inner setting (networks and communication, relative priority, leadership engagement, available resources), and individuals involved (knowledge and beliefs, cultural considerations). Stakeholders described strong attitudinal support (relative advantage), yet therapist time and capacity restraints are major PTSD treatment implementation barriers (available resources). Changes in hospital management were perceived as potentially allowing for greater access to behavioral health services, including EBTs. Patient engagement barriers such as stigma, mistrust, and care preferences were also noted (patient needs and resources). Recommendations included tailoring the intervention to meet existing workflows (adaptability), system alignment efforts focused on improving detection, referral, and care coordination processes (networks and communication), protecting clinician time for training and consultation (leadership engagement), and embedding a researcher in the practice (available resources).

Conclusions

Our evaluation identified key CFIR determinants of implementation of PTSD treatments in safety net integrated primary care settings. Our project also demonstrates that successful implementation necessitates strong stakeholder engagement.

Peer Review reports

Background

Safety net settings provide health care services to patients regardless of their ability to pay [1], and play a major role in serving populations with low access to care, including low-income and racial and ethnic minoritized groups. Posttraumatic stress disorder (PTSD) disproportionately affects patients seen in safety net hospitals, with prevalence estimates as high as 46% [2–5]. Despite high prevalence, PTSD is under-recognized and undertreated, with only 13% of individuals with PTSD receiving care [3]. Evidence-based treatments (EBTs) for mental health, including PTSD, are largely unavailable in safety net settings and, when available, are often exclusively provided in specialty care. However, low provider adoption of EBTs in specialty care settings is associated with structural challenges, such as high workload demands, little protected time for training, and attitudinal barriers [6, 7]. Additionally, patients’ perceptions of lower-quality care [8] and experiences of mental health stigma, discrimination [9–12], and medical mistrust [11] have resulted in underutilization of specialty care. For these reasons, patients in low-resource settings may prefer care in less stigmatized non-specialty settings (e.g., primary care) [13–17]. Gaps in access to EBTs have driven large racial and economic disparities [8, 18–20].

Integrated behavioral health (IBH) models utilize collaboration between behavioral health professionals and physicians to provide comprehensive medical and behavioral health care [21]. Successful implementation of EBTs in integrated primary care may address treatment access and quality gaps by expanding access and providing care in a less stigmatized setting [12, 14]. However, there may be unique challenges to implementation in primary care due to competing clinical and administrative priorities in interdisciplinary settings, and brief treatment and referral models of care [22]. EBTs for PTSD have only recently been adapted for primary care, though these data are limited to Veterans Health Administration (VHA) settings [23–26], including some with specialized, residential PTSD programs [27]. Given the complexities of implementation, it is essential to engage stakeholders in system and treatment redesign and in developing plans to promote adoption and sustainability of EBTs for PTSD. Possemato and colleagues (2018) detailed how stakeholder-engaged formative evaluation was essential in successful EBT implementation within VHA primary care [28], yet given differences between the VHA and safety net hospitals, there is a need to further optimize and test these EBTs in integrated primary care clinics within safety net hospitals [29, 30].

Implementation is a complex process that requires information from all organizational levels to develop and modify an intervention for system integration [31, 32]. Developmental formative evaluation can be used to optimize EBTs prior to their enactment to maximize the likelihood of success by identifying anticipated barriers and facilitators to implementation and tailoring interventions to meet the needs of individuals and organizations involved [31–34]. The parent study will test the effectiveness and implementation outcomes of a brief PTSD treatment in safety net integrated primary care, adapted for the local setting and population based on the stakeholder data reported here. We used a concurrent mixed methods developmental formative evaluation [33], which relies on a data-driven, learning-oriented approach to assess program and organization priorities related to EBT implementation. Our developmental formative evaluation assessed (a) drivers of current and ideal practices for PTSD treatment in primary care (e.g., patient screening, clinical objectives for quality and care); (b) facilitators and barriers to intervention adoption in this setting (guided by the Consolidated Framework for Implementation Research [CFIR] [35]); and recommendations on how to address these barriers/facilitators. Our finalized implementation blueprint that specifies expert-recommended implementation strategies [36] derived from this developmental formative evaluation is published elsewhere [37].

Methods

Setting

This study took place in the largest safety net hospital in New England, with hospital-based integrated primary care clinics serving approximately 50,000 patients. Most patients (70%) are insured by Medicaid (public health insurance for those requiring financial assistance), and over half (56%) of Medicaid-insured patients at the hospital are in need of behavioral health services. As such, the hospital has adopted an IBH model, which relies on collaboration, coordination, and colocation of primary care physicians (PCPs) and behavioral health specialists. PCPs are represented by residents and physicians. The IBH team includes clinical social workers and psychologists (7.0 FTE) and psychiatrists (1.5 FTE). IBH therapists have 50% of their time reserved for 30-min scheduled visits. The remaining time is unscheduled to allow for warm handoffs. The small number of IBH clinical staff relative to needs of the population underscores the need for low-intensity, time-limited interventions, and highly accessible stepped-up care when referring out. As such, IBH uses a stepped care approach to ensure that patients are offered the most effective, yet least intensive, care first, and treatment is “stepped-up” to higher level care when needed. The hospital also employs community wellness advocates, trusted members of the community, to support high-risk patients via advocacy, education, and care coordination.

Selected intervention

Our parent study will inform the development of a stepped care approach to PTSD treatment that spans primary care and speciality care. The central aim is to optimize and pilot test a “step one” intervention in safety net integrated primary care. Skills Training in Affective and Interpersonal Regulation (STAIR) [38–41], recently adapted for use in VHA primary care (STAIR-PC) [23], was selected in consultation with IBH leadership as an appropriate step one intervention in primary care during the study planning period due to its brevity and low intensity. STAIR-PC is a five-session cognitive behavioral therapy for PTSD that includes psychoeducation and coping skills training to address emotion regulation and interpersonal difficulties common in trauma survivors [23]. STAIR is an empirically supported treatment [38, 39], including in safety net settings [40] and when delivered by non-specialists [41], and STAIR-PC has demonstrated preliminary effectiveness [23], including when delivered by paraprofessionals (peers) [42].

At the time of this study, there were no published data to inform expected response rates for this protocol, yet studies published using a similar protocol have since found clinically meaningful change in 67% of participants (peer-delivery) [42], and in VHA samples, 13–46% retained their PTSD diagnosis [23, 43]. The parent trial will assess not only PTSD symptom reduction, but also whether engagement in a “step one” intervention may improve subsequent engagement in mental health services (e.g., by reducing mental health stigma and other barriers).

Participants

Participants were hospital employees in integrated primary care, selected to represent the distinct roles in patient care and potential different viewpoints on implementation determinants. Primary care participants included primary care providers and leadership, and IBH participants included potential interventionists in the future trial (IBH therapists, community wellness advocates) and behavioral health leadership. Participants were recruited at clinical team meetings and by email. We first recruited our potential interventionists, provided an orientation to the intervention, and then used purposive sampling to recruit additional key informants nominated by primary care and behavioral health leadership, for a total of 22 participants. Participation was voluntary and included 1 h of time, between online surveys and an in-person semi-structured interview with [S.E.V.]. Surveys varied by role, with some specific to potential interventionists (see Table 1). Data were collected from October 2018 to February 2019. Participants were remunerated $20. This study received an exempt determination from the Institutional Review Board.

Table 1 Descriptive information on participant roles and study measures completed
Full size table

We used a concurrent mixed-methods formative evaluation, where qualitative and quantitative data are collected in parallel, and analysis for integration of these data begins after data collection for both methods has been completed [47]. Following guidelines, we analyzed our two forms of data separately and then merged these data to present a convergent, integrated overview. Our evaluation was guided by the CFIR [35] to characterize the implementation determinants of STAIR-PC in a safety-net IBH setting. CFIR is one of the most widely used frameworks in implementation science and allowed us to identify indicators of anticipated implementation outcomes (i.e., predictors of future implementation success/failure), which would inform adaptations prior to trial start that would maximize actual implementation success [48]. We assessed four core CFIR domains to examine implementation determinants of a brief cognitive behavioral therapy for PTSD: (1) intervention characteristics, (2) inner setting, (3) outer setting, and (4) characteristics of individuals involved (Table 2). Two survey measures were used to assess the CFIR inner setting domain: (1) the implementation climate scale (ICS) [44] consists of 6 items (from 0 – “not at all” to 4 – “to a very great extent”), which assess level of agreement with statements reflecting three organizational climate dimensions of innovation use: expected, supported, and rewarded. We used the ICS to characterize the level of support for implementing EBTs for PTSD in primary care; and (2) the Levels of Integration Measure (LIM) [45] measures the integration of care between IBH clinicians and PCPs across 6 domains, including integrated clinical practice, systems integration, training, relationships, shared decision making, beliefs and commitment, and leadership (from 1 – “strongly disagree” to 5 – “strongly agree). We utilized the LIM to understand commitment to the IBH model, which we anticipated may be a predictor of later implementation. The ICS and LIM were completed by all participants.

Table 2 Interview guide based on selected CFIR constructs
Full size table

We used the Evidence-Based Practice Attitudes Scale (EBPAS-15) [46] to assess the CFIR domain of characteristics of the individual. This 15-item survey assesses the level of agreement with four dimensions of attitudes towards EBTs generally (not specific to STAIR-PC): (1) intuitive appeal of EBTs, (2) likelihood of adopting EBTs, given requirements, (3) openness to new practices, and (4) perceived divergence of usual practice. Scores range from 0 – “not at all” to 4 – “to a very great extent” to indicate how strongly an attitude is held. The EBPAS-15 was completed only by IBH participants (our potential interventionists and two stakeholders from behavioral health leadership).

A qualitative interview guide was developed in consultation with our primary care community advisory board (CAB). The CAB (N = 9) includes hospital employees in integrated primary care with decision-making authority and highly relevant expertise. The CAB is involved in all phases of the parent study, including interpretation of findings from this formative evaluation and subsequent implementation blueprint development. Full detail on our approach to CAB engagement is published elsewhere [37]; to reduce burden on participants, we worked with the CAB to select only interview questions from the CFIR Interview Guide Tool (https://cfirguide.org/guide/app/#/) deemed most relevant to the local setting. See Table 2 for the full interview guide. Interviews were completed by all participants. Only primary interventionists were asked to comment on appropriateness of the candidate intervention. Although none had delivered STAIR-PC, most therapists were familiar with cognitive behavioral therapy.

Data analysis

We ran frequencies and descriptives of survey data to characterize the overall implementation climate and attitudes in the local setting and conducted post hoc analyses to assess differences between participant type (IBH vs. primary care) and survey responses.

For qualitative data, we applied directed content analysis [49], which started with an a priori coding framework derived from CFIR but allowed for the addition of emergent codes during the codebook development and refinement process. We used a two-phase process for codebook development, consisting of a team-based approach [50] to analyzing interview data. Three team members coded data in phase 1 and 4 members in phase 2. Phase 1 of codebook development utilized a rapid coding procedure [51] based on audio review of interview data. The purpose of the rapid coding procedure was to quickly identify major ordinate themes that could be presented to CABs to operationalize adaptations to STAIR-PC prior to trial start [37].

Phase 2 of codebook development was a conventional approach [49], wherein coders were able to expand, refine, and finalize the codebook prior to coding of interviews that had been transcribed verbatim by study staff. The coding team met weekly to mitigate bias, discuss differing perspectives, and improve inter-coder reliability. The final coding phase consisted of repeating this process until no new codes emerged from the data, and consensus was reached. After finalizing the codebook, transcripts were double-coded using NVivo 12 software (QSR International) until a benchmark of at least 20% of the interviews with inter-coder agreement above 80% was met (i.e., 6 double-coded transcripts attained 97% inter-coder agreement). Then, the remaining transcripts were equally divided among the two primary coders for independent coding. The team met weekly during independent coding to ensure ongoing consensus.

Data integration

We use the Journal Article Reporting Standards (JARS) for Mixed Methods Research [52] to guide reporting. In our concurrent mixed methods approach [47], data integration occurred in two ways: (1) through a connecting method, as our survey data links with the qualitative data through our purposive sampling of primary care and IBH stakeholders [47], and (2) through a joint display approach to synthesize a correlation between our quantitative data (“low” or “high”) and responder sentiments described in the interview (“positive” or “negative”).

Our joint display is presented in Table 4. The qualitative data provided a prospective analysis of specific CFIR constructs pertaining to (a) drivers of current and ideal practices; (b) facilitators and barriers to adoption of an intervention. Quantitative data were embedded within qualitative findings for each CFIR domain and construct to describe the assessment of specific organizational and attitudinal factors within respondants’ narrative descriptions.

Results

Our developmental formative evaluation involved 22 participants, including 11 potential interventionists (8 IBH therapists, 3 community wellness advocates) and 11 PCPS and clinic leaders and administrators (6 PCPs, 5 clinic leaders and administrators representing psychology, psychiatry, social work, and operations). No participants refused to participate or dropped out. Sociodemographic information was not collected to preserve employee privacy.

Surveys

Table 3 provides descriptives of survey findings. Overall, participants endorsed moderate organizational support (CFIR inner setting) for implementation of evidence-based practices (ICS M = 2.09, SD = 0.81). Participants reported low ratings on the ICS subscales staff selection (M = 1.78, SD = 1.05) and rewards (M = 0.85, SD = 0.71), suggesting that although participants favorably rated their organization’s attitudinal support of evidence-based practices, the organization does not explicitly seek to hire staff who are trained in or value evidence-based practices (CFIR characteristics of individuals involved), nor does the organization provide financial incentives for adoption of these approaches (CFIR inner setting). Among IBH participants (n = 13), there was moderately strong attitudinal support for adoption of EBTs (EBPAS-15 M = 2.80, SD = 0.72), with ratings comparable to those observed among community mental health service providers [46, 53]. Overall, participants reported slightly positive attitudes towards behavioral health integration in the clinic (LIM M = 3.51, SD = 0.37). Importantly, mean and standard deviation for survey scores did not significantly vary between primary care and IBH clinician participants for the ICS (PCP 37.67 (9.19); IBH 36.85 (6.67); p = 0.82) and LIM (PCP 121.44 (9.34); IBH 122.85 (14.86); p = 0.79), emphasizing that priorities for program implementation for PTSD were similar across stakeholder type. Indeed, a joint display comparison of high and low survey scores shows shared sentiments across both primary care and IBH individuals from the ICS and LIM (Table 4).

Table 3 Descriptive data from surveys
Full size table
Table 4 Quotes from participants corresponding with high and low survey scores
Full size table

Interview findings

Although our interview guide focused on determinants of STAIR-PC, responses were often pertinent to the overall provision of brief mental health interventions in the integrated primary care setting, and not specific to PTSD treatment. See Table 5 for more detail on CFIR constructs (based on the original CFIR) that we identified as influencing implementation, and initial recommendations generated through interviewee feedback and CAB engagement.

Table 5 Factors that influence implementation of an EBT for PTSD in safety net primary care
Full size table

CFIR: intervention characteristics

Relative advantage

Respondents affirmed that a brief five-session cognitive behavioral therapy intervention was appropriate for use in primary care and was considered a more advantageous intervention than current practices for several reasons. An IBH provider shared that, relative to first-line EBTs for PTSD which are longer and more intensive, the structure of a brief manualized treatment would help orient patients to the short-term approach to PTSD therapy while also providing effective care: “if we’re [IBH] already planning for five sessions, then we [IBH] can market it [manualized PTSD intervention] that way for patients when they come in for the intake… plus, manualized treatments are super convenient.”

They expressed how low-intensity and brief treatment in IBH may be more suitable for patients seen in primary care, and especially for patients with barriers to engagement in high-intensity specialty care (e.g., stigma, mental health literacy, affective avoidance, or suppression). They hypothesized that patients may be more open to engage in stepped-up care (e.g., first-line EBTs for PTSD) if needed after STAIR-PC, as one PCP stated: “[for] people who will never ‘land’ in [specialty care], if we give this [manualized PTSD treatment] and they have a good experience, they may be more willing to do the next level.”

Adaptability

Respondents shared how the proposed intervention could fit the local practice with some modifications. As one PCP noted, “[physicians] want this [type of therapy]. [When we] have a very short [and] effective way to treat PTSD — where we teach people these skills they can practice at home to reduce their symptoms — [PCPs] can [offer this treatment option] and keep an eye on [treatment progress].” One IBH provider described how STAIR-PC was an appropriate fit based on patient needs and the IBH setting: “I think [a manualized PTSD treatment] is in line with the other interventions that we are trying to offer in IBH. So I think [the treatment fits the setting] completely. Both within an IBH model and for our patient population, where I think, you know, our focus tends to be on how do we help improve their functioning right now in their lives? So I think it fits within the scope of the work that we’re doing, and I don't see any reason why it wouldn’t be a good fit for this patient population.”

CFIR: outer setting

Patient needs and resources

Respondents noted that socioeconomic determinants of health were common barriers to engagement in and prioritization of PTSD treatment. As one IBH provider explained, “[engagement] could be challenging given the complexity of working with [patients experiencing] …homelessness or financial instability.” Additionally, respondents described transportation and caregiving barriers that contribute to patients’ late or missed appointments, resulting in less provider contact and high patient dropout. As one PCP emphasized, “the biggest thing for a lot of patients is competing priorities, living in a world where finding a job and being safe is the first priority and convincing people that it [mental health] is worth their time [comes second].”

Emergent themes

Several themes regarding cultural considerations emerged as a subconstruct within this CFIR domain, reflective of the need for increased considerations of race and racism, stigma, shame related to mental health care, and language and literacy barriers among patients with PTSD. Respondents described how patients may be hesitant to disclose trauma and seek treatment for PTSD due to mental health stigma and stigma of referral to specialty care. As one community wellness advocate shared, “the word [name of specialty care clinic] is very stigmatized.” Respondents felt that a stepped care approach to PTSD treatment, with triaged levels, may help patients engage at their preferred setting and intensity of treatment. Further, respondents hypothesized that stigma may be a reason to expect better engagement in primary care settings (v. specialty mental health) for the proposed 5-session intervention.

Providers additionally described the role of racism in the patient-provider relationship. As one community wellness advocate stated: “a lot of patients have mentioned to me, especially if [they’re] a person of color, [that] they [patients] want to be able to relate [to] and see someone of color.” This perspective highlights how systems in which providers do not reflect diversity of patients may face additional challenges earning trust and engaging patients.

CFIR: inner setting

Networks and communications

Respondents emphasized the importance of care coordination, including the need for clear internal and external referral processes between clinicians within the primary care setting.

Referrals to IBH

The process for referring to IBH utilized warm handoffs. Several mentioned that, while ideal, warm handoffs can be logistically difficult: “I think there’s a kind of subtle disincentive to use [a warm handoff] in that no matter how fast it [a warm handoff] is, you’re grabbing a room and you’re taking time, [an] extra 10 or 15 min to have that warm handoff take place.” Despite these challenges, the majority of providers supported the use of warm handoffs and recognized their role in improving patient engagement. As another PCP explained, “the opportunity to do long warm handoff[s] will definitely increase that [patient] buy-in.” Respondents reported that the practice of relying on patients to spontaneously disclose trauma history or PTSD symptoms during routine primary care visits may miss patients with mild-to-moderate PTSD symptom severity or those who are reluctant to disclose. The high severity referrals received by IBH, and access issues when attempting to refer out to specialty care led some respondents to believe that patients seen in IBH may be higher in symptom severity than typically seen in IBH models of care.

Referrals to specialty care

By contrast, the referral process to specialty care is less specified and rarely utilizes warm handoffs. IBH respondents voiced reservations about referring out to specialty care, noting concerns about patient dropout or uncertainty about current therapies offered in specialty care. One IBH provider described the siloed nature of IBH and specialty care: “I’ve only ever been to [the specialty care clinic] once or twice, and I don’t know that many people who work there, so…I have very little idea what actually goes on over there.”

Relative priority

At the time of this evaluation, the hospital was shifting to an accountable care organization (ACO), which links reimbursements to quality metrics and reductions in the cost of care. Some respondents felt that the central focus on ACO quality metrics may pose challenges to adding a screener for PTSD. Under the ACO, depression care screening is prioritized, and hospitals are evaluated, scored, and provided funding based on the frequency and consistency of Patient Health Questionnaire (PHQ-2 and -9) administration. Other respondents felt positively that the ACO shift may be a stepping stone to improving PTSD outcomes. One PCP noted how the redesigned system prompted by the ACO may allow greater access to behavioral health services, stating, “one of the real opportunities [of] the ACO is in restructuring the way that we think about [the] clinic more broadly and more aggressively than we thus far have.”

Leadership engagement

Importantly, respondents raised the need to have regular communication with between clinical leaders and team members about different resources. As one PCP noted, “there’s a lot going on, and…. [as part of the leadership team] … I have a good handle of all the programs …but, I think I'm probably the minority. The PCPs who are there once a week may not know of all the resources that are available to our patients.” Additionally, respondents noted that buy-in more broadly from operations managers and population health leaders was critical to successful implementation, including securing protected time and resources to support adoption.

Available resources

Workflow and productivity demand, documentation and case management tasks, and administrative meetings create extensive time pressures on all providers that limit availability for trainings needed for successful implementation. Respondents emphasized the need for additional protected time for training and suggested, in the meantime, that researchers capitalize on existing protected times for training.

Respondents also identified the availability of appointments for active, ongoing therapy in IBH as a major barrier to PTSD intervention implementation. Furthermore, rescheduling missed appointments in a timely fashion is challenging due to high patient volume and low efficiency. Respondents emphasized the need to address these workflow challenges to accommodate EBT scheduling demands (at least two visits per month).

Respondents also noted the importance of embedding the researcher (study principal investigator) in the practice, with one IBH therapist stating, “I would find [it] really helpful [to have the principal investigator embedded in the clinical support team] because …if there’s anything wrong or different later [that impacts the roll-out], …that could help keep the momentum going.”

CFIR: characteristics of individuals involved

Knowledge and beliefs about the intervention

Respondents described the need for educational trainings tailored to PCPs and IBH clinicians to improve understanding of trauma, PTSD diagnosis, and effective treatments. As one IBH provider expressed, “providers… just don’t know what to do with trauma.”

PTSD assessment

Respondents explained how some providers applied variable heuristics and clinical judgment to decide which patients to screen for trauma and PTSD. PCPs described gathering a thorough trauma history with full details of index events, which is not the standard (or advised) process for screening. Others described screening for PTSD symptoms only when PCPs knew or suspected trauma exposure based on voluntary disclosure or patient characteristics (race and immigration status). As one PCP shared: “the only people I know [who] have PTSD are the ones who volunteer it [trauma history or diagnosis].” Respondents also described how some PCPs assign a PTSD diagnosis based on trauma exposure alone, without directly assessing for symptoms. Still, other PCPs may avoid asking patients about trauma due to concerns around distressing patients or insufficient time to respond to anticipated distress.

Current treatment options

PCPs also described lack of knowledge on the types of treatments available via each referral pathway, partly due to lack of access to therapy notes in the electronic medical record. PCPs shared their general enthusiasm for co-learning with their IBH colleagues and how this project might help them to stay abreast of PTSD treatment options.

Data integration

See Table 4 for our joint display presentation of findings and exemplar quotes. In terms of treatment plans for PTSD to meet patient needs, knowledge of and prioritization of trauma in case conceptualization and treatment planning was reflected in high scores on the ICS, whereas lack of knowledge in trauma assessment and treatment, which was particularly a challenge among primary care providers compared to IBH clinicians, was evident in low ICS scores. Enthusiasm for evidence-based practice across primary care and IBH is consistent with high LIM scores, however, challenges (low LIM scores) were attributed to lack of clinic integration and a focus on abiding by state insurance programs.

Discussion

Our developmental formative evaluation to support the implementation of an EBT for PTSD in safety net primary care illuminated our understanding of current practices and attitudes and identified facilitators and barriers to implementation. Overall, respondents were optimistic that offering a brief, low-intensity treatment for PTSD in primary care may fit the practice and patients’ desires to receive care in a less stigmatized environment, though they also recognized non-specific challenges to patient engagement, including mental health stigma, medical mistrust, and competing clinical priorities (hierarchy of needs; e.g., housing). Although there was strong attitudinal support for EBTs, cross-cutting time and resource constraints posed major barriers to implementation. Respondents pointed to various system-alignment challenges that would need to be addressed prior to roll out — workflow and scheduling challenges and the need to address training gaps across provider types to support diagnosis, treatment, and referral processes for PTSD. Notably, our survey results also highlighted that the local setting has not prioritized hiring staff trained in evidence-based practices and that current providers have low self-efficacy for treating PTSD. This is one place where safety net settings may contrast with the VHA, which has made access to training in EBTs for PTSD readily available [54]. Improving access to training is one important avenue to address when preparing safety net systems for delivering PTSD treatment.

Our findings extend the literature in identifying PTSD treatment implementation barriers specific to safety net integrated primary care settings. We found organizational support for EBTs was similar or higher than ratings of implementation climate observed in other mental health [44] and substance use settings [55]. Consistent with research on determinants to implementation of EBTs for PTSD across settings [2, 6, 22, 56], important factors included organizational/leadership support for EBTs, clinic operations, intervention feasibility in the practice, provider attitudes towards EBTs, therapist time and capacity, and patient engagement (which may be particularly challenging in low-resource settings due to competing social needs [6]). However, extant research has primarily focused on higher-intensity, burdensome interventions (e.g., cognitive processing therapy, prolonged exposure) in specialty care settings. Unique challenges in primary care include lack of PCP knowledge of PTSD treatment options and lack of brief EBTs for PTSD that are conducive with primary care models [22]. In a review of early research on primary care-based PTSD interventions, Possemato (2011) found promising preliminary effectiveness of brief interventions, yet few have been successfully adopted in usual primary care, and implementation process outcomes are scarce [22]. Our CFIR-guided formative evaluation helps to position these brief treatments within a continuum of care and highlights the critical need to align systems prior to implementation, such as setting up clear pathways for patients to move through the safety net system and receive efficacious treatments at multiple points of service.

One observation we made in conducting this formative evaluation is how important inner setting constructs are to provider buy-in and adoption [56]. For example, PCPs and leadership were more receptive to screening when treatments were readily available in IBH (as part of the same project), and IBH therapists were more receptive to delivering EBTs when there were also efforts to address workflow challenges and simultaneously build capacity in specialty care. The latter is beyond the scope of this project, but has become a focus of departmental initiatives aimed at therapy capacity building across the continuum of care.

Limitations

Given our use of purposive sampling in this study, results are limited in their generalizability or reproducibility. Furthermore, findings may have been biased by our efforts to focus mainly on barriers (rather than facilitators) to implementation, in order to develop strategies to overcome these. Respondents were also instructed that they could talk about their own practice or the modal practice in their local setting — this again may have biased findings. Our sample included many champions or people with particular knowledge about PTSD. Their views may not reflect the attitudes of all providers in the practice, demonstrating the need for practice-wide assessments of implementation factors. That said, few respondents had prior experience with manualized treatment for PTSD, and therefore responses about CFIR domains were often generalities about behavioral health treatment or cognitive behavioral therapies–-not specific to STAIR-PC. Post-trial data will help us better understand the acceptability of STAIR-PC specifically, as therapists and other stakeholders become more familiar with the intervention as they apply it to their patients in usual care.

We focused the formative evaluation on hospital employees’ perspectives because we anticipated that system alignment challenges would be paramount, yet we acknowledge that patient perspectives are missing. In addition to engaging our primary care CAB, we will also be engaging a patient CAB in the next phase of the parent study, where we develop our implementation blueprint to guide the clinical trial; and we will be conducting exit interviews with trial participants to gain their perspectives on the intervention and implementation. Recognizing the siloed nature of IBH and specialty care, IBH respondents proposed improvements in assessment, screening, referral processes, and treatment available in specialty care. Our data fell short of explicating cross-clinic barriers to implementation that could be identified through the use of process mapping [57].

Future steps

Respondents noted how patients of color may have greater mistrust of providers due to past negative interactions with health care, including poor quality of care, discrimination, and historical maltreatment of minoritized people. These findings suggest that additional engagement supports, such as effort to improve mental health literacy, reduce stigma, and gain trust are warranted. We propose that engagement of peers or community wellness advocates in this role may be appropriate [58]. Additionally, respondents acknowledged the need to consider variability in language and literacy, as many patients do not speak English as their primary language or are illiterate. Although our formative work suggested that no modifications to core components of the intervention were warranted [37], we identified the need to expand the scope of clinician consultation in the trial to support culturally responsive application of PTSD treatment. For example, our group consultation model will encourage discussions around integrating social contextual factors in case conceptualization and exploring the impact of racism on stress. To ensure that engagement barriers related to racism and stigma are addressed thoroughly, we will conduct exit interviews with providers and patients to assess the need for cultural adaptations to intervention components and will use ongoing CAB engagement to ensure fit with the patient population. We will also seek to expand the intervention with adaptations for patients with advanced age or disabilities. Additional detail on our formative work on racism as an implementation consideration is published elsewhere [59].

Conclusion

Successful implementation of evidence-based PTSD treatments in safety net hospitals necessitates a strong implementation blueprint [36] or toolkit [60] that includes multi-level interventions and protocols (e.g., detection and referral procedures, integration and collaboration, training programs for various provider types and settings, dual capacity building in non-specialty and specialty care clinics, hiring practices, provider incentives, allocated time for training and consultation) [61]. Indeed, clinic operations that are committed to learning from point of care patient interactions and incorporating a continuous improvement and data-driven process within the delivery and sustainability of EBTs for PTSD have been identified as an important determinant in reach of EBTs within either a primary care or mental health care setting [62]. The heavy lift of successful implementation further illustrates the need for institutions to invest in sustainably embedding implementation scientists within clinical practice [63–65].

Availability of data and materials

Not applicable.

Abbreviations

PTSD:

Posttraumatic stress disorder

CFIR:

Consolidated framework for implementation research

EBT:

Evidence-based treatment

PCP:

Primary care physician

IBH:

Integrated behavioral health

STAIR-PC:

Skills training in affective and interpersonal regulation for primary care

ICS:

Implementation climate scale

LIM:

Levels of integration measure

EBPAS:

Evidence-based practice attitudes scale

CAB:

Community advisory board

ACO:

Accountable care organization

PHQ:

Patient health questionnaire

VHA:

Veterans health administration

References

  1. Institute of Medicine (US) Committee on the Changing Market MC, Lewin ME, Altman S. Americas’s Health Care Safety Net. National Academies Press (US); 2000. Available from: https://www.ncbi.nlm.nih.gov/books/NBK224523/. cited 2023 Jan 23.

  2. Foa EB, Gillihan SJ, Bryant RA. Challenges and Successes in Dissemination of Evidence-Based Treatments for Posttraumatic Stress: Lessons Learned From Prolonged Exposure Therapy for PTSD. Psychol Sci Public Interest J Am Psychol Soc. 2013;14(2):65–111.

    Google Scholar 

  3. Alim TN, Graves E, Mellman TA, Aigbogun N, Gray E, Lawson W, et al. Trauma exposure, posttraumatic stress disorder and depression in an African-American primary care population. J Natl Med Assoc. 2006;98(10):1630–6.

    PubMed  PubMed Central  Google Scholar 

  4. Roberts AL, Gilman SE, Breslau J, Breslau N, Koenen KC. Race/ethnic differences in exposure to traumatic events, development of post-traumatic stress disorder, and treatment-seeking for post-traumatic stress disorder in the United States. Psychol Med. 2011;41(1):71–83.

    Article  CAS  PubMed  Google Scholar 

  5. Gillespie CF, Bradley B, Mercer K, Smith AK, Conneely K, Gapen M, et al. Trauma exposure and stress-related disorders in inner city primary care patients. Gen Hosp Psychiatry. 2009;31(6):505–14.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Marques L, Dixon L, Valentine SE, Borba CPC, Simon NM, Wiltsey SS. Providers’ perspectives of factors influencing implementation of evidence-based treatments in a community mental health setting: A qualitative investigation of the training-practice gap. Psychol Serv. 2016;13(3):322–31.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Frueh BC, Cusack KJ, Grubaugh AL, Sauvageot JA, Wells C. Clinicians’ perspectives on cognitive-behavioral treatment for PTSD among persons with severe mental illness. Psychiatr Serv Wash DC. 2006;57(7):1027–31.

    Article  Google Scholar 

  8. Institute of Medicine (US) Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Smedley BD, Stith AY, Nelson AR, editors. Washington (DC): National Academies Press (US); 2003. Available from: http://www.ncbi.nlm.nih.gov/books/NBK220358/. cited 2023 Jan 23.

  9. Próspero M, Vohra-Gupta S. The Use of Mental Health Services Among Victims of Partner Violence on College Campuses. J Aggress Maltreatment Trauma. 2008;16(4):376–90.

    Article  Google Scholar 

  10. Valentine SE, Dixon L, Borba CPC, Shtasel DL, Marques L. Mental illness stigma and engagement in an implementation trial for Cognitive Processing Therapy at a diverse community health center: a qualitative investigation. Int J Cult Ment Health. 2016;9(2):139–50.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Kaltman S, Hurtado de Mendoza A, Gonzales FA, Serrano A. Preferences for trauma-related mental health services among Latina immigrants from Central America, South America, and Mexico. Psychol Trauma Theory Res Pract Policy. 2014;6:83–91.

    Article  Google Scholar 

  12. Fox J, Desai MM, Britten K, Lucas G, Luneau R, Rosenthal MS. Mental-Health Conditions, Barriers to Care, and Productivity Loss Among Officers in An Urban Police Department. Conn Med. 2012;76(9):525–31.

    PubMed  PubMed Central  Google Scholar 

  13. Le Cook B, Manning W, Alegria M. Measuring disparities across the distribution of mental health care expenditures. J Ment Health Policy Econ. 2013;16(1):3–12.

    PubMed  PubMed Central  Google Scholar 

  14. Garland AF, Lau AS, Yeh M, McCabe KM, Hough RL, Landsverk JA. Racial and ethnic differences in utilization of mental health services among high-risk youths. Am J Psychiatry. 2005;162(7):1336–43.

    Article  PubMed  Google Scholar 

  15. Lê Cook B, McGuire TG, Lock K, Zaslavsky AM. Comparing Methods of Racial and Ethnic Disparities Measurement across Different Settings of Mental Health Care. Health Serv Res. 2010;45(3):825–47.

    Article  PubMed  PubMed Central  Google Scholar 

  16. McGuire TG, Alegria M, Cook BL, Wells KB, Zaslavsky AM. Implementing the Institute of Medicine Definition of Disparities: An Application to Mental Health Care. Health Serv Res. 2006;41(5):1979–2005.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Cook BL, McGuire T, Miranda J. Measuring trends in mental health care disparities, 2000 2004. Psychiatr Serv Wash DC. 2007;58(12):1533–40.

    Article  Google Scholar 

  18. Dinwiddie GY, Gaskin DJ, Chan KS, Norrington J, McCleary R. Residential segregation, geographic proximity and type of services used: Evidence for racial/ethnic disparities in mental health. Soc Sci Med. 2013;1982(80):67–75.

    Article  Google Scholar 

  19. Cook BL, McGuire TG, Zaslavsky AM. Measuring racial/ethnic disparities in health care: methods and practical issues. Health Serv Res. 2012;47(3 Pt 2):1232–54.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Alegría M, Mulvaney-Day N, Woo M, Torres M, Gao S, Oddo V. Correlates of Past-Year Mental Health Service Use Among Latinos: Results From the National Latino and Asian American Study. Am J Public Health. 2007;97(1):76–83.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Prom MC, Canelos V, Fernandez PJ, Gergen Barnett K, Gordon CM, Pace CA, et al. Implementation of Integrated Behavioral Health Care in a Large Medical Center: Benefits, Challenges, and Recommendations. J Behav Health Serv Res. 2021;48(3):346–62.

    Article  PubMed  Google Scholar 

  22. Possemato K. The current state of intervention research for posttraumatic stress disorder within the primary care setting. J Clin Psychol Med Settings. 2011;18(3):268–80.

    Article  PubMed  Google Scholar 

  23. Jain S, Ortigo K, Gimeno J, Baldor DA, Weiss BJ, Cloitre M. A Randomized Controlled Trial of Brief Skills Training in Affective and Interpersonal Regulation (STAIR) for Veterans in Primary Care. J Trauma Stress. 2020;33(4):401–9.

    Article  PubMed  Google Scholar 

  24. Rauch SAM, Cigrang J, Austern D, Evans A. Expanding the Reach of Effective PTSD Treatment Into Primary Care: Prolonged Exposure for Primary Care. Focus J Life Long Learn Psychiatry. 2017;15(4):406–10.

    Article  Google Scholar 

  25. Schnurr PP, Simiola V, Ruzek J, Thompson R, Hoff R, Cook JM. Testing a comprehensive model of implementation and sustained use for EBTs for PTSD: A national investigation in VA residential settings. Implement Sci IS. 2015;10(Suppl 1):A30.

    Article  Google Scholar 

  26. Anhang Price R, Sloss EM, Cefalu M, Farmer CM, Hussey PS. Comparing Quality of Care in Veterans Affairs and Non-Veterans Affairs Setting. J Gen Intern Med. 2018;33(10):1631–8.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Cook JM, O’Donnell C, Dinnen S, Bernardy N, Rosenheck R, Desai R. A Formative Evaluation of Two Evidence-Based Psychotherapies for PTSD In VA Residential Treatment Programs. J Trauma Stress. 2013;26(1):56–63.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Possemato K, Johnson EM, Wray LO, Webster B, Stecker T. The implementation and testing of a referral management system to address barriers to treatment seeking among primary care veterans with PTSD. Psychol Serv. 2018;15(4):457–69.

    Article  PubMed  Google Scholar 

  29. Borowsky SJ, Rubenstein LV, Meredith LS, Camp P, Jackson-Triche M, Wells KB. Who Is at Risk of Nondetection of Mental Health Problems in Primary Care? J Gen Intern Med. 2000;15(6):381–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Petterson S, Miller BF, Payne-Murphy JC, Phillips RL. Mental health treatment in the primary care setting: patterns and pathways. Fam Syst Health J Collab Fam Healthc. 2014;32(2):157–66.

    Article  Google Scholar 

  31. Elwy AR, Wasan AD, Gillman AG, Johnston KL, Dodds N, McFarland C, et al. Using formative evaluation methods to improve clinical implementation efforts: Description and an example. Psychiatry Res. 2020;283:112532.

    Article  PubMed  Google Scholar 

  32. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation Hybrid Designs. Med Care. 2012;50(3):217–26.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, et al. The role of formative evaluation in implementation research and the QUERI experience. J Gen Intern Med. 2006;21 Suppl 2(Suppl 2):S1-8.

    Article  PubMed  Google Scholar 

  34. Palinkas LA, Garcia AR, Aarons GA, Finno-Velasquez M, Holloway IW, Mackie TI, et al. Measuring Use of Research Evidence: The Structured Interview for Evidence Use. Res Soc Work Pract. 2016;26(5):550–64.

    Article  PubMed  Google Scholar 

  35. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Valentine SE, Fuchs C, Carlson M, Elwy AR. Leveraging multistakeholder engagement to develop an implementation blueprint for a brief trauma-focused cognitive behavioral therapy in primary care. Psychol Trauma. 2022;14(6):914–23. https://doi.org/10.1037/tra0001145.

    Article  PubMed  Google Scholar 

  38. Cloitre M, Koenen KC, Cohen LR, Han H. Skills training in affective and interpersonal regulation followed by exposure: A phase-based treatment for PTSD related to childhood abuse. J Consult Clin Psychol. 2002;70:1067–74.

    Article  PubMed  Google Scholar 

  39. Cloitre M, Stovall-McClough KC, Nooner K, Zorbas P, Cherry S, Jackson CL, et al. Treatment for PTSD Related to Childhood Abuse: A Randomized Controlled Trial. Am J Psychiatry. 2010;167(8):915–24.

    Article  PubMed  Google Scholar 

  40. MacIntosh HB, Cloitre M, Kortis K, Peck A, Weiss BJ. Implementation and Evaluation of the Skills Training in Affective and Interpersonal Regulation (STAIR) in a Community Setting in the Context of Childhood Sexual Abuse. Res Soc Work Pract. 2018;28(5):595–602.

    Article  Google Scholar 

  41. Trappler B, Newville H. Trauma Healing Via Cognitive Behavior Therapy in Chronically Hospitalized Patients. Psychiatr Q. 2008;78:317–25.

    Article  Google Scholar 

  42. Smith AM, Stewart K, Baul T, Valentine SE. Peer delivery of a brief cognitive-behavioral treatment for posttraumatic stress disorder: A hybrid effectiveness-implementation pilot study. J Clin Psychol. 2020;76(12):2133–54.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Weiss BJ, Azevedo K, Webb K, Gimeno J, Cloitre M. Telemental Health Delivery of Skills Training in Affective and Interpersonal Regulation (STAIR) for Rural Women Veterans Who Have Experienced Military Sexual Trauma. J Trauma Stress. 2018;31(4):620–5. https://doi.org/10.1002/jts.22305.

    Article  PubMed  Google Scholar 

  44. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implement Sci. 2014;9(1):157.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Fauth J, Tremblay G. The Integrated Care Evaluation Project: Full Report. Keene: Antioch University New England; 2011.

    Google Scholar 

  46. Aarons GA. Mental Health Provider Attitudes Toward Adoption of Evidence-Based Practice: The Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004;6(2):61–74.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Fetters MD, Curry LA, Creswell JW. Achieving Integration in Mixed Methods Designs—Principles and Practices. Health Serv Res. 2013;48(6 Pt 2):2134–56.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Damschroder LJ, Reardon CM, OpraWiderquist MA, Lowery J. Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022;17(1):7.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Hsieh HF, Shannon S. Three Approaches to Qualitative Content Analysis. Qual Health Res. 2005;15:1277–88.

    Article  PubMed  Google Scholar 

  50. Patton MQ. Qualitative Research & Evaluation Methods. SAGE; 2002. p. 692.

    Google Scholar 

  51. Neal JW, Neal ZP, VanDyke E, Kornbluh M. Expediting the Analysis of Qualitative Data in Evaluation: A Procedure for the Rapid Identification of Themes From Audio Recordings (RITA). Am J Eval. 2015;36(1):118–32.

    Article  Google Scholar 

  52. Levitt HM, Bamberg M, Creswell JW, Frost DM, Josselson R, Suárez-Orozco C. Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. Am Psychol. 2018;73(1):26.

    Article  PubMed  Google Scholar 

  53. Aarons GA, Glisson C, Hoagwood K, Kelleher K, Landsverk J, Cafri G. Psychometric properties and United States national norms of the Evidence-Based Practice Attitude Scale (EBPAS). Psychol Assess. 2010;22(2):356–65. https://doi.org/10.1037/a0019188.

    Article  PubMed  Google Scholar 

  54. Karlin BE, Ruzek JI, Chard KM, Eftekhari A, Monson CM, Hembree EA, et al. Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the Veterans Health Administration. J Trauma Stress. 2010;23(6):663–73.

    Article  PubMed  Google Scholar 

  55. Ehrhart MG, Torres EM, Hwang J, Sklar M, Aarons GA. Validation of the Implementation Climate Scale (ICS) in substance use disorder treatment organizations. Subst Abuse Treat Prev Policy. 2019;14:35.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Yamokoski C, Barron S, Fowler J, Fast E, Flores H. Barriers and Facilitators to the Implementation of Intensive Treatments for PTSD: Early Lessons Learned From the Field. Cogn Behav Pract. 2022. Available from: https://www.sciencedirect.com/science/article/pii/S1077722922000815. cited 2023 Jan 25.

  57. Kim B, Rodrigues S, Dell NS, Elwy R. Process mapping appointments to identify improvements in care delivered to patients who screen positive for depression. Eur J Pers Centered Healthc. 2016;4(3):433–8.

    Article  Google Scholar 

  58. Barnett ML, Gonzalez A, Miranda J, Chavira DA, Lau AS. Mobilizing Community Health Workers to Address Mental Health Disparities for Underserved Populations: A Systematic Review. Adm Policy Ment Health. 2018;45(2):195–211.

    Article  PubMed  PubMed Central  Google Scholar 

  59. McCuistian C, Kimball SL, Buitron de la Vega P, Godfrey LB, Fortuna LR, Valentine SE. Consideration of racism experiences in the implementation of trauma-focused therapy in primary care. Health Serv Res. 2022;57 Suppl 2(Suppl 2):235–48. https://doi.org/10.1111/1475-6773.13998.

    Article  PubMed  Google Scholar 

  60. Hempel S, O’Hanlon C, Lim YW, Danz M, Larkin J, Rubenstein L. Spread tools: a systematic review of components, uptake, and effectiveness of quality improvement toolkits. Implement Sci. 2019;14(1):83.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Cook JM, O’Donnell C, Dinnen S, Coyne JC, Ruzek JI, Schnurr PP. Measurement of a model of implementation for health care: toward a testable theory. Implement Sci. 2012;7(1):59.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Sayer NA, Rosen CS, Bernardy NC, Cook JM, Orazem RJ, Chard KM, et al. Context Matters: Team and Organizational Factors Associated with Reach of Evidence-Based Psychotherapies for PTSD in the Veterans Health Administration. Adm Policy Ment Health. 2017;44(6):904–18.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Nash DM, Bhimani Z, Rayner J, Zwarenstein M. Learning health systems in primary care: a systematic scoping review. BMC Fam Pract. 2021;22(1):126.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Shiner B, Watts BV. Invited Commentary: Baby Steps to a Learning Mental Health-Care System-Can We Do the Work? Am J Epidemiol. 2021;190(7):1220–2.

    Article  PubMed  Google Scholar 

  65. Elwy AR, Maguire EM, McCullough M, George J, Bokhour BG, Durfee JM, et al. From implementation to sustainment: A large-scale adverse event disclosure support program generated through embedded research in the Veterans Health Administration. Healthc Amst Neth. 2021;8 Suppl 1:100496.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study was funded by the National Institute of Mental Health (K23MH117221; PI: Valentine). Other than study oversight (annual reporting), the funding body is not involved in the conduct of research, interpretation of data, or writing of this manuscript.

Author information

Authors and Affiliations

  1. Department of Psychiatry, Boston Medical Center, Boston, MA, USA

    Sarah E. Valentine, Cara Fuchs & Laura B. Godfrey

  2. Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA

    Sarah E. Valentine & Cara Fuchs

  3. Boston University School of Public Health, Boston, MA, USA

    Elyse A. Olesinski & Natalya Sarkisova

  4. Department of Psychiatry and Human Behavior, Alpert Medical School of Brown University, Providence, RI, USA

    A. Rani Elwy

  5. Center for Healthcare Organization and Implementation Research, VA Bedford Healthcare System, Bedford, MA, USA

    A. Rani Elwy

Authors
  1. Sarah E. Valentine
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cara Fuchs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elyse A. Olesinski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Natalya Sarkisova
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Laura B. Godfrey
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. A. Rani Elwy
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SEV designed and coordinated the study, collected the data, and wrote the manuscript. CF and ARE oversaw the study and wrote the manuscript. NS and EAO analyzed the data from semi-structured interviews and wrote the manuscript. LBG wrote the manuscript. All authors read, critically reviewed, and approved the final manuscript.

Corresponding authors

Correspondence to Sarah E. Valentine or A. Rani Elwy.

Ethics declarations

Ethics approval and consent to participate

This study H-37613 received an exempt determination by the Institutional Review Board (IRB) of Boston Medical Center and Boston University Medical Campus. All respondents were informed of the study’s objectives upfront and consented to participate.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valentine, S.E., Fuchs, C., Olesinski, E.A. et al. Formative evaluation prior to implementation of a brief treatment for posttraumatic stress disorder in primary care. Implement Sci Commun 4, 48 (2023). https://doi.org/10.1186/s43058-023-00426-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-023-00426-2

Keywords

Implementation Science Communications

ISSN: 2662-2211