- Open Access
Collaborative chronic care model implementation within outpatient behavioral health care teams: qualitative results from a multisite trial using implementation facilitation
Implementation Science Communications volume 2, Article number: 33 (2021)
This paper reports on a qualitative evaluation of a hybrid type II stepped-wedge, cluster randomized trial using implementation facilitation to implement team-based care in the form of the collaborative chronic care model (CCM) in interdisciplinary outpatient mental health teams. The objective of this analysis is to compare the alignment of sites’ clinical processes with the CCM elements at baseline (time 1) and after 12 months of implementation facilitation (time 2) from the perspective of providers.
We conducted semi-structured interviews to assess the extent to which six CCM elements were in place: work role redesign, patient self-management support, provider decision support, clinical information systems, linkages to community resources, and organizational/leadership support. Interviews were transcribed and a priori CCM elements were coded using a directed content analysis approach at times 1 and 2. We sought consensus on, and compared, the extent to which each CCM element was in place at times 1 and 2.
We conducted 27 and 31 telephone interviews at times 1 and 2, respectively, with outpatient mental health providers at nine participating sites. At time 1 and time 2, three CCM elements were most frequently present across the sites: work role redesign, patient self-management support, and provider decision support. The CCM elements with increased implementation from time 1 to time 2 were work role redesign, patient self-management support, and clinical information systems. For two CCM elements, linkages to community resources and organizational/leadership support, some sites had increased implementation at time 2 compared to time 1, while others had reductions. For the provider decision support element, we saw little change in the extent of its implementation.
Sites increased the extent of implementation on several CCM elements. The most progress was made in the CCM elements where sites had CCM-aligned processes in place at time 1. Teams made progress on elements they could more easily control, such as work role redesign. Our results suggest that maximizing the benefits of CCM-based outpatient mental health care may require targeting resources and training toward specific CCM elements—especially in the use of clinical information systems and linking with community resources.
Clinical Trials NCT02543840.
In 2012, mental health leaders within the Department of Veterans Affairs (VA) launched the Behavioral Health Interdisciplinary Program (BHIP) to build effective interdisciplinary general outpatient mental health teams. These teams serve a panel of about a thousand Veteran patients and include between 5 and 10 team members (e.g., licensed mental health practitioners and non-licensed clinical/administrative support staff.) . The guiding model for this initiative is the collaborative chronic care model (CCM), an evidence-based approach to structuring clinical care [2, 3] with the goal of ensuring coordinated, patient-centered, and anticipatory care. There are 6 elements in the CCM including work role redesign, patient self-management support, provider decision support, clinical information systems, linkages to community resources, and organization and leadership support . The CCM originated in the 1990s as the Chronic Care Model for chronic medical illnesses treated in primary care [2, 3]. More recent work, however, has involved applying the principles of CCM-based care to mental health conditions in both primary care and specialty mental health settings. Recent reviews of over 50 controlled trials of CCM-based care have demonstrated that it can improve mental health outcomes and satisfaction with no added net costs [5, 6].
Despite this evidence base, implementing CCM-based care in real-world clinical settings has proven difficult. For example, Smith and colleagues  have identified barriers to CCM implementation including competing system-level priorities and the need for modified clinical workflows and tracking. Unützer et al.  found patient factors (e.g., depression severity), clinical factors (e.g., years as a CCM practices), and the amount of implementation support were associated with patient outcomes. These challenges may be especially relevant to the CCM elements of work role redesign, clinical information systems, and organization and leadership support. To our knowledge, however, no previous research has specifically set out to track the extent to which each of the CCM elements was pursued over the course of an implementation trial.
Between 2016 and 2018, as part of a partnered evaluation with the VA’s Office of Mental Health and Suicide Prevention (OMHSP), our team conducted a hybrid type II stepped wedge cluster randomized trial to implement the evidence-based CCM in BHIP teams. Given the positive evidence regarding the success of the CCM to improve health outcomes and satisfaction, VA adopted the CCM as the model for the project using a workbook-guided approach . We utilized an evidence-based implementation strategy known as blended or external-internal facilitation, in which an external facilitator brings both content and process improvement expertise to a site and works closely with an internal facilitator, who offers the knowledge of the site’s organizational culture and existing procedures . We found that Veterans who received care from teams implementing the CCM had significantly lower mental health hospital admission rates compared with their peers at the same sites who were treated by teams who had not implemented CCM. Additionally, mental health status improved among Veterans who were treated by teams and had three or more mental health conditions. Clinicians within the teams experienced significant improvement in two aspects of team functioning (role clarity and focus on team priorities) . Given the support from operational partners and results from this trial, the VA plans to implement CCM-aligned general mental health teams more widely throughout the healthcare system .
In addition to understanding the effectiveness of CCM implementation, we also assessed whether CCM-based care can be implemented under general clinical practice conditions in general mental health clinics (the implementation portion of the hybrid type II design). As part of this trial, we conducted provider interviews to understand how facilitated implementation of CCM elements was associated with improvements in collaborative care. Miller et al.  explored how aligned care practices were to the CCM elements at the trial baseline and found that there were both consistent and inconsistent processes in place within all 6 CCM elements. Staff supported the goals of CCM-based care although the CCM elements were not addressed holistically or in an integrated way . The objective of this follow-up study is to compare the alignment of sites’ clinical processes with the CCM elements at baseline (time 1) and after 12 months of implementation facilitation (time 2), from the perspective of providers. This study provides critical information to guide future implementation of CCM-based care utilizing implementation facilitation. This study is particularly timely as healthcare organizations, including the VA, continue to adopt collaborative care and team-based care more generally.
This study was approved by the VA Boston Healthcare System Institutional Review Board. Nine teams from nine different Veterans Affairs Medical Centers (VAMCs) participated in this CCM implementation project; all participating teams received implementation facilitation support during the trial but the start time of that facilitation was staggered per the stepped-wedge design . There were 3 facilitators and each facilitator was assigned to three sites. For each site, facilitation involved an in-depth pre-site visit assessment and orientation of the site to the facilitation process and the CCM; 1½-day face-to-face site visit by the external facilitator to launch the implementation; 6 months of weekly videoconferences and/or conference calls with the team, weekly meetings between the external and internal facilitators, and ad hoc telephone and email communications; and 6 months of step-down facilitation contacts as needed (sites varied in the degree to which they used step-down facilitation). Sites were assigned to study waves using a balancing algorithm . Thus, sites that were waiting for implementation facilitation to begin served as controls; during their control condition phase, they received technical assistance, including the distribution of a BHIP-CCM Enhancement Guide (i.e., workbook based on CCM elements) (available upon request) as well as monthly conference calls. These calls were attended by the internal facilitator from each site and anyone else that the internal facilitator decided to extend the invitation to (most commonly active members of the site’s team). Attendance by those other than the internal facilitators was not regular. The initial few monthly calls were led by the external facilitators to introduce the CCM and implementation overview, followed by subsequent calls being discussion-oriented with sites sharing their experiences and asking questions of one another regarding how CCM can be best operationalized.
We conducted semi-structured qualitative interviews with team members at the nine VAMC sites at two time points—first at baseline before external facilitators actively engaged with the sites (time 1) and then after facilitation support had concluded (time 2). Time 1 interviews were scheduled as close to the start of each wave as possible—5 sites within the same month as facilitation and 4 sites within 1 month after the start of facilitation. The interviewers were members of the larger study team but separate from the implementation facilitation intervention and had no interaction with BHIP team members at the 9 sites outside of the data collection efforts. We recruited participants via email and sent up to 3 emails to BHIP staff participating in the CCM implementation project. The interview questions focused on the extent to which team processes and patient care were aligned with CCM-based principles including work role redesign, patient self-management support, provider decision support, clinical information systems, linkages to community resources, and organization and leadership support (see Table 1 for the codebook of CCM elements, definitions, and examples; see Supplemental file for interview guide). During time 2 interviews, we asked additional questions about whether any changes made during the project could be attributed to implementation facilitation (as opposed to other ongoing projects, initiatives, or contextual factors such as staff hiring or clinic reorganizations). Interviews lasted 30–60 min depending upon participant availability and were digitally audio-recorded and professionally transcribed verbatim. Two participants did not want to be recorded and notes were taken. Interviewers had background in health services research, implementation science, and qualitative methods.
Transcripts were coded using a directed content analysis approach  where we constructed an a priori framework of codes developed from the six CCM elements using NVIVO 10 software. Figure 1 displays our data analysis workflow. Initially, qualitative analysis team members coded the same three randomly selected transcripts independently. Our team met weekly over the course of early coding to establish a reliable and valid coding framework through discussion and consensus.
After determining the framework was reliable and valid, two team members independently coded each transcript using this framework and then met to come to consensus on any coding disagreements. Coding dyad assignments rotated among six team members (JS, BK, CM, KD, SC, ARE). Once coding was completed for each site, one team member created a descriptive summary of the evidence for each CCM element across all respondents at that site and included exemplar quotes for each element. Upon completion of summaries for each CCM element, the same coding team member categorized the extent to which each CCM element had been implemented. The categories we used included “little to no evidence present,” “inconsistently present,” “stably to narrowly present,” “several aspects stably present,” or “stably and broadly established.” Each completed site summary (including both text and ratings) was then reviewed by the larger team (JS, BK, CM, KD, SC, ARE) to come to consensus on the extent each CCM element had been implemented at each site. The ratings were discussed until our group of 5 team members participating in the meetings came to 100% consensus. After reaching consensus on nine site summaries, the study team created a cross-site matrix to compare the extent of CCM implementation across sites .
We conducted 27 interviews at time 1 and 31 interviews at time 2. Of the 58 interviews completed, 39 of these were conducted with unique respondents (i.e., only interviewed at time 1 or time 2). Thus, 19 participants were interviewed at both time 1 and time 2. Each site had between one and five respondents per timepoint. In total, 91 BHIP team members at the 9 sites were invited to participate. There were 33 team members who did not participate in the study—7 declined participation, 2 left the VA, and 24 did not respond after three recruitment emails. All 58 transcripts were included in the analysis for this manuscript. Our sample of 39 BHIP staff consisted of 11 social workers (28%), 9 psychologists (24%), 6 registered nurses (14%), 4 psychiatrists (10%), 4 vocational rehabilitation counselors (10%), and 5 other disciplines on the CCM implementation teams (e.g., addiction counselors, peer support specialists; 14%) participated in this study. Respondents had been involved in providing VA mental health services for a median of 10–15 years (ranging from 2 months to 25 years). In addition, respondents had been on the team for a median of 1.5 years (ranging from 2 months to about 2 years).
At time 1 and time 2, three CCM elements were most frequently present across the sites: work role redesign, patient self-management support, and provider decision support. The CCM elements with increased implementation from time 1 to time 2 were work role redesign, patient self-management support, and clinical information systems. Table 2 displays the extent of CCM element implementation at time 1 and time 2 across all sites. We observed that there was heterogeneity across sites across all CCM elements. Two CCM elements had a mixed change in the extent of implementation, meaning that there were some sites with increased implementation and others with reduced implementation in comparison to time 1. Specifically, within the community linkages element, three sites had an increase in the extent of implementation while two sites had a decrease in the extent of implementation. Similarly, for the organizational leadership and support element, one site had an increase in the extent of implementation while two sites had a decrease in the extent of implementation. The provider decision support element had few changes in the extent of implementation during our study period. In the following sections, we highlight the elements where there was increased or mixed change in the extent of implementation. Table 3 describes CCM element changes participants attributed to the implementation facilitation and Table 4 includes sample quotes for each CCM element discussed below.
CCM elements with increased extent of implementation
Work role redesign
Participants felt that CCM implementation provided them with the time to focus on redesigning care processes that allowed teams to become more efficient. At time 1, staff mentioned having no-show procedures and same-day access procedures in place. Furthermore, time 1 interviews indicated that most teams had regular meeting times established although attendance was variable, and a significant portion of within-team communication occurred informally outside of team meetings. At time 2, staff still discussed same-day patient access and no-show procedures; however, there was more discussion around improved team functioning, cohesion, and coordination. Many respondents discussed developing team structures such as formalized intake procedures for patients, brief huddles for staff, and deliberate changes in team composition or work roles to reduce care fragmentation.
Patient self-management support
Time 1 interviews revealed an emphasis on evidence-based practices involving self-management components (e.g., cognitive behavioral therapy), telephone contact with patients, and use of treatment plans. At time 2, in addition to themes presented above, staff from several sites noted that patients were now attending team meetings to discuss their care. Teams also conducted additional patient education, such as providing brochures or informational packets to patients on the team describing available services and team structure.
Clinical information systems
At time 1, many staff mentioned use of clinical reminders to ensure that required screening tools were used consistently. There was limited evidence of patients and their health information being reviewed collectively as a panel by the teams, and cases were instead considered more often on an individual basis. Additionally, registry-based care where the team can track outcomes across the whole team’s caseload to provide targeted feedback to providers was only highlighted by one provider. At time 2, staff expressed more interest in tracking patients using a patient registry even though most of the teams had not begun utilizing patient registries. At time 2, providers at over half of the sites noted ongoing or upcoming efforts to incorporate measurement-based care into routine practice, involving administration of clinical assessments to guide ongoing care.
CCM elements with mixed change in extent of implementation
At time 1, staff discussed having inconsistent knowledge about community resources such as Vet Centers (which offer outreach, counseling, and referral services to eligible veterans), Alcoholics Anonymous groups, and Veteran Service Organizations. Additionally, referral to community providers was most frequently described as an individual process where providers would make their own referrals based on their knowledge of resources or would get expert consultation from other staff members within or outside of their team. At time 2, familiarity with community resources still varied across team members and providers often were still handling referrals individually. However, at some sites, progress was made on creating and periodically updating guides of available community resources for the teams to use. We also found team meetings provided a structure for more communication about community resources as a team. There were two sites where the extent of implementation decreased due to knowledgeable staff leaving the job or difficulty building community partnerships.
Organizational and leadership support
At time 1, staff at many sites felt that leaders broadly supported the implementation of CCM-based care through resources (e.g., staffing and training) and through having the CCM implemented. Some sites described having leaders who were less directive and less involved with CCM implementation. Often times, team members recognized that different levels of leadership were involved in CCM implementation and that direct supervisors or middle managers created a linkage between executive leaders’ expectations and team member needs. However, at time 2, we found more mixed viewpoints about leadership support for CCM implementation. For example, although some staff reported very supportive leaders, there were more reports of leaders not following through with resources or, in one case, actively not supporting the project.
CCM elements with little to no change in extent of implementation
Provider decision support
At time 1, some staff (e.g., psychologists and social workers) at all 9 sites mentioned utilizing evidence-based psychotherapies to provide care to their patients. In addition, medication algorithms were available at all sites and used to assist in making decisions regarding prescriptions. Sites were more variable in the procedures providers used to seek recommendations from clinical experts and for referring patients to specialty services when needed. At time 2, staff reported very similar levels of implementation within this CCM element. At two sites, there were some limited improvements in information sharing regarding providers’ knowledge and expertise within the team as well as participation from additional staff outside the team to better coordinate care. A few staff mentioned receiving new evidence-based psychotherapy trainings at time 2, but not enough of a change in training to precipitate a change in the extent of implementation at time 2.
In this paper, we have described qualitative analyses regarding the extent to which the six evidence-based elements of CCM-based care were implemented in nine outpatient general mental health clinics. This work builds on our prior quantitative  and qualitative analyses . To our knowledge, our study is the first to use directed content analysis  to investigate change over time in CCM implementation.
After 12 months of implementation facilitation , we found evidence for increased implementation of several CCM elements. This finding appeared strongest for the CCM elements for which infrastructure was already in place at time 1; this likely supported the element’s eventual improved implementation by time 2. For instance, for the element of work role redesign, the start of regular CCM team meetings at time 1 helped bring more structure to previously informal communication among team members, evidenced by improved team communication seen at time 2. For the element of patient self-management, emphasis on evidence-based practices that focus on patient self-management, as well as communicating with patients flexibly (e.g., via telephone) and collaboratively (e.g., completing treatment plans together), was already under way at time 1. At time 2, the evidence of patients actually attending CCM team meetings can be seen as an extension of the pre-existing collaborative approach to involving patients in their own care. For the element of clinical information systems, clinical reminders already existed at time 1 to help ensure that required screenings and symptom measurements are conducted regularly. Then at time 2, we saw of evidence of increased discussion of measurement-based care, which is about actually using the conducted measurements to actively shape care delivery.
We saw little change in the extent of implementation after 12 months of implementation facilitation for the provider decision support CCM element. Although staff consistently utilized evidence-based psychotherapies, medication algorithms, and standardized referrals, there were limited gains in integrated information sharing mechanisms both within the team and outside the team. However, teams felt they had established techniques for obtaining the expertise they needed. At time 1, we found that spontaneous discussions and outreach were the modalities for information exchange which is consistent with the concepts of healthcare teamwork . These results suggest the importance of strong interpersonal relationships among team members and other interfacing providers in line with relational coordination . Further, the use of personal coordination was supplemented with more formal standardized coordination (e.g., referrals placed through the electronic health record) suggesting the importance of both types of coordination in communicating within the team and with other providers outside of the team .
Our results indicate that when an implementation effort was perceived to have some leadership support at the start of implementation, it did not necessarily mean that the perception held at the end of implementation. Demonstrated leadership support (e.g., engagement, involvement, and actions) has been shown to be a distinguishing factor in sustainability [19, 20]. Thus, organizations that are looking to sustainably implement the CCM may want to plan and conduct explicit and ongoing steps to collaborate with leadership to help support front-line implementation teams throughout the entire duration of implementation. Although team members can take steps to try and convince leadership through meetings, presentations, and data on patient satisfaction or clinical outcomes, leadership ultimately make their decision about support based on other contextual factors outside of the general mental health team. Thus, teams need to think carefully about how best to engage leadership to maximize leadership support especially focused on middle managers who often make decisions about resources and training in line with executive leadership’s vision. Limited funding and resources have been found to be barriers to sustainment in previous studies , and the implementation science field is increasingly calling for an increased focus on mechanisms that specifically promote sustainability . To this end, we are currently pursuing recently funded subsequent work trialing CCM implementation that targets sustainment of CCM elements beyond the duration of active facilitation.
Our study had several strengths and limitations. This work was part of a rigorous implementation trial collecting data both on health outcomes of CCM implementation in general mental health clinics, as well as implementation outcomes such as CCM-based clinical impacts over a 1-year period. Thus, we have a rich dataset available in which to put the CCM implementation intervention in context. In addition, this study represents the first qualitative description of how care processes aligned with the evidence-based CCM elements changed over time during a trial in outpatient mental health care teams. However, there were also some limitations to report. The number of participating team members varied by site, as did the staff interviewed at both time points. In cases where sites had a small number of respondents, we were cautious in interpreting CCM elements’ extent of implementation. Also, we were not able to examine whether underlying CCM-oriented care practices were a result of the monthly conference calls that were a part of the control condition through monitoring of these conference calls and identifying practices that result from these calls that may have served as implementation strategies themselves. Future studies ought to consider explicitly incorporating such examination. Finally, we focused available study resources on assessing the extent of CCM implementation only from the perspective of BHIP-CCM team members. We might have gained additional insights on CCM implementation by including other stakeholder perceptions outside of the team.
There are several suggestions for future work to build upon current findings. Given renewed interest in identifying the mechanisms by which healthcare processes and implementation strategies exert their effects, studies that explicitly elucidate the relationship between CCM implementation and clinical outcomes are needed. For instance, building on our finding that perceived leadership support at the beginning of implementation did not necessarily hold at the end of implementation, subsequent research should assess the impact of change in leadership support on clinical outcomes over time. Additionally, further work should examine how care structures and processes based on the CCM can be incorporated into general mental health care delivery beyond the VA, especially given how contexts (e.g., available resources) may vary widely among healthcare systems. Also, whether the extent of the step-down facilitation effort (which varied across sites for this study) impacts sustainment of CCM elements is an open question; as mentioned above, we currently have a subsequent study under way that focuses on CCM sustainment, and we look forward to assessing this relationship as part of that work. Finally, more work needs to be done to assess the type, amount, and content of implementation facilitation support needed by contextually different sites implementing the CCM.
While considering our findings’ implications for future CCM implementation, we revisited the implications that we previously reported , to carefully reflect on whether any updates are needed based on this current study’s findings. The first of three previously reported implications was that many respondents were in favor of the overarching goal of CCM-based care (i.e., care delivery that is more coordinated, anticipatory, and evidence-based). We found this to consistently be the case following 12 months of implementation, demonstrated through (i) increased team structures and processes, (ii) communication of the structures and available services to Veterans, and (iii) incorporation of routine clinical assessments to guide ongoing care. These changes also speak to the second previously reported implication, of CCM elements being pursued separately from formal systems and structures. Namely, they exemplify the formalization of previously more impromptu discussions and outreach that characterized the work of our participating provider teams. The third of the three previously reported implications was the need for adequate human resources, funding, and workforce development for CCM implementation. Findings from this current study reiterate this need, as they highlight participants’ perspectives on (i) knowledgeable staff being key to maintaining community partnerships, (ii) whether their leadership has delivered on promised resources, and (iii) experience with trainings for enhancing staff’s abilities to deliver evidence-based care. In this study, we observed that all CCM elements required participation from staff and stakeholders outside the team to make progress on CCM elements. In this study, the distinction between processes that were within versus outside of the team’s control was particularly important. Our results indicate the salience of stakeholder engagement, building collaborations across services, and identifying processes that may be difficult to tackle because they require many people with different priorities to work in concert.
Availability of data and materials
The data analyzed during the current study are not publicly available because individual privacy could be compromised.
Behavioral Health Interdisciplinary Program
Cognitive Behavioral Therapy
Collaborative Chronic Care model
Electronic Health Record
Office of Mental Health and Suicide Prevention
Department of Veterans Affairs
Veterans Affairs Medical Center
Bauer MS, Miller C, Kim B, Lew R, Weaver K, Coldwell C, et al. Partnering with health system operations leadership to develop a controlled implementation trial. Implement Sci. 2016;11(1):1.
Von Korff M, Gruman J, Schaefer J, Curry SJ, Wagner EH. Collaborative management of chronic illness. Ann Intern Med. 1997;127(12):1097–102. https://doi.org/10.7326/0003-4819-127-12-199712150-00008.
Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. Milbank Q. 1996;74(4):511–44. https://doi.org/10.2307/3350391.
Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the chronic care model in the new millennium. Health Aff. 2009;28(1):75–85. https://doi.org/10.1377/hlthaff.28.1.75.
Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: Systematic review and meta-analysis. Am J Psychiatry. 2012;169(8):790–804. https://doi.org/10.1176/appi.ajp.2012.11111616.
Miller CJ, Grogan-Kaylor A, Perron BE, Kilbourne AM, Woltmann E, Bauer MS. Collaborative chronic care models for mental health conditions: cumulative meta-analysis and metaregression to guide future research and implementation. Med Care. 2013;51(10):922–30. https://doi.org/10.1097/MLR.0b013e3182a3e4c4.
Smith SN, Almirall D, Prenovost K, Liebrecht C, Kyle J, Eisenberg D, Bauer MS, Kilbourne AM. Change in patient outcomes after augmenting a low-level implementation strategy in community practices that are slow to adopt a collaborative chronic care model: a cluster randomized implementation trial. Med Care. 2019;57(7):503–11. https://doi.org/10.1097/MLR.0000000000001138.
Unützer J, Carlo AC, Arao R, Vredevoogd M, Fortney J, Powers D, et al. Variation In The effectiveness of collaborative care for depression: does it matter where you get your care? Study examines variations in outcomes for patients who received treatment for depression in primary care settings using the collaborative care model. Health Aff. 2020;39(11):1943–50.
Bauer MS, Weaver K, Kim B, Miller C, Lew R, Stolzmann K, Sullivan JL, Riendeau R, Connolly S, Pitcock J, Ludvigsen SM, Elwy AR. The collaborative chronic care model for mental health conditions: from evidence synthesis to policy impact to scale-up and spread. Med Care. 2019;57(10 Suppl 3):S221–7. https://doi.org/10.1097/MLR.0000000000001145.
Kirchner JE, Ritchie MJ, Pitcock JA, Parker LE, Curran GM, Fortney JC. Outcomes of a partnered facilitation strategy to implement primary care–mental health. J Gen Intern Med. 2014;29(4):904–12. https://doi.org/10.1007/s11606-014-3027-2.
Bauer MS, Miller CJ, Kim B, Lew R, Stolzmann K, Sullivan J, et al. Effectiveness of implementing a collaborative chronic care model for clinician teams on patient outcomes and health status in mental health: a randomized clinical trial. JAMA Netw Open. 2019;2(3):e190230–e.
Miller CJ, Sullivan JL, Kim B, Elwy AR, Drummond KL, Connolly S, Riendeau RP, Bauer MS. Assessing collaborative care in mental health teams: qualitative analysis to guide future implementation. Admin Policy Mental Health Mental Health Serv Res. 2019;46(2):154–66. https://doi.org/10.1007/s10488-018-0901-y.
Lew RA, Miller CJ, Kim B, Wu H, Stolzmann K, Bauer MS. A method to reduce imbalance for site-level randomized stepped wedge implementation trial designs. Implement Sci. 2019;14(1):46. https://doi.org/10.1186/s13012-019-0893-3.
Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88. https://doi.org/10.1177/1049732305276687.
Kim B, Sullivan JL, Ritchie MJ, Connolly SL, Drummond KL, Miller CJ, Greenan MA, Bauer MS. Comparing variations in implementation processes and influences across multiple sites: what works, for whom, and how? Psychiatry Res. 2020;283:112520. https://doi.org/10.1016/j.psychres.2019.112520.
Miller CJ, Kim B, Silverman A, Bauer MS. A systematic review of team-building interventions in non-acute healthcare settings. BMC Health Serv Res. 2018;18(1):146. https://doi.org/10.1186/s12913-018-2961-9.
Gittell JH. Relational coordination: Guidelines for theory, measurement and analysis. Waltham: Brandeis University; 2011.
Benzer JK, Cramer IE, Burgess JF, Mohr DC, Sullivan JL, Charns MP. How personal and standardized coordination impact implementation of integrated care. BMC Health Serv Res. 2015;15(1):448. https://doi.org/10.1186/s12913-015-1079-6.
Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39(1):55–76. https://doi.org/10.1146/annurev-publhealth-040617-014731.
Penno LN, Davies B, Graham ID, Backman C, MacDonald I, Bain J, et al. Identifying relevant concepts and factors for the sustainability of evidence-based practices within acute care contexts: a systematic review and theory analysis of selected sustainability frameworks. Implement Sci. 2019;14(1):108. https://doi.org/10.1186/s13012-019-0952-9.
Hailemariam M, Bustos T, Montgomery B, Barajas R, Evans LB, Drahota A. Evidence-based intervention sustainability strategies: a systematic review. Implement Sci. 2019;14(1):1–12.
Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):1–13.
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs. The following individuals contributed substantively to the conduct of this study. Each was affiliated with the US Department of Veterans Affairs during this work and received a salary for these efforts. Oversight of data collection and human participant documentation: Marjorie Nealon Siebert. Data collection at time 1: Sally Holmes and Cara Frigand. Participation in team meetings discussing the analysis and results: Alicia Williamson, Michael Ruderman, Allie Silverman, and Madisen Brown. The results of this study were presented at the Annual AcademyHealth Meeting; June, 2019; Washington, DC.
This study was funded by grants QUE-15-289 and CIN-13-403 from the VA and was designed and conducted in partnership with the VA Office of Mental Health and Suicide Prevention.
Ethics approval and consent to participate
This study was approved by the VA Boston Healthcare System Institutional Review Board. All participants provided informed consent for the study.
Consent for publication
Dr Bauer reported receiving royalties from Springer Publishing and New Harbinger Publishing for books related to the CCM during the conduct of the study. Dr Miller reported grants from US Department of Veterans Affairs, Health Services Research and Development Service during the conduct of the study. Dr Elwy reported grants from Department of Veterans Affairs, Health Services Research and Development Service, Quality Enhancement Research Initiative during the conduct of the study. Dr. Sullivan reported grants from Department of Veterans Affairs, Health Services Research and Development Service, Quality Enhancement Research Initiative during the conduct of the study. No other disclosures were reported.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Sullivan, J.L., Kim, B., Miller, C.J. et al. Collaborative chronic care model implementation within outpatient behavioral health care teams: qualitative results from a multisite trial using implementation facilitation. Implement Sci Commun 2, 33 (2021). https://doi.org/10.1186/s43058-021-00133-w
- Chronic care
- Mental health
- Qualitative research