Skip to main content
Download PDF
Download ePub

Best–worst scaling methodology to evaluate constructs of the Consolidated Framework for Implementation Research: application to the implementation of pharmacogenetic testing for antidepressant therapy

Implementation Science Communications volume 3, Article number: 52 (2022) Cite this article

Abstract

Background

Despite the increased demand for pharmacogenetic (PGx) testing to guide antidepressant use, little is known about how to implement testing in clinical practice. Best–worst scaling (BWS) is a stated preferences technique for determining the relative importance of alternative scenarios and is increasingly being used as a healthcare assessment tool, with potential applications in implementation research. We conducted a BWS experiment to evaluate the relative importance of implementation factors for PGx testing to guide antidepressant use.

Methods

We surveyed 17 healthcare organizations that either had implemented or were in the process of implementing PGx testing for antidepressants. The survey included a BWS experiment to evaluate the relative importance of Consolidated Framework for Implementation Research (CFIR) constructs from the perspective of implementing sites.

Results

Participating sites varied on their PGx testing platform and methods for returning recommendations to providers and patients, but they were consistent in ranking several CFIR constructs as most important for implementation: patient needs/resources, leadership engagement, intervention knowledge/beliefs, evidence strength and quality, and identification of champions.

Conclusions

This study demonstrates the feasibility of using choice experiments to systematically evaluate the relative importance of implementation determinants from the perspective of implementing organizations. BWS findings can inform other organizations interested in implementing PGx testing for mental health. Further, this study demonstrates the application of BWS to PGx, the findings of which may be used by other organizations to inform implementation of PGx testing for mental health disorders.

Peer Review reports

Background

Depression and anxiety disorders are common in the US, affecting approximately one in five people [1]. These conditions are frequently treated with antidepressants; however, identifying optimal treatment(s) for a given patient can be challenging when considering side-effects, patient comorbidities, clinical symptoms, concomitant medications, and prior treatment history [2]. Antidepressant medication trials can be a frustrating process that may take weeks while patients wait for the medications to elicit their full effects. Up to 42% of the variation in response to antidepressants may be explained by genetic factors [3] and there is a growing body of research supporting the clinical utility of pharmacogenetic (PGx) information to guide drug dosing or selection [4].

Pharmacogenetic testing, like other novel interventions, is subject to similar challenges of uptake and adoption into healthcare. Assessing internal and external organizational barriers to implementation is an essential step to facilitate the adoption of novel interventions. Implementation science with its robust methods, including defined and validated frameworks, strategies, and outcome measures, can be leveraged to accelerate PGx implementation as applied to mental health clinical practice [5]. With the growing body of evidence motivating providers and patients to seek PGx testing to guide the prescribing of medications for treating mental health conditions, successful approaches to implementation for antidepressants need to be identified. One dynamic approach for the appraisal of choices in health-related settings is the stated preferences approach. Stated preferences approaches are increasingly being applied to as an assessment tool when engaging implementation stakeholders [6]. We highlight herein the utility of applying a stated preferences approach to evaluate the relative importance of constructs in the Consolidated Framework for Implementation Research (CFIR), and how latent class analyses may extend and enhance the interpretation of findings by identifying latent class membership among respondents according to their organizational characteristics.

Methods

Setting

This study was conducted within the Implementing Genomics in Practice (IGNITE) network—a multidisciplinary consortium in the United States (US) focused on the development, implementation, and dissemination of methods that integrate genomic medicine into clinical care [7, 8]. Informed by the CFIR, IGNITE researchers previously identified constructs that were critical to the adoption of genomic medicine and PGx [9,10,11]. In this paper, we focused on examining latent classes to better identify the specific factors within the CFIR that are important for the implementation of PGx testing for antidepressant prescribing by organizations. The survey targeted funded and affiliate members of the IGNITE network that had either implemented or were planning to implement clinical genotyping to guide antidepressant prescribing.

Study procedures

We developed an electronic survey to (1) measure institutional and practice environment characteristics and (2) to evaluate factors important for the implementation of PGx testing to guide antidepressant therapy using stated preferences [11] (Additional file 1). Specifically, the experiment was designed using the best–worst scaling (BWS) technique—a methodology for assessing priorities by asking respondents what they view as best and worst amongst a given set of factors [12]. Respondents were asked to provide a consensus on prioritization from their site rather than respond based on their own perspectives only. The online survey was administered between September and December 2020 to 17 sites in the IGNITE network that had responded to a previous survey about institutional characteristics and programmatic drivers [13] and completed by organizational representatives leading implementation efforts with input from mental health providers. Prior to its administration, the survey was pilot tested among members of the research team with experience in PGx implementation. Both the survey data collection and the BWS analysis were conducted using Lighthouse Studio (Version 9.9.2; Sawtooth Software, Provo, UT).

Measures

We used BWS to identify which factors were most important for implementing PGx for antidepressants. The BWS exercise evaluated constructs from the CFIR because it is a widely applied, stakeholder-engaging framework that has informed implementation efforts across IGNITE [9, 10]. The CFIR includes 37 constructs organized into five major domains that influence the implementation of interventions: (1) outer setting, (2) inner setting, (3) characteristics of individuals, (4) intervention characteristics, and (5) process of implementation. The constructs for each of the CFIR domains comprised 5 independent BWS exercises. A copy of the survey is available in the supplementary material.

Experimental design

The BWS technique has been applied to evaluate preferences across a range of healthcare applications [12]. This technique is useful for quantitatively prioritizing a relatively large number of observed factors while maintaining cognitive and administrative simplicity [14]. BWS requires that respondents choose the best (highest ranking) and worst (lowest ranking) factors in a series of choice tasks, thus yielding the relative importance of these factors. The BWS exercise asked respondents to rank all constructs of the CFIR within each of its five domains—outer setting (4 factors evaluated in 3 choice sets), inner setting (14 factors evaluated in 9 choice sets), intervention characteristics (9 factors evaluated in 6 choice sets), characteristics of individuals (5 factors evaluated in 3 choice sets), and process (8 factors evaluated in 5 choice sets). In addition to CFIR constructs, the BWS domains included three additional constructs from the Genomic Medicine Integrative Research (GMIR) Framework [15] that may be relevant to PGx testing (social determinants of health, ability of system to educate patients and clinicians, and patient needs and resources: bio-psychosocial factors). The GMIR Framework was developed to inform the integration of genomic medicine into clinical practice and includes four domains: healthcare system factors, social determinants, clinician factors, and individual and family factors [15]. An example choice set is presented in Fig. 1.

Fig. 1
figure 1

Example of one choice set from the best–worst scaling experiment representing the Inner Setting domain of the Consolidated Framework for Implementation Research

Full size image

Data analysis

We used multinomial logistic regression in Sawtooth Software to estimate the probabilities of respondents choosing particular alternatives. Probability scores were then transformed into scaled probability scores to allow for comparisons across constructs. Additionally, we applied a Bayesian approach using a Monte Carlo Markov chain to compare and update respondents’ estimates on the basis of the distribution of preferences from other respondents. The individual utility estimates of each construct were averaged after 10,000 random draws.

We used Sawtooth Software’s latent class analysis module to identify latent classes of respondents with similar choice patterns. The latent class models assumed that the presentation of alternative constructs could have heterogenous effects on choices across latent classes of respondents. Given the small sample size in this study, we limited the latent class specification to two classes for each of the models. The latent class formula computes the probability that each participant is a member of each class. To account for this heterogeneity, the models assume that there are latent classes within the sample such that each class has preference weights that are identical within the class and that are systematically different from preference weights in other classes. Within each latent class, the preference weights were estimated using conditional logit. Summary statistics for site characteristics (whether PGx testing has been implemented or was being planned, and whether the participating organization is [or not] affiliated with an academic medical center) were estimated by latent class. The study was approved by the University of Florida Institutional Review Board. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies were used to guide research reporting (Additional file 2) [16].

Results

Relative importance scores for the overall sample

All 17 IGNITE sites that were invited to participate completed the survey (response rate = 100%) (Additional file 3). Results of the institutional and practice environment characteristics component of the survey have been reported elsewhere [11]. Figure 2 illustrates the relative importance scores for each construct organized by CFIR domain across the entire sample (n = 17). Patient needs and resources—biopsychosocial factors was rated as the most important construct (relative score, 41.7%) for outer setting, leadership engagement (relative score, 16.6%) for inner setting, knowledge and beliefs about the intervention (relative score 49.9%) for characteristics of individuals, evidence strength and quality (relative score, 27.9%) for intervention characteristics, and champions (relative score, 26.9%) for process.

Fig. 2
figure 2

Relative importance of each construct from the Consolidated Framework for Implementation Research (CFIR) organized by domain applied to the implementation of pharmacogenetics testing to guide antidepressant treatment. *Additional constructs from the Genomic Medicine Integrative Research Framework

Full size image

Relative importance scores and site characteristics by latent class

Table 1 summarizes the importance scores by latent class for the highest-ranked constructs within each CFIR domain, latent class membership (i.e., proportion of sites in each latent class), and site characteristics by class (i.e., proportion of sites within each class with the corresponding characteristic). For outer setting, the highest-ranked construct in class 1 (i.e., highest importance score) was patient needs and resources—biopsychosocial factors (42.7%) compared with external policy and incentives (49.5%) in class 2. For inner setting, the highest-ranked construct in class 1 was available resources (15.3%) compared with leadership engagement (19.6%) in class 2. For characteristics of individuals, the highest-ranked construct in class 1 was individual stage of change (40.8%) compared with knowledge and beliefs about the intervention (52.4%) in class 2. For intervention characteristics, the highest-ranked construct in class 1 was relative advantage (27.8%) compared with evidence strength and quality (29.0%) for class 2. Finally, for process, the highest-ranked construct in class 1 was champions (30.6%) compared with formally appointed internal implementation leaders (31.0%) in class 2. When compared with demographic characteristics of the organization, neither of the two classes was more likely than the other to have a particular identity (academic medical center or had already implemented PGx testing), although more centers that had already implemented were classified in class 2 and academic medical centers were more often classified in class 1.

Table 1 Importance scores and latent class membership for the highest-ranked constructs within each domain and site characteristics by class
Full size table

Discussion

This study demonstrates the use of BWS methodology to assess the prioritization of contextual factors for the implementation of PGx testing to guide antidepressant therapy. Our analyses revealed that patient needs and resources, leadership engagement, knowledge and beliefs about the intervention, evidence strength and quality, and identifying champions were the most important constructs across the five CFIR domains from the organizational perspective when implementing PGx testing. These preferences differed little by key characteristics of the organization (i.e., whether the site had implemented PGx testing and whether it was an academic medical center). Although BWS methods have been used to inform implementation of evidence-based practices [6], many applications focus on the patient or the individual receiving the intervention, and few studies have used stated preferences to assess the perspective of the implementing organization. The results from our study provide important information for institutions seeking to advance precision medicine approaches to mental health care, and the latent class analysis allows for identification of classes with different priorities.

Applying latent class analysis to the CFIR construct prioritization can inform the developing and tailoring of PGx testing and implementation strategies by identifying priorities by classes of implementers. There are several caveats of latent class analysis that merit further consideration. While grouping based on latent class facilitates data presentation and interpretation, participating organizations do not actually belong to a single group, and class membership for each organization is assigned based on the highest probability of belonging to one of the latent classes. Ranking priorities can help set a roadmap for planning implementation and the order in which certain implementation strategies should be deployed. The BWS rankings of the CFIR constructs were very similar by class membership with subtle differences. CFIR domains for which the rankings were different between the two classes included Inner Setting, where available resources was ranked higher in class 1 and leadership engagement was ranked higher in class 2; Intervention characteristics, where relative advantage was ranked higher in class 1 and evidence strength and quality was ranked higher in class 2; Characteristics of individuals, individual stage of change was ranked higher in class 1 and knowledge and beliefs about the intervention in class 2; and Process, where preparing champions ranked higher in class 1 and formally appointed internal implementation leader ranked higher in class 2. These differences may reflect the stage of implementation as those centers that had already implemented PGx testing more often comprised membership in class 2.

Implementers and mental health clinicians highly value the strength and quality of the evidence linking genotype with drug response when designing their implementations [17, 18]. Such evidence informs decisions made in the implementation process, including which genes/alleles to test, which drugs and patient populations to target, and what clinical recommendations to provide based on genotype results. A challenge with evidence in the area of antidepressant pharmacogenetics is that while, several randomized controlled trials have shown the benefit of PGx testing [19,20,21,22] on remission rates in patients treated for depression, these trials used different combinations of genes and different proprietary dosing algorithms in tailoring antidepressants, which makes the interpretation of the clinical utility of PGx testing challenging for practicing clinicians. As such, implementers may need to review the entirety of the clinical evidence, and especially consider the strength of the evidence for genetic associations with antidepressant response, in designing implementations. Stakeholders are also weighing the advantage of PGx testing as compared to prescribing medications without testing, recognizing that these advantages need to be established in their specific clinical environment. [17]. Institutions who are more experienced with PGx implementation may recognize the value of leadership engagement in supporting the initial infrastructure needs to start PGx testing, as well as to help sustain the program with financial, capital, and personnel needs. For example, for genotyping to be performed during a hospitalization, the cost is covered under the diagnosis related group (DRG), but there must be leadership buy-in for this to happen and to be sustained as institutional priorities change. Institutions still in the planning phase are likely prioritizing an evaluation of the additional resources required to implement a PGx program, such as for hiring PGx experts, establishing and validating an in-house assay for performing the testing, and designing and building informatics resources to integrate PGx results into electronic health records [23]. Implementers placed emphasis on users’ knowledge and beliefs as the skilled and enthusiastic use of PGx testing will inform prescribing decisions and help optimize drug therapy for a patient. Preparing of physician champions is well-recognized strategy for implementation as champions can serve as resources for building clinical decision support and for educating and supporting other physicians in how to order testing and apply the results to prescribing decisions [24]. Institutions that are more experienced with implementation may realize this strategy alone is not sufficient for implementation success and have appointed formal implementation leaders to oversee all phases of implementation and provide a cohesive strategy. Dedicated precision medicine or PGx teams have been cited as leading these types of implementations [25, 26]. Another construct that was identified as important but not necessarily different by class included patient needs and resources. Educating patients about their PGx results along with how it may impact response to drug therapy is an important component of PGx implementation. Greater patient knowledge about genetics has been associated with favorable attitudes towards PGx testing [27]. Patient resources such as the ability to pay for testing or insurance coverage for testing will also need to be considered.

These results highlight how priorities may shift as organizations progress through the various phases of implementation. However, some organizations may have different priorities based on their mission, organizational setting, and the populations they serve. Therefore, results may reflect inherent differences in organizational characteristics rather than an evolution through implementation maturity. Here it is worthwhile to consider that BWS latent class analysis may be most useful as a first step in a sequential mixed methods study in which the next step would involve qualitative research to explain and confirm the BWS findings with stakeholders.

Few centers have formalized processes for clinical PGx testing due to several implementation challenges [28, 29]. Our latent class analysis of the BWS ranking of CFIR constructs in centers that have implemented or are planning to implement PGx testing aligns with the results of previous studies that have reported on facilitators and barriers of PGx implementation [17, 28,29,30]. These studies have previously identified the limited evidence for clinical utility, unclear cost-effectiveness for PGx testing, and limited physician education regarding PGx testing as major barriers limiting PGx adoption. These barriers were identified by providers or physicians, who are important, but not the sole stakeholders in the implementation process. Additionally, reimbursement for PGx testing is rapidly evolving with Medicare and several private insurers now reimbursing PGx testing [31]. While the constructs of evidence strength and quality, cost, and knowledge and beliefs were also identified in our study, we previously published the process these early adopter sites deployed in overcoming these challenges when implementing PGx testing for antidepressants such as creating educational modules for providers about PGx [11].

This study has some limitations. The sample size was small, limiting statistical inference related to differences in priorities by class. Respondents in the study represented US-based organizations participating in the IGNITE network; therefore, the findings reflect priorities of early adopters of PGx testing in the US and the rankings of CFIR constructs may be quite different for subsequent adopters of PGx testing and for implementers outside of the US. Representatives from each of the 17 sites were asked to respond to the survey providing a consensus on prioritization from their site, which may have varied in terms of composition (i.e., implementers, mental health providers, pharmacists, administrators) and could have influenced the survey response. However, processes for implementing PGx testing and factors important for implementation are relevant for centers seeking to begin implementation.

Conclusion

This study demonstrates the feasibility of applying BWS methodology from the perspectives of the organization to capture the importance of factors for implementing pharmacogenetic testing. Organizations were consistent and ranked patient needs and resources, leadership engagement, knowledge and beliefs about the intervention, evidence strength and quality, and identifying champions as the most important constructs for pharmacogenetic implementation. Experience from early adopters in our study identifies which important constructs and the order in which they should be addressed when designing strategies for the implementation of PGx testing. These rankings were confirmed by the latent class analysis. Future research should evaluate the use of BWS when evaluating implementation strategies for PGx testing.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

PGx:

Pharmacogenetic

IGNITE:

Implementing Genomics in Practice network

CFIR:

Consolidated Framework for Implementation Research

BWS:

Best-worst scaling

References

  1. Ornstein SM, Nietert PJ, Jenkins RG, Litvin CB. The prevalence of chronic diseases and multimorbidity in primary care practice: a PPRNet report. J Am Board Fam Med. 2013;26(5):518–24.

    Article  Google Scholar 

  2. Gabriel FC, de Melo DO, Fraguas R, Leite-Santos NC. Mantovani da Silva RA, Ribeiro E: Pharmacological treatment of depression: a systematic review comparing clinical practice guideline recommendations. PLoS ONE. 2020;15(4):e0231700.

    Article  CAS  Google Scholar 

  3. Tansey KE, Guipponi M, Hu X, Domenici E, Lewis G, Malafosse A, Wendland JR, Lewis CM, McGuffin P, Uher R. Contribution of common genetic variants to antidepressant response. Biol Psychiatry. 2013;73(7):679–82.

    Article  CAS  Google Scholar 

  4. Bousman CA, Bengesser SA, Aitchison KJ, Amare AT, Aschauer H, Baune BT, Asl BB, Bishop JR, Burmeister M, Chaumette B, et al. Review and consensus on pharmacogenomic testing in psychiatry. Pharmacopsychiatry. 2021;54(1):5–17.

    Article  Google Scholar 

  5. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24–34.

    Article  Google Scholar 

  6. Salloum RG, Shenkman EA, Louviere JJ, Chambers DA. Application of discrete choice experiments to enhance stakeholder engagement as a strategy for advancing implementation: a systematic review. Implement Sci. 2017;12(1):140.

    Article  Google Scholar 

  7. Weitzel KW, Alexander M, Bernhardt BA, Calman N, Carey DJ, Cavallari LH, Field JR, Hauser D, Junkins HA, Levin PA, et al. The IGNITE network: a model for genomic medicine implementation and research. BMC Med Genomics. 2016;9(1):1.

    Article  Google Scholar 

  8. Cavallari LH, Lee CR, Beitelshees AL, Cooper-DeHoff RM, Duarte JD, Voora D, Kimmel SE, McDonough CW, Gong Y, Dave CV, et al. Multisite investigation of outcomes with implementation of CYP2C19 genotype-guided antiplatelet therapy after percutaneous coronary intervention. JACC Cardiovasc Interv. 2017;11(2):181–91.

    Article  Google Scholar 

  9. Sperber NR, Carpenter JS, Cavallari LH, Damschroder LJ, Cooper-DeHoff RM, Denny JC, Ginsburg GS, Guan Y, Horowitz CR, Levy KD, et al. Challenges and strategies for implementing genomic services in diverse settings: experiences from the Implementing GeNomics In pracTicE (IGNITE) network. BMC Med Genomics. 2017;10(1):35.

    Article  Google Scholar 

  10. Orlando LA, Sperber NR, Voils C, Nichols M, Myers RA, Wu RR, Rakhra-Burris T, Levy KD, Levy M, Pollin TI, et al. Developing a common framework for evaluating the implementation of genomic medicine interventions in clinical care: the IGNITE Network’s Common Measures Working Group. Genet Med. 2018;20(6):655–63.

    Article  Google Scholar 

  11. Tuteja S, Salloum RG, Elchynski AL, Smith DM, Rowe E, Blake KV, Limdi NA, Aquilante CL, Bates J, Beitelshees AL, et al. Multisite evaluation of institutional processes and implementation determinants for pharmacogenetic testing to guide antidepressant therapy. Clin Transl Sci. 2022;15(2):371–83.

    Article  Google Scholar 

  12. Flynn TN, Louviere JJ, Peters TJ, Coast J. Best–worst scaling: what it can do for health care research and how to do it. J Health Econ. 2007;26(1):171–89.

    Article  Google Scholar 

  13. Tuteja S, Salloum RG, Ekchynski A, Smith M, Blake KV, Rowe E, Limdi NA, Aquilante CL, Bates J, Beitelshees AL, et al: Multi-site evaluation of institutional processes and implementation framework characteristics for pharmacogenetic testing to guide antidepressant therapy. Clin Transl Sci. 2022;15(2):371-383.

  14. Gallego G, Bridges JF, Flynn T, Blauvelt BM, Niessen LW. Using best-worst scaling in horizon scanning for hepatocellular carcinoma technologies. Int J Technol Assess Health Care. 2012;28(3):339–46.

    Article  Google Scholar 

  15. Horowitz CR, Orlando LA, Slavotinek AM, Peterson J, Angelo F, Biesecker B, Bonham VL, Cameron LD, Fullerton SM, Gelb BD, et al. The genomic medicine integrative research framework: a conceptual framework for conducting genomic medicine research. Am J Hum Genet. 2019;104(6):1088–96.

    Article  CAS  Google Scholar 

  16. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–7.

    Article  Google Scholar 

  17. Vest BM, Wray LO, Brady LA, Thase ME, Beehler GP, Chapman SR, Hull LE, Oslin DW. Primary care and mental health providers’ perceptions of implementation of pharmacogenetics testing for depression prescribing. BMC Psychiatry. 2020;20(1):518.

    Article  Google Scholar 

  18. Luzum JA, Petry N, Taylor AK, Van Driest SL, Dunnenberger HM, Cavallari LH. Moving pharmacogenetics into practice: it’s all about the evidence! Clin Pharmacol Ther. 2021;110(3):649–61.

    Article  Google Scholar 

  19. Singh AB. Improved antidepressant remission in major depression via a pharmacokinetic pathway polygene pharmacogenetic report. Clin Psychopharmacol Neurosci. 2015;13(2):150–6.

    Article  Google Scholar 

  20. Pérez V, Salavert A, Espadaler J, Tuson M, Saiz-Ruiz J, Sáez-Navarro C, Bobes J, Baca-García E, Vieta E, Olivares JM, et al. Efficacy of prospective pharmacogenetic testing in the treatment of major depressive disorder: results of a randomized, double-blind clinical trial. BMC Psychiatry. 2017;17(1):250.

    Article  Google Scholar 

  21. Bradley P, Shiekh M, Mehra V, Vrbicky K, Layle S, Olson MC, Maciel A, Cullors A, Garces JA, Lukowiak AA. Improved efficacy with targeted pharmacogenetic-guided treatment of patients with depression and anxiety: a randomized clinical trial demonstrating clinical utility. J Psychiatr Res. 2018;96:100–7.

    Article  Google Scholar 

  22. Greden JF, Parikh SV, Rothschild AJ, Thase ME, Dunlop BW, DeBattista C, Conway CR, Forester BP, Mondimore FM, Shelton RC, et al. Impact of pharmacogenomics on clinical outcomes in major depressive disorder in the GUIDED trial: A large, patient- and rater-blinded, randomized, controlled study. J Psychiatr Res. 2019;111:59–67.

    Article  Google Scholar 

  23. Liu M, Vnencak-Jones CL, Roland BP, Gatto CL, Mathe JL, Just SL, Peterson JF, Van Driest SL, Weitkamp AO. A tutorial for pharmacogenomics implementation through end-to-end clinical decision support based on ten years of experience from PREDICT. Clin Pharmacol Ther. 2021;109(1):101–15.

    Article  Google Scholar 

  24. Arwood MJ, Chumnumwat S, Cavallari LH, Nutescu EA, Duarte JD. Implementing pharmacogenomics at your institution: establishment and overcoming implementation challenges. Clin Transl Sci. 2016;9(5):233–45.

    Article  CAS  Google Scholar 

  25. Duarte JD, Dalton R, Elchynski AL, Smith DM, Cicali EJ, Lee JC, Duong BQ, Petry NJ, Aquilante CL, Beitelshees AL, et al. Multisite investigation of strategies for the clinical implementation of pre-emptive pharmacogenetic testing. Genet Med. 2021;23(12):2335–41.

    Article  Google Scholar 

  26. Empey PE, Stevenson JM, Tuteja S, Weitzel KW, Angiolillo DJ, Beitelshees AL, Coons JC, Duarte JD, Franchi F, Jeng LJB, et al. Multisite investigation of strategies for the implementation of CYP2C19 genotype-guided antiplatelet therapy. Clin Pharmacol Ther. 2018;104(4):664–74.

    Article  Google Scholar 

  27. Lee G, Varughese LA, Conway L, Stojinski C, Ashokkumar S, Monono K, Matthai W, Kolansky DM, Giri J, Tuteja S. Attitudes toward pharmacogenetics in patients undergoing CYP2C19 testing following percutaneous coronary intervention. Pers Med. 2022;19(2):93–101.

    Article  CAS  Google Scholar 

  28. Virelli CR, Mohiuddin AG, Kennedy JL. Barriers to clinical adoption of pharmacogenomic testing in psychiatry: a critical analysis. Transl Psychiatry. 2021;11(1):509.

    Article  Google Scholar 

  29. Jameson A, Fylan B, Bristow GC, Sagoo GS, Dalton C, Cardno A, Sohal J, McLean SL. What are the barriers and enablers to the implementation of pharmacogenetic testing in mental health care settings? Front Genet. 2021;12:740216.

    Article  Google Scholar 

  30. Hull LE, Lynch KG, Oslin DW. VA primary care and mental health providers’ comfort with genetic testing: survey results from the PRIME care study. J Gen Intern Med. 2019;34(6):799–801.

    Article  Google Scholar 

  31. Empey PE, Pratt VM, Hoffman JM, Caudle KE, Klein TE. Expanding evidence leads to new pharmacogenomics payer coverage. Genet Med. 2021;23(5):830–2.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the following for assistance in completion of the surveys: Emily Cicali from the University of Florida, the Pharmacogenomics Implementation Committee Colorado (PICColo) Evaluation Working Group at the University of Colorado, Whitney Mason and Joy Thomas from Intermountain Precision Genomics, April Schultz from Sanford Health Imagenetics, Cindy Prows from the University of Cincinnati, and Michelle Liu and Bart Roland from Vanderbilt University Medical Center.

The IGNITE Pharmacogenetics (PGx) Working Group includes both funded and affiliate members of the IGNITE Network (https://www.genome.gov/Funded-Programs-Projects/Implementing-Genomics-in-Practice-IGNITE). Application for affiliate membership is open to institutions with an interest in clinical genomic or pharmacogenomic testing. The IGNITE PGx Working Group was formed in 2015 with the goal of broadly engaging institutions that had implemented PGx in practice to share experiences with implementation and collectively disseminate implementation strategies.

Funding

Research reported in this publication was supported by grants from the National Institutes of Health (U01 HG007269, U01 HG010232, and U01 HG010245, NCATS UL1 TR001427 and by the NIH IGNITE Network (https://gmkb.org/ignite/). Additional funding was provided by K23 HL143161 and the Penn Center for Precision Medicine for S.T.; Indiana University Precision Health Initiative for E.R. and T.C.S.; Propeller Health, American Lung Association for K.B.; NIH for K.B., B.Q.D., and L.B.R.; K24HL133373 for N.A.L.; U01HG007775 for A.L.B.; U54TR001857 and the Pitt/UPMC Institute for Precision Medicine for P.E.E.; Agency for Healthcare Research and Quality (AHRQ) and Patient-Centered Outcomes Research Institute (PCORI) K12HS026379, NIH/NCATS KL2TR002492 for P.M.; and NIH/NCATS UL1TR001427 for L.H.C. The content is solely the responsibility of the authors and does not necessarily represent the official views of the sponsors.

Author information

Authors and Affiliations

  1. University of Florida Clinical and Translational Science Institute, Gainesville, FL, USA

    Ramzi G. Salloum & Larisa H. Cavallari

  2. University of Florida College of Medicine, Gainesville, FL, USA

    Ramzi G. Salloum

  3. University of Minnesota Medical School, Minneapolis, MN, USA

    Jeffrey R. Bishop & Pawel Mroz

  4. University of Minnesota College of Pharmacy, Minneapolis, MN, USA

    Jeffrey R. Bishop

  5. University of Florida College of Pharmacy, Gainesville, FL, USA

    Amanda L. Elchynski, Kristin Wiisanen & Larisa H. Cavallari

  6. MedStar Health, Georgetown University Medical Center, Washington, DC, USA

    D. Max Smith & Sandra M. Swain

  7. Indiana University School of Medicine, Indianapolis, IN, USA

    Elizabeth Rowe & Todd C. Skaar

  8. Nemours Children’s Health, Jacksonville, FL, USA

    Kathryn V. Blake

  9. University of Alabama Heersink School of Medicine, Birmingham, AL, USA

    Nita A. Limdi

  10. School of Medicine and Pharmacy, University of Colorado, Aurora, CO, USA

    Christina L. Aquilante

  11. Durham VA Healthcare System, Durham, NC, USA

    Jill Bates

  12. University of Maryland School of Medicine, Baltimore, MD, USA

    Amber L. Beitelshees

  13. University of North Carolina Medical Center, Chapel Hill, NC, USA

    Amber Cipriani

  14. Nemours Children’s Health, Wilmington, DE, USA

    Benjamin Q. Duong

  15. University of Pittsburgh School of Pharmacy, Pittsburgh, PA, USA

    Philip E. Empey

  16. Intermountain Healthcare, Salt Lake City, UT, USA

    Christine M. Formea

  17. Moffitt Cancer Center, Tampa, FL, USA

    J. Kevin Hicks

  18. Corporal Michael J. Cresenz VA Medical Center, Philadelphia, PA, USA

    David Oslin

  19. University of Michigan College of Pharmacy, Ann Arbor, MI, USA

    Amy L. Pasternak & Kristen M. Ward

  20. North Dakota State University/Sanford Health, Fargo, ND, USA

    Natasha Petry

  21. Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA

    Laura B. Ramsey

  22. Fairview Pharmacy Services, Minneapolis, MN, USA

    Allyson Schlichte

  23. Vanderbilt University Medical Center, Nashville, TN, USA

    Sara L. Van Driest

  24. University of Pennsylvania Perelman School of Medicine, Smilow Center for Translational Research, 3400 Civic Center Boulevard, Bldg. 421 11th Floor, Room 143, Philadelphia, PA, 19104-5158, USA

    Sony Tuteja

Authors
  1. Ramzi G. Salloum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey R. Bishop
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amanda L. Elchynski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. D. Max Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elizabeth Rowe
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kathryn V. Blake
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nita A. Limdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Christina L. Aquilante
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jill Bates
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Amber L. Beitelshees
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Amber Cipriani
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Benjamin Q. Duong
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Philip E. Empey
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Christine M. Formea
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. J. Kevin Hicks
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Pawel Mroz
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. David Oslin
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Amy L. Pasternak
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Natasha Petry
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Laura B. Ramsey
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Allyson Schlichte
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Sandra M. Swain
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Kristen M. Ward
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Kristin Wiisanen
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Todd C. Skaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Sara L. Van Driest
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Larisa H. Cavallari
    View author publications

    You can also search for this author in PubMed Google Scholar

  28. Sony Tuteja
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RGS conceptualized and designed the study, performed the data analysis, interpreted the data, and drafted the manuscript. JRB performed the research, provided input on the design and analysis of the study, and made substantial contributions to the manuscript drafts. AE performed the research, interpreted the results, and made substantive edits to the manuscript. DMS performed the research, interpreted the results, and made substantive edits to the manuscript. ER performed the research, interpreted the results, and made substantive edits to the manuscript. KVB performed the research, interpreted the results, and made substantive edits to the manuscript. NAL performed the research, interpreted the results, and made substantive edits to the manuscript. CLA performed the research, interpreted the results, and made substantive edits to the manuscript. JB performed the research, interpreted the results, and made substantive edits to the manuscript. ALB performed the research, interpreted the results, and made substantive edits to the manuscript. AC performed the research, interpreted the results, and made substantive edits to the manuscript. BQD performed the research, interpreted the results, and made substantive edits to the manuscript. PEE performed the research, interpreted the results, and made substantive edits to the manuscript. CMF performed the research, interpreted the results, and made substantive edits to the manuscript. JKH performed the research, interpreted the results, and made substantive edits to the manuscript. PM performed the research, interpreted the results, and made substantive edits to the manuscript. DO performed the research, interpreted the results, and made substantive edits to the manuscript. ALP performed the research, interpreted the results, and made substantive edits to the manuscript. NP performed the research, interpreted the results, and made substantive edits to the manuscript. LBR performed the research, interpreted the results, and made substantive edits to the manuscript. AS performed the research, interpreted the results, and made substantive edits to the manuscript. SMS performed the research, interpreted the results, and made substantive edits to the manuscript. KMW performed the research, interpreted the results, and made substantive edits to the manuscript. KW performed the research, interpreted the results, and made substantive edits to the manuscript. TCS performed the research, interpreted the results, and made substantive edits to the manuscript. SLV performed the research, interpreted the results, and made substantive edits to the manuscript. LHC performed the research, provided input on the design and analysis of the study, and made substantial contributions to the manuscript drafts. ST conceptualized, designed, conducted the study, interpreted the data, and drafted the manuscript. All authors made substantial contributions to interpretation of the results and drafting the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sony Tuteja.

Ethics declarations

Ethics approval and consent to participate

The survey protocol used in this study was approved by the University of Florida Institutional Review Board.

Consent for publication

Not applicable.

Competing interests

D.M.S. has institution-associated research funding from Kailos Genetics. P.E.E. performs consulting at Cipherome. S.M.S. has a consulting/advisory or non-promotional speaking role from Exact Sciences (Genomic Health), Genentech/Roche, Daiichi Sankyo, Athenex, Natura, and Silverback Therapeutics; IDMC from AstraZeneca; support for third party writing assistance from Genentech/Roche; and institution-associated research funding from Genentech and Kailos Genetics. L.B.R. receives research funding from BTG, Intl. The other authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Electronic survey.

Additional file 2.

STROBE statement.

Additional file 3.

Characteristics of sites that had implemented  or were planning to implement pharmacogenetic testing to guide antidepressant therapy.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Salloum, R.G., Bishop, J.R., Elchynski, A.L. et al. Best–worst scaling methodology to evaluate constructs of the Consolidated Framework for Implementation Research: application to the implementation of pharmacogenetic testing for antidepressant therapy. Implement Sci Commun 3, 52 (2022). https://doi.org/10.1186/s43058-022-00300-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-022-00300-7

Keywords

Implementation Science Communications

ISSN: 2662-2211