Skip to main content
Download PDF
Download ePub

Implementation research priorities for addressing the maternal health crisis in the USA: results from a modified Delphi study among researchers

Implementation Science Communications volume 4, Article number: 83 (2023) Cite this article

Abstract

Background

Maternal health outcomes in the USA are far worse than in peer nations. Increasing implementation research in maternity care is critical to addressing quality gaps and unwarranted variations in care. Implementation research priorities have not yet been defined or well represented in the plans for maternal health research investments in the USA.

Methods

This descriptive study used a modified Delphi method to solicit and rank research priorities at the intersection of implementation science and maternal health through two sequential web-based surveys. A purposeful, yet broad sample of researchers with relevant subject matter knowledge was identified through searches of published articles and grant databases. The surveys addressed five implementation research areas in maternal health: (1) practices to prioritize for broader implementation, (2) practices to prioritize for de-implementation, (3) research questions about implementation determinants, (4) research questions about implementation strategies, and (5) research questions about methods/measures.

Results

Of 160 eligible researchers, 82 (51.2%) agreed to participate. Participants were predominantly female (90%) and White (75%). Sixty completed at least one of two surveys. The practices that participants prioritized for broader implementation were improved postpartum care, perinatal and postpartum mood disorder screening and management, and standardized management of hypertensive disorders of pregnancy. For de-implementation, practices believed to be most impactful if removed from or reduced in maternity care were cesarean delivery for low-risk patients and routine discontinuation of all psychiatric medications during pregnancy. The top methodological priorities of participants were improving the extent to which implementation science frameworks and measures address equity and developing approaches for involving patients in implementation research.

Conclusions

Through a web-based Delphi exercise, we identified implementation research priorities that researchers consider to have the greatest potential to improve the quality of maternity care in the USA. This study also demonstrates the feasibility of using modified Delphi approaches to engage researchers in setting implementation research priorities within a clinical area.

Peer Review reports

Background

Although the USA made tremendous gains in reducing maternal mortality during most of the twentieth century, this trend has reversed, and maternal mortality has steadily increased in recent decades [1]. Currently, the USA fares worse than most other high-income nations in maternal health outcomes [2, 3]. In 2019, there were 20.1 maternal deaths for every 100,000 live births [4] and five to ten times as many cases of severe maternal morbidity [5]. Maternal health is further marked by grave disparities in outcomes by race and geography, which persist even when controlling for factors such as education and insurance coverage [6].

Reviews of maternal morbidity and mortality cases find that 40 to 60% of these cases are potentially preventable [4, 7]. Although clinical guidelines and maternal safety bundles exist to standardize care for the most important contributors to morbidity and mortality [8], they are under-implemented in many maternity care settings in the USA. Inadequate implementation of guidelines and unwarranted variations in clinical practices are reflected in large differences in maternal outcomes between delivering hospitals [6, 9,10,11,12], such as fivefold differences in obstetric complication rates [11] and tenfold differences in cesarean delivery rates [10]. Quality improvement initiatives in some states have demonstrated that standardizing care for complications such as hemorrhage and hypertension can both improve outcomes and reduce racial and geographic disparities [13,14,15]. However, even in successful initiatives, roughly one-third of hospitals fail to make improvements [13, 16], and these initiatives rarely extend to outpatient and community settings, where improvements in care quality may be needed most.

Implementation science and research hold great potential to assist quality improvement efforts addressing the implementation gaps in maternity care. Implementation research seeks to contribute generalizable knowledge about the implementation of evidence-based practices into routine care [17, 18]. This evidence can inform the strategies used by QI initiatives whose goals are to achieve local improvements at the level of a healthcare facility, health system, or state [19]. Implementation research studies can identify contextual determinants that influence the underuse of evidence-based practices in maternity care [20, 21] and assess which implementation strategies are effective in specific contexts [22,23,24]. Initiatives to address the overuse of ineffective or potentially harmful practices can be aided by emerging evidence regarding the unique challenges involved with de-implementation [25, 26]. Although the potential benefits are clear, maternal health is lagging far behind other fields in the application of implementation science methods [27]. There is an urgent need for more investment in implementation research to address the maternal health crisis in the USA [28, 29].

One strategy for catalyzing research investments, and directing investments to areas that can generate the greatest impact, is establishing research priorities [30]. There are many approaches for establishing research priorities, ranging from unstructured expert panels to highly structured and replicable questionnaire-based methods [31,32,33]. A set of research priorities for improving maternal health in the USA was recently published by an expert panel convened by the National Institute of Child Health and Human Development (NICHD) [34]. While the priorities proposed by the NICHD panel included important epidemiologic and clinical effectiveness questions, they did not address implementation research [28, 34], leaving a dearth of guidance for funders and researchers. Knowledge gaps include which evidence-based practices are the most important to prioritize for implementation research, which ineffective practices should be the focus of de-implementation, and which implementation strategies are most promising for testing in maternity care settings. As the development of the methods (e.g., frameworks, measures, and study designs [35, 36]) for implementation science progresses, there is a further need to understand which methods are most appropriate to deploy or adapt for harmonized research in maternity care. To address these gaps, we undertook a structured exercise to establish implementation research priorities for improving maternal health in the USA.

Methods

Approach

We conducted a descriptive study, following research priority-setting best practices [31]. The study was organized by an interdisciplinary steering group of four maternal health researchers engaged in implementation research with backgrounds in obstetrics, maternal–fetal medicine, nursing, and public health. All steering group members have implementation science training and prior survey and/or qualitative research experience.

In the absence of priority-setting methods specific to implementation research, we considered existing methods reviewed in the health sciences literature [31,32,33]. We selected the Delphi technique for its ability to incorporate and synthesize input from a large and broad group of stakeholders. The Delphi technique is a consensus-building approach originally developed by the RAND Corporation [37] that involves two or more rounds of input from stakeholders [37, 38]. The first round is typically an open-ended idea-generating round in which participants submit their suggestions in response to a prompt [38]. During later rounds, participants are asked to rate the relative importance of the suggestions remaining from the prior round [38]. To increase the inclusiveness, rigor, and transparency of the Delphi process for prioritizing research questions, we incorporated several previously-published modifications: identifying participants with related scientific expertise through a literature search of published authors, soliciting research questions for specific areas of inquiry, defining multiple criteria for rating suggested research questions, and limiting the number of rounds to 2 [39]. This study was reviewed and determined to be exempt by the Institutional Review Board of the University of Pennsylvania School of Medicine (Protocol #844,389).

Identification and recruitment of participants

For this initial priority-setting exercise, we sought to include a broad sample of researchers with subject matter knowledge in both maternal health and implementation research. To identify eligible researchers, we used a multi-step approach (see Additional file 1 for detailed descriptions). We first searched the National Institutes of Health (NIH) RePORTER system for US-based grants that included maternal health and implementation research keywords in the abstract and grants funded under any of the dissemination and implementation research funding opportunity announcements with a maternal health keyword in the abstract. For all relevant grants, we extracted the name of the principal investigator. Second, we searched PubMed in February 2021 for US-based articles that included both a maternal health and implementation research keyword and extracted the names of first and senior authors. Third, we used a snowball sampling approach to increase the diversity of the sample by asking early participants to recommend colleagues with relevant expertise, particularly those from underrepresented backgrounds. All identified researchers were sent an email invitation to participate in an Implementation Science for Maternal Health National Working Group in February 2021. The invitation described the expectations for working group volunteers (i.e., completing two brief surveys over a 3-month period) and included a survey that collected demographic data and assessed their perceived level of engagement with implementation research and maternal health research.

Data collection

Two Delphi surveys were administered using the Qualtrics web-based survey platform in March and May of 2021. The first included open-ended questions that solicited research topics in five areas (Table 1): (1) evidence-based practices to prioritize for implementation, (2) practices not supported by evidence to be prioritized for de-implementation, (3) research questions regarding determinants of implementation in maternity care, (4) research questions regarding implementation strategies that should be studied in maternity care, and (5) research questions related to the development and/or adaptation of implementation science methods and measures for maternity care. All question prompts included explanations of implementation research concepts and an example response (see Additional file 2).

Table 1 Abbreviated open-ended prompts from Delphi survey 1
Full size table

The interdisciplinary steering group reviewed and consolidated the open-ended responses from the first survey into fixed-choice responses for the second survey. We omitted suggestions that were out of scope for the prompt (61.5% of exclusions) or only mentioned once (38.5% of exclusions). Submissions were most commonly judged out of scope when they were non-specific questions (e.g., “how do we get busy providers to use EBPs?,” “what are the training and capacity building needs?”). Of the 497 individual recommendations submitted across the five areas in survey #1, 340 (68%) were reflected in 87 consolidated items in survey #2 (Additional file 2). During consolidation, the team identified two distinct categories of questions regarding implementation strategies—effectiveness of discrete strategies and broader questions about selection, tailoring, and testing of strategies—and these were presented separately in survey #2.

Two rating approaches were used in survey #2. For clinical practices that were recommended for broader implementation/de-implementation, respondents selected the three practices that they expected to have the greatest impact on maternal health if more widely implemented (among 20 practices) or de-implemented (among 17 practices). For each of the three selected practices, respondents rated as “high,” “medium,” or “low” the feasibility of wide implementation/de-implementation in the USA, the likelihood that this would improve outcomes, and the likelihood that this would reduce disparities. For the research questions regarding determinants of implementation (12 options), implementation strategies to test for effectiveness (14 options), broader research questions regarding how/when to use implementation strategies in maternal health implementation (11 options), and methods/measures (14 options), respondents selected their preferred five from each group and ranked each set of selections in the order of their perceived importance for advancing implementation research in maternal health.

Analysis

Descriptive statistics, including frequencies and percentages, were calculated for participant characteristics and the selection of practices and research question items in each section of survey #2 using Stata 15. For the clinical practices that respondents selected for implementation/de-implementation, the average ratings for each of the three criteria were calculated. Bubble charts were developed to visually display respondents’ relative ratings of each practice according to the multiple criteria. For the research topics and questions selected by participants, the average relative ranking of the item by those who selected it (from 1 to 5) was calculated. The prioritization results were initially shared with the Implementation Science for Maternal Health Interest Group through a virtual adjunct meeting of the 14th Annual Conference on the Science of Dissemination and Implementation in Health.

Results

Of 160 eligible individuals, 82 (51.2%) agreed to participate in the Implementation Science for Maternal Health National Working Group (Fig. 1). Fifty-seven (69.5%) completed survey 1, which elicited open-ended responses regarding priorities. Forty-five survey 1 respondents (78.9%) and three additional working group volunteers completed survey 2, which asked participants to select and rank top choices among the consolidated responses from survey 1.

Fig. 1
figure 1

Participant flow chart

Full size image

Characteristics of working group participants are detailed in Table 2. Approximately half of the participants were clinical providers, and the remaining half held other roles. Participants varied widely in the types of advanced degrees obtained and well-represented both mid- to senior and early-stage career investigators. Over 90% of participants identified as female. Nearly 75% of participants identified as White, 13% as Asian, and 7% as Black. There were no significant differences in demographic data among those who completed either survey as compared to the composition of the working group as a whole (data not shown).

Table 2 Characteristics of the implementation science for maternal health national working group*
Full size table

Table 3 has the practices most recommended for both implementation and de-implementation in maternal health. For implementation, participants focused on (1) improved postpartum care, including home visiting programs and short interval visits; (2) perinatal and postpartum mood disorder screening and management, including collaborative care models; and (3) standardized, evidence-based practices for the management of hypertensive disorders of pregnancy. For de-implementation, practices believed to be most impactful if removed from or reduced in maternity care were (1) cesarean delivery for low-risk patients, (2) routine discontinuation of all psychiatric medications during pregnancy, and (3) routine separation of infants and parents at birth.

Table 3 Practices most recommended for implementation and de-implementation in survey #1, as consolidated by the investigative team
Full size table

Participants were also asked to rate their top 3 selected practices for implementation and de-implementation on the feasibility of implementation and de-implementation, likelihood of improved outcomes with implementation and de-implementation, and likely impact on reducing disparities, using a scale of 1–3 (1 = low; 3 = high). Figure 2a and b visually depict these ratings in addition to demonstrating how many participants selected the practice in their top 3 (see Additional file 3, for numeric ratings of practices recommended for implementation (Additional file 3: Table S1) and de-implementation (Additional file 3: Table S2)). While practices were generally rated highly in all domains, this depiction allows us to identify practices not only believed to be of value by many, but also believed to be feasible, with a goal of reducing disparities. For implementation, standardized, evidence-based practices for the management of hypertensive disorders of pregnancy and standardized, evidence-based practices for the management of obstetric hemorrhage come to the forefront. For de-implementation, the focus remains on routine separation of infants and parents at birth and de-implementing routine discontinuation of psychiatric medications during pregnancy.

Fig. 2
figure 2

a Practices most recommended for implementation, as represented in a bubble chart. Numbered practices in Table 3, section A, correspond to the bubble labels. X-axis = feasibility of routinely implementing this practice in US maternity care settings (scale of 1–3). Y-axis = likelihood that wide implementation of this practice will improve outcomes (scale of 1–3). Bubble size indicates how many survey #2 participants selected the practice in their top 3 practices for implementation. Bubble color indicates the quartile of the likelihood that wide implementation of this practice will reduce disparities in maternity outcomes rating. b Practices most recommended for de-implementation, as represented in a bubble chart. Numbered practices in Table 3, section B, correspond to the bubble labels. X-axis = feasibility of de-implementing this practice in US maternity care settings (scale of 1–3). Y-axis = likelihood that de-implementation of this practice will improve outcomes (scale of 1–3). Bubble size indicates how many survey #2 participants selected the practice in their top 3 practices for de-implementation. Bubble color indicates the quartile of the likelihood that wide implementation of this practice will reduce disparities in maternity outcomes rating

Full size image

When eliciting contextual determinants likely to exert the greatest influence on implementation in maternity care, participants focused on reimbursement policies, as well as implicit bias and racism (Table 4). In regard to implementation strategies most important to test for effectiveness in maternity care, participants selected building a coalition of partners and altering incentives to promote adoption. Three-quarters of participants selected the most important research question related to strategies to advance the field of implementation research in maternal health to be, “How can implementation strategies be selected and/or adopted specifically to promote equity?” Another commonly selected research question related to strategies was, “What implementation strategies lead to sustainability in improved implementation of evidence-based practices in maternity care?” Finally, of research goals related to methods and measures that would most help advance the field of implementation research in maternal health, participants most valued improving the extent to which implementation science frameworks and measures address social determinants of health and equity and developing approaches for involving patients in implementation and implementation research.

Table 4 Top ten research topics most selected to prioritize for future study in four categories
Full size table

Discussion

This work uses rigorous methods to establish priorities for research at the intersection of implementation science and maternal health. Specifically, this work identifies evidence-based practices most important to evaluate for implementation, as well as low-value interventions most critical to evaluate for de-implementation. Beyond evaluating specific evidence-based or low-value practices, priorities were also determined for research questions regarding determinants of implementation in maternity care, research questions regarding implementation strategies that should be studied, and research questions related to the development and/or adaptation of implementation science methods and measures for maternity care.

In 2019, NICHD convened 2 workshops to identify research gaps and priorities for maternal mortality and morbidity research in the USA [38]. Expert participants in maternal health developed consolidated lists of research gaps, challenges, and opportunities in this field. Yet, none of the findings directly addressed implementation research, a key ingredient to addressing maternal morbidity and mortality by bringing evidence-based practices to patients who need them [28]. The recent publication of NIH funding opportunities focused on implementation science within the Improving Pregnancy Outcomes Vision for Everyone (IMPROVE) Initiative [40, 41] indicates increasing recognition of the potential for implementation research to contribute to addressing the national maternal health crisis. Several recently-funded IMPROVE Initiative studies involve implementation components, including an RCT of an internet-based program for reducing perinatal depression and a sequential multiple assignment randomized trial (SMART) of a perinatal lifestyle intervention [42]. Although prior global efforts have addressed implementation research priorities for low-resource settings, this study is the first to our knowledge to specifically address priorities at the intersection of implementation science and maternal health in the USA [43,44,45]. Given the highly context-dependent nature of both maternal health intervention needs [46] and implementation challenges [47,48,49], implementation research priorities are not readily transferrable between the global setting and the USA.

Prioritizing evidence-based practices for implementation research and support is recognized as important given resource limitations [50]. In considering the clinical practices most selected for implementation focus in this exercise—improved postpartum care, perinatal mental healthcare, and standardized management of hypertensive disorders—work has only just begun. For example, improved postpartum blood pressure surveillance can occur with the implementation of innovative remote blood pressure monitoring programs [51]. In the field of perinatal mental health, implementation work is evaluating whether effective interventions can be delivered by non-specialists or even digitally [52, 53]. Implementation of nurse-driven and semiautonomous treatment algorithms for peripartum hypertension has shown promise at providing appropriately timed treatment [54, 55]. Yet, these clinical examples represent just the tip of the iceberg of implementation; larger-scale implementation studies are needed to determine how best to incorporate these practices into the diverse maternity populations and practice models that exist in the USA, including community settings where evidence-based practices are needed most.

In regard to the practices most selected for de-implementation, unnecessary cesarean delivery was the highest priority. Cesarean is considered one of the underlying causes of both maternal mortality and morbidity, and a consensus safety bundle to reduce cesarean has been the subject of several state-based implementation efforts [16, 56]. However, further researchn is needed to address de-implementation barriers, particularly related to unit culture [20]. Additionally, the perceived benefits of some obstetrics interventions are debated, including some interventions recommended by panelists for de-implementation. De-implementation decisions should take into consideration the local clinical context (e.g., staffing, resources, patient population), evidence regarding effectiveness in different contexts, and feasibility and potential unintended consequences of de-implementation.

This work also establishes the feasibility of using adapted Delphi approaches to solicit input on implementation research priorities from a broad range of researchers within a subfield. While several papers have suggested important research directions within implementation science, these have tended to focus more broadly on conceptual areas such as implementation strategies [57], sustainability [58], and mechanisms [59]. Methods that have been used or proposed for broad implementation research agenda setting include concept mapping [58], literature reviews [57], and expert panel discussions [49, 59]. A strength of the web-based Delphi approach is the ability to solicit input from a broad group of researchers to maximize the diversity of perspectives. This increases the inclusiveness of the exercises [31], as well as the reliability, which has been shown to stabilize as participation approaches 50 individuals [60]. Additionally, Delphi techniques collect and present suggestions for rating anonymously among participants to minimize bias that might result from interpersonal factors, such as deference to the most vocal or well-known participants in the group [33, 61].

Several of the Delphi modifications we incorporated were originally developed by the Child Health and Nutrition Research Initiative (CHNRI) for priority-setting exercises convened by the World Health Organization [39] and have been used repeatedly for establishing maternal and child health research priorities in low-resource contexts [32]. The survey prompts used by CHNRI [39] required adaptation to align with a focus on implementation research. The five areas in this exercise—evidence-based interventions, overused/ineffective practices, determinants of implementation, implementation strategies, and research methods—reflect major areas of inquiry in the developing field of implementation science [62]. A particular innovation of the CHNRI approach to increase rigor and transparency is defining rating criteria to make explicit the values that are being applied to rank different topics, which may otherwise be implicit and variable for different participants [31]. To lessen the survey burden and minimize between-round attrition, which can be high in Delphi exercises [33, 63], we reduced the number of items to be scored to only the participant’s top ten selections in the interventions area.

Strategies that seek input from a broad sample of researchers do present some challenges. While electronic Delphi exercises facilitate the inclusion of more participants, the format prevents discussions which can lead to rescuing of omitted suggestions, refinement of ideas, and group consensus during in-person Delphi exercises [33]. Additionally, implementation science is a relatively new field within maternal health, and the scope of implementation research questions proved difficult to conceptualize for some participants, as demonstrated by submissions that were judged to be out of scope or that closely mirrored the examples provided in question prompts. Furthermore, the inclusion of examples in question prompts may have limited the scope of responses for all participants. Similar difficulties have been observed in other research priority-setting exercises that engage broad participants [64, 65], and these difficulties were addressed by the steering committee when consolidating suggestions for the second round survey.

There was limited diversity among researchers who participated in this exercise, although the demographic characteristics of participants are likely consistent with the profile of researchers engaged in this implementation research for maternal health, highlighting the importance of efforts to increase researcher diversity. In addition, no data were collected on those who declined participation in the working group to examine if characteristics differed as compared to those who accepted participation, nor were we able to compare priority selections by participant characteristics due to the small sample size. Another major limitation of this work is the lack of inclusion of patient and community-based support individuals, such as doulas. Engaging patients in Delphi exercises to prioritize research questions may be most feasible when focused on specific clinical conditions for which patients with lived experience can be identified. Meaningful patient and community engagement may also require opportunities for discussion to clarify and resolve differences in perspective between patients and researchers [66].

Conclusions

Increasing implementation research in maternal health has great potential to improve the quality of care and reduce poor outcomes in the USA. Research priority-setting exercises can help to generate a catalog of topics, on which there is consensus among researchers, for the field to focus on in the coming years. Such a list may help researchers direct their energies, as well as aid funders in selecting research investments. This study demonstrates the feasibility of using adapted Delphi approaches to engage researchers in setting implementation research priorities within a clinical area. Approaches for incorporating patient and community perspectives in the development of implementation research questions are also needed.

Availability of data and materials

De-identified datasets used for this study are available from the corresponding author upon reasonable request.

Abbreviations

CHNRI:

Child Health and Nutrition Research Initiative

IMPROVE:

Improving Pregnancy Outcomes Vision for Everyone

NICHD:

National Institute of Child Health and Human Development

NIH:

National Institutes of Health

RePORTER:

Research Portfolio Online Reporting Tools: Expenditures and Results

References

  1. Douthard RA, Martin IK, Chapple-McGruder T, Langer A, Chang SUS. Maternal mortality within a global context: historical trends, current state, and future directions. J Women’s Health. 2021;30(2):168–77. https://doi.org/10.1089/jwh.2020.8863.

    Article  Google Scholar 

  2. MacDorman MF, Declercq E, Cabral H, Morton C. Recent increases in the U.S. maternal mortality rate: disentangling trends from measurement issues. Obstetr Gynecol. 2016;128(3):447–55. https://doi.org/10.1097/AOG.0000000000001556.

    Article  Google Scholar 

  3. Tikkanen R, Gunja M, FitzGerald M, Zephyrin L. Maternal mortality and maternity care in the United States compared to 10 other developed countries. Commonwealth Fund 2020. https://doi.org/10.26099/411v-9255.

  4. Petersen EE, Davis NL, Goodman D, et al. Vital signs: pregnancy-related deaths, United States, 2011–2015, and strategies for prevention, 13 states, 2013–2017. MMWR Morb Mortal Wkly Rep. 2019;68(18):423–9. https://doi.org/10.15585/mmwr.mm6818e1.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Phibbs CM, Kozhimannil KB, Leonard SA, et al. A comprehensive analysis of the costs of severe maternal morbidity. Womens Health Issues. Published online January 11, 2022:S1049–3867(21)00192–4. https://doi.org/10.1016/j.whi.2021.12.006.

  6. Howell EA, Egorova N, Balbierz A, Zeitlin J, Hebert PL. Black-white differences in severe maternal morbidity and site of care. Am J Obstet Gynecol. 2016;214(1):122.e1-7. https://doi.org/10.1016/j.ajog.2015.08.019.

    Article  PubMed  Google Scholar 

  7. Lawton B, MacDonald EJ, Brown SA, et al. Preventability of severe acute maternal morbidity. Am J Obstet Gynecol. 2014;210(6):557.e1-6. https://doi.org/10.1016/j.ajog.2013.12.032.

    Article  PubMed  Google Scholar 

  8. Ahn R, Gonzalez GP, Anderson B, Vladutiu CJ, Fowler ER, Manning L. Initiatives to reduce maternal mortality and severe maternal morbidity in the United States. Ann Intern Med. 2020;173(11):3–10. https://doi.org/10.7326/M19-3258.

    Article  Google Scholar 

  9. Main EK, Chang SC, Cheng YW, Rosenstein MG, Lagrew DC. Hospital-level variation in the frequency of cesarean delivery among nulliparous women who undergo labor induction. Obstetr Gynecol. 2020;136:1179. https://doi.org/10.1097/AOG.0000000000004139.10.1097/AOG.0000000000004139. (Latest Articles).

    Article  Google Scholar 

  10. Kozhimannil KB, Law MR, Virnig BA. Cesarean delivery rates vary tenfold among US hospitals; reducing variation may address quality and cost issues. Health Aff. 2013;32(3):527–35. https://doi.org/10.1377/hlthaff.2012.1030.

    Article  Google Scholar 

  11. Glance LG, Dick AW, Glantz JC, et al. Rates of major obstetrical complications vary almost fivefold among US hospitals. Health Aff. 2014;33(8):1330–6. https://doi.org/10.1377/hlthaff.2013.1359.

    Article  Google Scholar 

  12. Mujahid MS, Kan P, Leonard SA, et al. Birth hospital and racial and ethnic differences in severe maternal morbidity in the state of California. Am J Obstet Gynecol. 2021;224(2):219.e1-219.e15. https://doi.org/10.1016/j.ajog.2020.08.017.

    Article  PubMed  Google Scholar 

  13. Main EK, Chang SC, Dhurjati R, Cape V, Profit J, Gould JB. Reduction in racial disparities in severe maternal morbidity from hemorrhage in a large-scale quality improvement collaborative. Am J Obstet Gynecol. 2020;223(1):123.e1-123.e14. https://doi.org/10.1016/j.ajog.2020.01.026.

    Article  PubMed  Google Scholar 

  14. Main EK, Cape V, Abreo A, et al. Reduction of severe maternal morbidity from hemorrhage using a state perinatal quality collaborative. Am J Obstet Gynecol. 2017;216(3):298.e1-298.e11. https://doi.org/10.1016/j.ajog.2017.01.017.

    Article  PubMed  Google Scholar 

  15. Schneider PD, Sabol BA, Lee King PA, Caughey AB, Borders AEB. The hard work of improving outcomes for mothers and babies: obstetric and perinatal quality improvement initiatives make a difference at the hospital, state, and national levels. Clin Perinatol. 2017;44(3):511–28. https://doi.org/10.1016/j.clp.2017.05.007.

    Article  PubMed  Google Scholar 

  16. Callaghan-Koru JA, DiPietro B, Wahid I, et al. Reduction in cesarean delivery rates associated with a state quality collaborative in Maryland. Obstetr Gynecol. 2021;138:583. https://doi.org/10.1097/AOG.0000000000004540.10.1097/AOG.0000000000004540. (Published online September 9).

    Article  Google Scholar 

  17. Ovretveit J, Mittman B, Rubenstein L, Ganz DA. Using implementation tools to design and conduct quality improvement projects for faster and more effective improvement. Int J Health Care Qual Assur. 2017;30(8):755–68. https://doi.org/10.1108/IJHCQA-01-2017-0019.

    Article  PubMed  Google Scholar 

  18. Leeman J, Rohweder C, Lee M, et al. Aligning implementation science with improvement practice: a call to action. Implement Sci Commun. 2021;2(1):99. https://doi.org/10.1186/s43058-021-00201-1.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Callaghan-Koru J, Farzin A, Ridout E, Curran G. Integrating implementation science with quality improvement to improve perinatal outcomes. Clin Perinatol. 2023;50(2):343. https://doi.org/10.1016/j.clp.2023.01.002.

    Article  PubMed  Google Scholar 

  20. VanGompel ECW, Perez SL, Datta A, Carlock FR, Cape V, Main EK. Culture that facilitates change: a mixed methods study of hospitals engaged in reducing cesarean deliveries. Ann Fam Med. 2021;19(3):249–57. https://doi.org/10.1370/afm.2675.

    Article  Google Scholar 

  21. Bonawitz K, Wetmore M, Heisler M, et al. Champions in context: which attributes matter for change efforts in healthcare? Implement Sci. 2020;15(1):62. https://doi.org/10.1186/s13012-020-01024-9.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Koblinsky M, Moyer CA, Calvert C, et al. Quality maternity care for every woman, everywhere: a call to action. The Lancet. 2016;388(10057):2307–20. https://doi.org/10.1016/S0140-6736(16)31333-2.

    Article  Google Scholar 

  23. Spigel L, Plough A, Paterson V, et al. Implementation strategies within a complex environment: a qualitative study of a shared decision-making intervention during childbirth. Birth. 2022;49:440. https://doi.org/10.1111/birt.12611.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Moniz MH, Bonawitz K, Wetmore MK, et al. Implementing immediate postpartum contraception: a comparative case study at 11 hospitals. Implement Sci Commun. 2021;2(1):42. https://doi.org/10.1186/s43058-021-00136-7.

    Article  PubMed  PubMed Central  Google Scholar 

  25. McKay VR, Morshed AB, Brownson RC, Proctor EK, Prusaczyk B. Letting go: conceptualizing intervention de-implementation in public health and social service settings. Am J Community Psychol. 2018;62(1–2):189–202. https://doi.org/10.1002/ajcp.12258.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Kern-Goldberger AR, Hamm RF, Raghuraman N, Srinivas SK. Reducing alarm fatigue in maternal monitoring on labor and delivery: a commentary on deimplementation in obstetrics. Am J Perinatol 2022. https://doi.org/10.1055/a-1785-9175 Published online April 26.

  27. Breman RB, Hamm RF, Callaghan-Koru JA. Letter to the editor of implementation science in response to “Implementation Science in maternity care, a scoping review” by Dadich, piper, and coates (2021). Implement Sci. 2021;16(1):79. https://doi.org/10.1186/s13012-021-01129-9.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Callaghan-Koru JA, Moniz MH, Hamm RF. Prioritize implementation research to effectively address the maternal health crisis. Am J Obstet Gynecol. 2021;225(2):212–3. https://doi.org/10.1016/j.ajog.2021.02.005.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Hamm RF, Iriye BK, Srinivas SK. Implementation science is imperative to the optimization of obstetric care. Am J Perinatol. 2020;38:643. https://doi.org/10.1055/s-0040-1721728.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Fleurence RL, Torgerson DJ. Setting priorities for research. Health Policy. 2004;69(1):1–10. https://doi.org/10.1016/j.healthpol.2003.11.002.

    Article  PubMed  Google Scholar 

  31. Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010;8(1):36. https://doi.org/10.1186/1478-4505-8-36.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21st century. J Glob Health. 2015;6(1):010507. https://doi.org/10.7189/jogh.06.010507.

    Article  PubMed Central  Google Scholar 

  33. Bryant J, Sanson-Fisher R, Walsh J, Stewart J. Health research priority setting in selected high income countries: a narrative review of methods used and recommendations for future practice. Cost Effective Res Alloc. 2014;12(1):23. https://doi.org/10.1186/1478-7547-12-23.

    Article  Google Scholar 

  34. Chinn JJ, Eisenberg E, Dickerson SA, et al. Maternal mortality in the United States: research gaps, opportunities and priorities. Am J Obstetr Gynecol. 2020;223(4):486. https://doi.org/10.1016/j.ajog.2020.07.021.

    Article  Google Scholar 

  35. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53. https://doi.org/10.1186/s13012-015-0242-0.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Wensing M. Reflections on the measurement of implementation constructs. Implement Res Pract. 2021;2:26334895211020124. https://doi.org/10.1177/26334895211020125.

    Article  Google Scholar 

  37. Niederberger M, Spranger J. Delphi technique in health sciences: a map. Front Public Health. 2020;8:457. https://doi.org/10.3389/fpubh.2020.00457.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Keeney S, McKenna H, Hasson F. The Delphi technique in nursing and health research. John Wiley & Sons; 2011.

    Book  Google Scholar 

  39. Rudan I. Setting health research priorities using the CHNRI method: IV Key conceptual advances. J Glob Health. 2020;6(1):010501. https://doi.org/10.7189/jogh-06-010501.

    Article  Google Scholar 

  40. National Institutes of Health. NOT-OD-22–125: Notice of Special Interest (NOSI): IMPROVE Initiative: Implementation science to advance maternal health and maternal health equity. Published 2022. https://grants.nih.gov/grants/guide/notice-files/NOT-OD-22-125.html Accessed 14,Sept 2022.

  41. National Institutes of Health. RFA-HD-23–037: Maternal Health Research Centers of Excellence Implementation Science Hub/Resource Center (U24 Clinical Trial Optional). Published 2022. https://grants.nih.gov/grants/guide/rfa-files/RFA-HD-23-037.html Accessed 14 Sept 2022.

  42. National Institutes of Health. Research Portfolio Online Reporting Tools Expenditure and Results (RePORTER). Published 2020. https://reporter.nih.gov/.

  43. George A, Young M, Bang A, et al. Setting implementation research priorities to reduce preterm births and stillbirths at the community level. PLOS Med. 2011;8(1):e1000380. https://doi.org/10.1371/journal.pmed.1000380.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Sharma R, Buccioni M, Gaffey MF, et al. Setting an implementation research agenda for Canadian investments in global maternal, newborn, child and adolescent health: a research prioritization exercise. Can Med Assoc Open Access J. 2017;5(1):E82–9. https://doi.org/10.9778/cmajo.20160088.

    Article  Google Scholar 

  45. Lawn JE, Bahl R, Bergstrom S, et al. Setting research priorities to reduce almost one million deaths from birth asphyxia by 2015. PLOS Med. 2011;8(1):1000389. https://doi.org/10.1371/journal.pmed.1000389.

    Article  Google Scholar 

  46. Miller S, Abalos E, Chamillard M, et al. Beyond too little, too late and too much, too soon: a pathway towards evidence-based, respectful maternity care worldwide. The Lancet. 2016;388(10056):2176–92. https://doi.org/10.1016/S0140-6736(16)31472-6.

    Article  Google Scholar 

  47. Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. 2019;19(1):1–21. https://doi.org/10.1186/s12913-019-4015-3.

    Article  Google Scholar 

  48. Edwards N, Barker PM. The importance of context in implementation research. JAIDS J Acq Immune Defic Syndr. 2014;67:S157. https://doi.org/10.1097/QAI.0000000000000322.

    Article  Google Scholar 

  49. Eccles MP, Armstrong D, Baker R, et al. An implementation research agenda. Implementation Sci. 2009;4(1):1–7. https://doi.org/10.1186/1748-5908-4-18.

    Article  Google Scholar 

  50. Ervin JN, Dibble MR, Rentes VC, et al. Prioritizing evidence-based practices for acute respiratory distress syndrome using digital data: an iterative multi-stakeholder process. Implement Sci. 2022;17(1):82. https://doi.org/10.1186/s13012-022-01255-y.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Janssen MK, Demers S, Srinivas SK, et al. Implementation of a text-based postpartum blood pressure monitoring program at 3 different academic sites. Am J Obstet Gynecol MFM. 2021;3(6):100446. https://doi.org/10.1016/j.ajogmf.2021.100446.

    Article  PubMed  Google Scholar 

  52. Singla DR, Lawson A, Kohrt BA, et al. Implementation and effectiveness of nonspecialist-delivered interventions for perinatal mental health in high-income countries: a systematic review and meta-analysis. JAMA Psychiat. 2021;78(5):498–509. https://doi.org/10.1001/jamapsychiatry.2020.4556.

    Article  Google Scholar 

  53. Martin-Key NA, Spadaro B, Schei TS, Bahn S. Proof-of-concept support for the development and implementation of a digital assessment for perinatal mental health: mixed methods study. J Med Internet Res. 2021;23(6):e27132. https://doi.org/10.2196/27132.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Martin C, Pappas J, Johns K, Figueroa H, Balli K, Yao R. Semiautonomous treatment algorithm for the management of severe hypertension in pregnancy. Obstet Gynecol. 2021;137(2):211–7. https://doi.org/10.1097/AOG.0000000000004235.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Miller MJ, Butler P, Gilchriest J, Taylor A, Lutgendorf MA. Implementation of a standardized nurse initiated protocol to manage severe hypertension in pregnancy. J Matern Fetal Neonatal Med. 2020;33(6):1008–14. https://doi.org/10.1080/14767058.2018.1514381.

    Article  PubMed  Google Scholar 

  56. Rosenstein MG, Chang SC, Sakowski C, et al. Hospital Quality improvement interventions, statewide policy initiatives, and rates of cesarean delivery for nulliparous, term, singleton, vertex births in California. JAMA. 2021;325(16):1631–9. https://doi.org/10.1001/jama.2021.3816.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Powell BJ, Fernandez ME, Williams NJ, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Frontiers in Public Health. 2019;7.. https://www.frontiersin.org/article/https://doi.org/10.3389/fpubh.2019.00003 Accessed 18 March 2022.

  58. Proctor E, Luke D, Calhoun A, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88. https://doi.org/10.1186/s13012-015-0274-5.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Lewis CC, Powell BJ, Brewer SK, et al. Advancing mechanisms of implementation to accelerate sustainable evidence-based practice integration: protocol for generating a research agenda. BMJ Open. 2021;11(10):e053474. https://doi.org/10.1136/bmjopen-2021-053474.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Yoshida S, Rudan I, Cousens S. Setting health research priorities using the CHNRI method: VI. Quantitative properties of human collective opinion. J Glob Health. 2016;6(1):010503. https://doi.org/10.7189/jogh.06.010503.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Barrett D, Heale R. What are Delphi studies? Evid Based Nurs. 2020;23(3):68–9. https://doi.org/10.1136/ebnurs-2020-103303.

    Article  PubMed  Google Scholar 

  62. National Institutes of Health. PAR-22–105: dissemination and implementation research in health (R01 clinical trial optional). Published 2022. https://grants.nih.gov/grants/guide/pa-files/par-22-105.html Accessed 12 Sept 2022.

  63. Gargon E, Crew R, Burnside G, Williamson PR. Higher number of items associated with significantly lower response rates in COS Delphi surveys. J Clin Epidemiol. 2019;108:110–20. https://doi.org/10.1016/j.jclinepi.2018.12.010.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Jones R, Lamont T, Haines A. Setting priorities for research and development in the NHS: a case study on the interface between primary and secondary care. BMJ. 1995;311(7012):1076–80. https://doi.org/10.1136/bmj.311.7012.1076.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  65. Elwyn G, Crowe S, Fenton M, et al. Identifying and prioritizing uncertainties: patient and clinician engagement in the identification of research questions. J Eval Clin Pract. 2010;16(3):627–31. https://doi.org/10.1111/j.1365-2753.2009.01262.x.

    Article  PubMed  Google Scholar 

  66. Steffensen MB, Matzen CL, Wadmann S. Patient participation in priority setting: co-existing participant roles. Soc Sci Med. 2022;294:114713. https://doi.org/10.1016/j.socscimed.2022.114713.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The Implementation Science for Maternal Health National Working Group includes volunteer researchers who completed at least one priority-setting survey. The members of the working group, in alphabetical order, are as follows: Megan Allyse, Ian Bennet, Debra Bingham, Kacie Blackman, Rachel Breman, Sarah Brewer, Jennifer Callaghan-Koru, Jerry Cochran, Andreea Creanga, Shayna Cunningham, Ellen Daley, Carla DeSisto, Narges Farahi, Linda Franck, Sarah Goff, Stacey Griner, Sadia Haider, Rebecca Hamm, Anna Hansen, Samantha Harden, Kimberly Harper, Lisa Hofler, Sarah Horvath, Jeanette Ickovics, Jennifer Johnson, Heather Kaplan, Charlan Kroelinger, Elysia Larson, Huynh-Nhu (Mimi) Le, Henry Lee, Ann McAlearney, Danielle McCarthy, Lois McCloskey, Cristian Meghea, Emily Miller, Elizabeth (Libby) Mollard, Michelle Moniz, Tiffany Moore Simas, Eydie Moses-Kolko, Gina Novick, Abigail Palmer Molina, Divya Patel, Neena Qasba, Nandini Raghuraman, Amy Romano, Melissa Rosenstein, Sangini Sheth, Melissa Simon, Sharla Smith, Sindhu Srinivas, Carolyn Sufrin, Rachel Tabak, Erika Thompson, Cheryl Vamos, Daniel Walker, Jackie Wallace, Jin Xiao, Lynn Yee, Chloe Zera, Nikki Zite.

Funding

There are no funding sources for this study.

Author information

Authors and Affiliations

  1. Maternal and Child Health Research Center, Department of Obstetrics & Gynecology, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA

    Rebecca F. Hamm

  2. Leonard Davis Institute of Health Economics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA

    Rebecca F. Hamm

  3. Department of Obstetrics and Gynecology, University of Michigan, Ann Arbor, MI, USA

    Michelle H. Moniz

  4. Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MI, USA

    Michelle H. Moniz

  5. Department of Epidemiology and Biostatistics, Dornsife School of Public Health, Drexel University, Philadelphia, PA, USA

    Inaya Wahid

  6. Department of Partnerships, Professional Education and Practice, School of Nursing, University of Maryland, Baltimore, MD, USA

    Rachel Blankstein Breman

  7. Office of Community Health and Research, University of Arkansas for Medical Sciences, Springdale, AR, USA

    Jennifer A. Callaghan-Koru

  8. Department of Obstetrics & Gynecology, University of Arkansas for Medical Sciences, Little Rock, AR, USA

    Jennifer A. Callaghan-Koru

  9. Center for Implementation Research, University of Arkansas for Medical Sciences, Little Rock, AR, USA

    Jennifer A. Callaghan-Koru

Authors
  1. Rebecca F. Hamm
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michelle H. Moniz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Inaya Wahid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rachel Blankstein Breman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jennifer A. Callaghan-Koru
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

Implementation Science for Maternal Health National Working Group

  • Megan Allyse
  • , Ian Bennet
  • , Debra Bingham
  • , Kacie Blackman
  • , Rachel Blankstein Breman
  • , Sarah Brewer
  • , Jennifer A. Callaghan-Koru
  • , Jerry Cochran
  • , Andreea Creanga
  • , Shayna Cunningham
  • , Ellen Daley
  • , Carla DeSisto
  • , Narges Farahi
  • , Linda Franck
  • , Sarah Goff
  • , Stacey Griner
  • , Sadia Haider
  • , Rebecca F. Hamm
  • , Anna Hansen
  • , Samantha Harden
  • , Kimberly Harper
  • , Lisa Hofler
  • , Sarah Horvath
  • , Jeanette Ickovics
  • , Jennifer Johnson
  • , Heather Kaplan
  • , Charlan Kroelinger
  • , Elysia Larson
  • , Huynh-Nhu (Mimi) Le
  • , Henry Lee
  • , Ann McAlearney
  • , Danielle McCarthy
  • , Lois McCloskey
  • , Cristian Meghea
  • , Emily Miller
  • , Elizabeth (Libby) Mollard
  • , Michelle H. Moniz
  • , Tiffany Moore Simas
  • , Eydie Moses-Kolko
  • , Gina Novick
  • , Abigail Palmer Molina
  • , Divya Patel
  • , Neena Qasba
  • , Nandini Raghuraman
  • , Amy Romano
  • , Melissa Rosenstein
  • , Sangini Sheth
  • , Melissa Simon
  • , Sharla Smith
  • , Sindhu Srinivas
  • , Carolyn Sufrin
  • , Rachel Tabak
  • , Erika Thompson
  • , Cheryl Vamos
  • , Daniel Walker
  • , Jackie Wallace
  • , Jin Xiao
  • , Lynn Yee
  • , Chloe Zera
  •  & Nikki Zite

Contributions

RH: conceptualization, methodology, validation, formal analysis, visualization, and writing—original draft. MM: methodology, validation, and writing—review and editing. IW: methodology, investigation, project administration, data curation, and writing—review and editing. RB: methodology, validation, and writing—review and editing. JCK: conceptualization, methodology, investigation, writing—original draft, and supervision.

Corresponding author

Correspondence to Jennifer A. Callaghan-Koru.

Ethics declarations

Ethics approval and consent to participate

This study was reviewed and determined to be exempt by the Institutional Review Board of the University of Pennsylvania (protocol # 844389). All participants were informed of the purpose of the study and consented to participate.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Search strategy.

Additional file 2.

Survey prompts.

Additional file 3: Table S1.

Disaggregated ratings of practices most recommended for implementation in Survey #1, as consolidated by the investigative team. Table S2. Disaggregated ratings of practices most recommended for de-implementation in Survey #1, as consolidated by the investigative team.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hamm, R.F., Moniz, M.H., Wahid, I. et al. Implementation research priorities for addressing the maternal health crisis in the USA: results from a modified Delphi study among researchers. Implement Sci Commun 4, 83 (2023). https://doi.org/10.1186/s43058-023-00461-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-023-00461-z

Keywords

Implementation Science Communications

ISSN: 2662-2211