Skip to main content

Evaluating feedback from an implementation advisory board to assess the rollout of a postpartum patient navigation program



Patient navigation is an individualized intervention to facilitate comprehensive care which has not yet been fully implemented in obstetric or postpartum care.


We aimed to develop and evaluate a mechanism to incorporate feedback regarding implementation of postpartum patient navigation for low-income birthing individuals at an urban academic medical center. This study analyzed the role of an Implementation Advisory Board (IAB) in supporting an ongoing randomized trial of postpartum navigation. Over the first 24 months of the trial, the IAB included 11 rotating obstetricians, one clinic resource coordinator, one administrative leader, two obstetric nurses, one primary care physician, one social worker, and one medical assistant. Members completed serial surveys regarding program implementation, effects on patient care, and areas for improvement. Quarterly IAB meetings offered opportunities for additional feedback. Survey responses and meeting notes were analyzed using the constant comparative method and further interpreted within the Exploration, Preparation, Implementation, Sustainment (EPIS) Framework.


Members of the IAB returned 37 surveys and participated in five meetings over 24 months. Survey analysis revealed four themes among the inner context: reduced clinician burden, connection of care teams, communication strategies, and clinic workflow. Bridging factors included improved patient access to care, improved follow-up, and adding social context to care. Innovation factors included availability of navigators, importance of consistent communication, and adaptation over time. Meeting notes highlighted the importance of bidirectional feedback regarding implementation, and members expressed positive opinions regarding navigators’ effects on patient care, integration into clinic workflow, and responsiveness to feedback. IAB members initially suggested changes to improve implementation; later survey responses demonstrated successful program adaptations.


Members of an implementation advisory board provided key insights into the implementation of postpartum patient navigation that may be useful to promote dissemination of navigation and establish avenues for the engagement of implementing partners in other innovations.

Trial registration, NCT03922334. Registered April 19, 2019. The results here do not present the results of the primary trial, which is ongoing.

Peer Review reports


As disparities in postpartum care persist despite standardized recommendations to optimize care, [1,2,3,4,5] postpartum patient navigation could serve to promote the long-term health of birthing individuals. Postpartum patient navigation, an individualized intervention which identifies and addresses barriers to care, is emerging in the field of obstetrics as a method to facilitate healthcare access and promote comprehensive postpartum care [6]. Navigators’ roles include assisting patients with care coordination, addressing social determinants of health, and fostering patient activation to promote high-quality care [7]. Despite this promise, patient navigation has not yet been integrated into usual postpartum care [6, 8].

As with all emerging interventions or the application of established interventions to novel contexts, ascertaining the determinants of implementation is necessary to facilitate programmatic success. Such identification would provide valuable information for creators of new postpartum navigation programs to optimize implementation if the intervention is shown to improve postpartum health. In the implementation of any novel intervention, various collaborators, including those who work closely with implementers or individuals who receive the intervention, may have particularly important perspectives on determinants of implementation. Given that engagement of clinical and administrative partners is integral to implementation science overall, [9] identifying and incorporating these partners within postpartum navigation programs is an important component of program development and evaluation. Thus, we aimed to examine the role of implementing partners’ feedback in the context of an ongoing randomized controlled trial of postpartum patient navigation in an urban academic medical center.


This study assessed the value of a clinical Implementation Advisory Board (IAB) in supporting an ongoing randomized controlled trial of postpartum patient navigation (NCT03922334) [10]. The trial, “Navigating New Motherhood 2 (NNM2)” aims to enroll 400 pregnant or postpartum individuals enrolled in Medicaid for obstetric care, and to randomize them to receive either 1 year of individualized postpartum patient navigation or usual care. This trial is being conducted at a large, urban academic medical center that cares for diverse birthing individuals and its primary aim is to determine whether participation in patient navigation program improves postpartum health outcomes. This analysis aims to present the findings from IAB surveys and meeting notes to identify determinants of program implementation, as perceived by the IAB, as well as assess their perspectives on program effects and integration into the clinic setting. In doing so, we aim to elucidate the potential value of an IAB for optimizing the ongoing implementation of novel innovations overall.

As part of study protocol, investigators identified implementing partners and assembled an IAB to create a formal process for clinical partners and research team members to exchange bidirectional feedback regarding the implementation and effects of the ongoing innovation. This process allowed for investigators to provide updates on research aims, while implementing partners could provide information on the real-time effects of the program, as well as make suggestions to promote integration into the clinic setting in which the program took place. Over the first two years of the trial, the IAB included 11 rotating obstetricians, one clinic resource coordinator, one administrative leader, two obstetric nurses, one primary care physician (PCP), one social worker, and one medical assistant. Research staff formally recruited IAB members, all of whom worked within the clinic site of program implementation, via direct face-to-face or electronic discussions about the program. IAB members participated voluntarily and were not compensated. Through serial surveys sent to all members every two months, the IAB provided real-time feedback to investigators and program staff regarding implementation of the intervention. While surveys were sent to all members, an initial screening question allowed respondents to terminate the survey if they had not interacted with the patient navigation program in the 2 months prior. Surveys included respondents’ identities and contained seven open-ended questions that generated responses allowing for in-depth qualitative interpretation. Surveys queried respondents’ opinions regarding program integration, modes of communication with navigators, areas for programmatic improvement, and effects of the program on clinical care (Table 1). The research team developed these questions a priori in alignment with study goals to assess implementation. This analysis uses the IAB responses to assess program implementation, and no data from the RCT are presented herein.

Table 1 Serial survey questions (q = 2 months) sent to IAB regarding program implementation

In addition to the surveys, hour-long quarterly meetings provided opportunities for additional feedback. Meetings included a presentation by the research team to IAB members about the progress of the study as well as structured discussion in which IAB members provided feedback to the research team. Additionally, the research team offered tips and education to the IAB about the patient navigation program, provided updates on programmatic evolution in response to prior feedback, and shared initial research findings from early-phase work [11, 12]. Meetings were initially in-person but converted to virtual at the start of the COVID-19 pandemic. One research team member took comprehensive notes to document all discussions at meetings.

Survey responses and IAB meeting notes were analyzed by the primary author using the constant comparative method and grounded theory to inductively identify initial themes. Given the small quantity of data, no analytic software was used, and analysis was conducted on word processing programs within a secure server. In an initial round of coding, the primary author coded survey response data and meeting notes and consolidated these codes into initial themes to identify determinants of program implementation as perceived by the implementation advisory board. This thematic analysis was discussed, refined, and finalized with the entire research team through an iterative process. The primary author then conducted a second round of analysis in which these themes were mapped onto three of four constructs (inner context, bridging factors, innovation factors) within the Exploration, Preparation, Implementation, Sustainment (EPIS) Framework, a common framework for the qualitative assessment of implementation strategies [13]. After mapping themes onto EPIS constructs, the analysis was reviewed and finalized with the entire research team to maximize reliability.

The EPIS Framework was selected primarily for its articulation of the relationships between the primary setting of the innovation and the larger context of implementation, suitable for a broad intervention such as patient navigation [14]. Multiple published studies have used the EPIS framework to primarily assess implementation, with less focus on other phases of the implementation process (i.e. exploration, preparation, and sustainment), supporting this framework as appropriate to structure our analysis of program implementation [13].


During the period of data collection, 18 individuals served on the IAB. Given the presence of a screening question which allowed individuals to terminate the survey if they had not interacted with the patient navigation program, 11 individuals provided qualitative survey data. We received 39 qualitative survey responses from these 11 unique individuals including obstetricians, a research coordinator, a clinic administration, nurses, primary care physicians, and a social worker. Among these 11 individuals, the average survey response rate was 68%. Given the survey contained only open-ended questions, there was variation in the level of detail provided in responses, though nearly all provided responses contained substantive information regarding implementation suitable for qualitative analysis. To supplement surveys, we held five meetings over 24 months with additional members of the IAB.

Survey responses revealed positive themes across the inner setting construct, defined as characteristics of the organization in which the innovation takes place. One obstetrician noted, “the only aspect I would change…is to have every one of our patients have a navigator.” Specific themes included facilitating a reduced burden of care for providers, allowing one obstetrician to “focus…more on…medical care than on…care coordination.” In addition, increased connection between obstetric, specialty, and primary care teams, successful communication strategies, and successes in clinic workflow characterized the inner setting (Table 2).

Table 2 Implementation Advisory Board members’ survey feedback about navigation implementation analyzed across EPISa constructs

Bridging factors, or themes related to the connection of the inner organization to broader systems, included improved patient access to care. The clinic social worker noted navigators assisted her “in identifying barriers to care” such as transportation and insurance issues. One obstetrician noted this resulted in patients being “less likely to cancel their appointments.” Likewise, improved follow-up in care among navigated patients and the incorporation of participants’ social needs into postpartum care added “so much to [one obstetrician’s] understanding of the patient’s complex social background” (Table 2).

Finally, positive innovation factors, or characteristics of the innovation itself, included a high degree of availability of navigators to providers, as one obstetrician stated “navigators are easily reached and have a presence in clinic.” Additional innovation factors included the importance of consistent communication between navigators and care teams and adaptation of the innovation over time, especially in areas for growth specifically identified by IAB feedback (Table 2). The clinic patient service representative noted navigators were “very much susceptible [i.e., accepting of] to feedback” and adapted “to the ever-changing policies [in clinic] impressively well.”

Analysis of IAB meeting notes demonstrated positive perceptions of program implementation, specifically regarding its smooth incorporation into clinic workflow and responsiveness to feedback. These meetings also provided members with the opportunity to expound on prior survey responses and provide concrete suggestions for program modifications to promote implementation. The group setting of IAB meetings allowed for brainstorming among multiple perspectives at once. In initial meetings, members suggested programmatic improvements regarding navigators’ communication strategies, specifically regarding navigators’ inconsistent use of the electronic medical record messaging system. In response to this feedback, navigators began integrating the electronic medical record into their daily tasks, contacting providers, scheduling appointments, and coordinating care through the EMR, and later data demonstrated this adaptation successfully ameliorated these issues, ultimately promoting implementation as communication “significantly improved” and had “only grown to be better.”


In summary, we identified that incorporation of an IAB into the implementation of a postpartum navigation program provided important insights and feedback regarding program rollout and evolution. Survey responses and meeting notes demonstrated positive assets of the innovation and its implementation, including attention to social components of care, navigator availability, and adaptable communication styles, which all were perceived to improve clinical care. A variety of implementing partners, including clinicians, administrators, and social workers, elucidated facilitators to implementation, as well as provided suggestions for improvement, allowing for program responsiveness and evolution. As navigators themselves worked closely with these partners in daily activities, this bidirectional feedback fostered adaptation of program activities.

Our results also support the overall utility of an IAB for optimizing the implementation of novel innovations. The IAB served as a real-time mechanism for adjustments to program implementation which incorporated a variety of perspectives. The bidirectional nature of feedback, in which research staff provided updates to implementing partners that were relevant to the program’s progress and adaptations while partners themselves made suggestions for programmatic improvement, was an effective way to tailor program rollout without interfering with ongoing program operations, as evidenced by improvements made directly in response to IAB comments (e.g. integration of the EMR into navigator task flow). These results support the use of a IAB for optimizing implementation of ongoing innovations.

This analysis has a few limitations to note. The lower survey response rate may introduce bias, as IAB members with more frequent interactions with navigators and strong opinions regarding implementation may have been more likely to respond. These data, however, would represent the pitfalls and assets of implementation as perceived by individuals most regularly interacting with the innovation, while IAB meetings provided an opportunity for all participants to offer opinions regarding implementation. Additionally, our method of qualitative analysis identified overarching themes in survey responses and was not conducive to identifying granular feedback and suggestions from the IAB, such as logistical modifications or identifying specific cases of missed opportunities. These suggestions, however, represent site-specific data which are less integral to our larger findings regarding the program overall and the role of the IAB in promoting its implementation. Lastly, the setting of implementation, an ongoing research study at an urban academic medical center, may impact the generalizability of our findings. Other settings, including smaller health systems not conducting research on the innovation, may not have the resources to incorporate an IAB in program rollout. Our findings, however, support the use of an IAB in implementation of postpartum navigation or other similar interventions, when feasible, as a useful mechanism for continuous feedback to streamline implementation of a novel program.

Future work may use the EPIS framework to explore the role of an IAB in program sustainment past the initial implementation phase, particularly as interventions move from the research context to long-term implementation. Furthermore, future incorporation of implementing partners into patient navigation programs or other maternal health interventions should include the perspectives of birthing people themselves and members of the communities in which they live. As we began the IAB at the beginning of our trial, no patients were available to participate, yet future work should incorporate these important perspectives.


Identifying feedback regarding the implementation of our postpartum navigation program, as perceived by an IAB, may elucidate determinants of implementation that can promote its future dissemination, [15] while the emphasis on program adaptation this study found supports the use of IABs in implementation for similar programs. Findings underscore the importance of including implementing partners in the rollout of novel interventions, both when the interventions are conducted in the research context and clinical practice.

Availability of data and materials

Data are not publicly available given the ongoing nature of the trial and the sensitive nature of the information. Data however are available from the authors upon reasonable request and with permission of the Northwestern University IRB, upon conclusion of the ongoing trial.


  1. ACOG Committee Opinion No. 736: Optimizing Postpartum Care. Obstet Gynecol. 2018;131(5):e140–50.

    Article  Google Scholar 

  2. Badreldin N, Grobman WA, Yee LM. Racial Disparities in Postpartum Pain Management. Obstet Gynecol. 2019;134(6):1147–53.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bullington BW, Arora KS. Fulfillment of Desired Postpartum Permanent Contraception: a Health Disparities Issue. Reprod Sci. 2022;29(9):2620–4.

    Article  CAS  PubMed  Google Scholar 

  4. Jones EJ, Hernandez TL, Edmonds JK, Ferranti EP. Continued Disparities in Postpartum Follow-Up and Screening Among Women With Gestational Diabetes and Hypertensive Disorders of Pregnancy: A Systematic Review. J Perinat Neonatal Nurs. 2019;33(2):136–48.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Kozhimannil KB, Interrante JD, Basile Ibrahim B, Chastain P, Millette MJ, Daw J, et al. Racial/Ethnic Disparities in Postpartum Health Insurance Coverage Among Rural and Urban U.S. Residents. J Womens Health (Larchmt). 2022;31(10):1397–402.

    PubMed  PubMed Central  Google Scholar 

  6. McKenney KM, Martinez NG, Yee LM. Patient navigation across the spectrum of women’s health care in the United States. Am J Obstet Gynecol. 2018;218(3):280–6.

    Article  PubMed  Google Scholar 

  7. Green HM, Diaz L, Carmona-Barrera V, Grobman WA, Yeh C, Williams B, et al. Mapping the postpartum experience through obstetric patient navigation for low-income individuals. J Womens Health (Larchmt). 2024.

  8. Hu F, Strohbach A, Martinez NG, Simon MA, Yee LM. Patient and Provider Perceptions of a Patient Navigation Program to Improve Postpartum Care Among Publicly Insured Women. Am J Perinatol. 2021;38(3):248–57.

    Article  PubMed  Google Scholar 

  9. Goodman MS, Sanders Thompson VL. The science of stakeholder engagement in research: classification, implementation, and evaluation. Transl Behav Med. 2017;7(3):486–91.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Yee LM. Navigating New Motherhood 2 (NNM2) National Insititutes of Health US National Library of Medicine; 2022 [Available from:

  11. Filicko A, Huennekens K, Davis K, Dolan BM, Williams BR, Feinglass J, et al. Primary Care Clinician Perspectives on Patient Navigation to Improve Postpartum Care for Patients with Low Income. Womens Health Rep (New Rochelle). 2022;3(1):1006–15.

    PubMed  Google Scholar 

  12. Ruderman RS, Dahl EC, Williams BR, Feinglass JM, Kominiarek MA, Grobman WA, et al. Obstetric Provider Perspectives on Postpartum Patient Navigation for Low-Income Patients. Health Educ Behav. 2023;50(2):260–7.

    Article  PubMed  Google Scholar 

  13. Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23.

    Article  PubMed  Google Scholar 

  15. Green HM, Carmona-Barrera V, Diaz L, Yeh C, Williams B, Davis K, et al. Implementation of postpartum navigation for low-income individuals at an urban academic medical center. PLoS ONE. 2023;18(2):e0282048.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references


The authors thank the members of the IAB for their participation.


This study was funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development R01 HD098178 (PI: Yee). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and Affiliations



BW, LD, VCB, KD, JF, MK, BD, WG, and LY conceptualized and implemented the implementation advisory board (IAB). BW disseminated and collected surveys, as well as oversaw IAB meetings and took notes on meeting contents. HG conducted survey and meeting note analysis and drafted the article. LD and VCB worked as patient navigators for the ongoing intervention. BD served as a participant on the IAB and advisor for the Navigating New Motherhood 2 study. LY is the principal investigator for the ongoing randomized controlled trial, Navigating New Motherhood 2; she secured funding for the trial and provided supervision and oversight for all elements of the study.

Corresponding author

Correspondence to Lynn M. Yee.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Northwestern University Institutional Review Board (STU#00209609) and all participants provided consent prior to participation.

Consent for publication

Not applicable.

Competing interests

The authors report no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Green, H.M., Williams, B., Diaz, L. et al. Evaluating feedback from an implementation advisory board to assess the rollout of a postpartum patient navigation program. Implement Sci Commun 5, 50 (2024).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: