Skip to main content
Download PDF
Download ePub

Prenatal oral health guidelines: a theory- and practice-informed approach to survey development using a modified-Delphi technique and cognitive interviews

Implementation Science Communications volume 3, Article number: 126 (2022) Cite this article

Abstract

Background

Pregnancy presents an opportune time for oral health promotion and intervention; however, implementation of the prenatal oral health guidelines remains a challenge among prenatal and oral health providers. The purpose of this study was twofold: To employ a theory-based approach to identify high-priority Consolidated Framework for Implementation Research (CFIR) constructs with the greatest potential to impact prenatal oral health guideline implementation, and to operationalize and pre-test survey items based on the prioritized CFIR constructs. Identifying barriers and facilitators to guideline implementation will inform the development of targeted interventions that address gaps in adherence which can positively impact oral-systemic health.

Methods

The online survey development process employed three rounds of a modified-Delphi technique with prenatal (i.e., MD/DO, CNM) and oral health (i.e., DMD) Practice Advisory Board Members, cognitive interviews with prenatal and oral health providers, and deliberations among the research team and a Scientific Advisory Board (OBGYN, pediatric dentist, and researchers). High-impact CFIR constructs were identified and translated into survey items that were subsequently piloted and finalized.

Results

During three modified-Delphi rounds, a total of 39 CFIR constructs were evaluated with final input and deliberations with the Practice Advisory Board, Scientific Advisory Board, and the research team achieving consensus on 19 constructs. The instrument was pre-tested with four prenatal and two oral health providers. Overall, participants reported that the survey items were feasible to respond to, took an appropriate length of time to complete, and were well-organized. Participants identified specific areas of improvement to clarify CFIR items. The final survey instrument included 21 CFIR items across four domains, with five constructs included from the intervention characteristics domain, two from the process domain, two from the outer setting domain, and 12 from the inner setting domain.

Conclusions

Lessons learned from the survey development process include the importance of soliciting diverse scientific and practice-based input, distinguishing between importance/impact and direction of impact (barrier/facilitator), and the need for additional qualitative methods during interdisciplinary collaborations. Overall, this study illustrated an iterative approach to identifying high-priority CFIR constructs that may influence the implementation of the prenatal oral health guidelines into practice settings.

Peer Review reports

Background

Oral health during pregnancy is a significant concern given the prevalence of poor oral health conditions (e.g., gingivitis, periodontal disease), gaps in oral healthcare, and associations between oral-systemic health including adverse pregnancy and birth outcomes (e.g., preterm birth), early childhood caries, and other chronic conditions [1,2,3,4,5,6]. Pregnancy presents an opportune time for oral health promotion and intervention that can impact the health of mothers and children [7, 8]. Subsequently, a national consolidated set of prenatal oral health guidelines was co-endorsed by both the American College of Obstetricians and Gynecologists and the American Dental Association [9] (Table 1). Nonetheless, guideline implementation into practice remains a challenge. Research suggests an ongoing lack of guideline awareness, implementation, and interprofessional coordination among prenatal providers and oral healthcare providers [10, 11].

Table 1 Select oral health guideline behaviors for prenatal and oral health providers
Full size table

Effective guideline implementation to improve healthcare quality and outcomes is a national priority [12, 13]. Barriers to guideline implementation are multi-factorial and implementation science frameworks are useful for understanding the extent to which guidelines are adopted and implemented [14, 15]. The Consolidated Framework for Implementation Research (CFIR) is a “meta-theoretical framework” created from a synthesis of 19 pre-existing theories and can guide assessments of implementation barriers and facilitators [16, 17]. The CFIR identifies five domains, consisting of total of 39 individual constructs, that influence implementation: (1) intervention characteristics (e.g., quality of evidence; relative advantage), (2) characteristics of individuals (e.g., knowledge, self-efficacy), (3) inner setting (e.g., culture, networks), (4) outer setting (e.g., peer pressure, external policy), and (5) process (e.g., planning, executing) [16]. It is recommended that researchers assess the importance of each construct and provide justification for those identified, yet few researchers describe this process [16, 17].

In a systematic review on CFIR application, only three studies provided a rationale, only one study reported working with their population of interest and using all 39 constructs, and none of the studies used quantitative methods [17,18,19,20]. Few studies describe the development of quantitative CFIR measures [21,22,23]. However, such measures have been developed for use during or after the implementation, using a retrospective identification of barriers and facilitators [23,24,25]. There is a need to apply the CFIR prospectively, before or early in the implementation process, to identify salient factors to inform implementation strategies [17].

The purpose of this study was to employ a quantitative, theory-based yet practical approach to identifying CFIR constructs with the highest potential to impact implementation. A secondary purpose was to operationalize, draft, and pre-test survey items based on the prioritized CFIR constructs. Lessons learned when balancing survey development with practical consideration are highlighted. Future research will use identified constructs to assess their impact on prenatal oral health practice behaviors.

Methods

This theory-informed survey development process involved collaboration with Practice Advisory Board Members (PAB) during multiple rounds of a modified-Delphi technique, cognitive interviews with prenatal and oral health providers, and deliberations among the research team and a Scientific Advisory Board (SAB). The development process included identifying high-impact CFIR constructs, creating survey items from priority constructs, and piloting and finalizing survey items to be administered to providers in a future study phase. PAB members were recruited via convenience sample from professional prenatal and oral health networks (n=8; 2 MD; 2 DO; 2 CNM; 2 DMD/DDS). The SAB included (1) a board-certified obstetrician and maternal-fetal medicine specialist focused on oral disease and pregnancy outcomes, (2) a pediatric dentist specializing in prenatal and early childhood oral health, and (3) a researcher specializing in implementation science and the CFIR. The research team included three public health researchers proficient in oral health, maternal and child health, and survey design/psychometrics. The team adhered to the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) [26]. A graphic illustrating the role of the stakeholder groups is presented in Fig. 1.

Fig. 1
figure 1

Stakeholder groups involved in the study

Full size image

Modified-Delphi technique

The research team developed an online questionnaire based on all CFIR constructs (n=39), with input from the SAB. A modified-Delphi process was used with three rounds of surveys administered to the PAB [27,28,29,30]. For each round, PAB members individually ranked CFIR constructs using Likert scales to prioritize factors perceived to be most salient (of high priority). Each round utilized a different approach to determine consensus, a process informed by systematic and practical considerations as the rounds progressed and challenges were encountered. The modified-Delphi process is presented in Table 2.

Table 2 Modified-Delphi process
Full size table

Round 1

All 39 constructs and their definitions were presented, with one item representing each construct. Participants were asked to indicate how important they believe each factor may be in influencing whether providers implement the guidelines into clinical practice. The response options were provided on a 5-point Likert scale ranging from (1) not at all important to (5) very important. Consensus was defined as ≥70% of responses falling within two consecutive categories (e.g., not at all important/not very important; important/very important) [27, 29, 31]. Prior to the survey, the PAB was provided with information on the guidelines and a brief introduction to the CFIR. Definitions provided for each CFIR construct were not operationalized or translated to prenatal oral health but were defined directly from CFIR guidance [32].

Round 2

Most constructs were rated as very important/important by the PAB in Round 1. The questionnaire was revised based on discussions between researchers and the SAB to address challenges with consensus and item wording. Revisions included the following: (1) Question stem was changed from importance to impact to reduce a bias toward inclusion and was modified to: How strong of an impact do you think each of the following factors will have on whether providers implement the prenatal oral health guidelines into their daily clinical practice?; (2) Items were modified to a translated, operationalized version to facilitate readability and application among providers; (3) Response options were changed to a four-point scale ranging from (1) no impact to (4) strong impact; and (4) Definition of consensus was changed; consensus was considered reached when a construct had a mean impact of ≥2.0 (moderate/strong impact). All 39 CFIR constructs were included in this round.

Round 3

This round included only those constructs without consensus or those with mixed results from earlier rounds. The purpose was to determine if consensus could be achieved when updated items from Round 2 were presented to PAB members.

Overall, while the intention of the modified-Delphi process was to narrow down constructs in a clear, linear manner across rounds, we encountered practical challenges where PAB members felt that most constructs were important to include and/or there was no consensus on constructs to be retained. These challenges may have been due to the lack of operationalization and translation of the CFIR constructs in Round 1, requiring revision of all CFIR constructs in Round 2.

Cognitive interviews

After the instrument was revised based on feedback from the SAB and modified-Delphi rounds with the PAB, cognitive interviews were conducted to assess content validity, feasibility, and acceptability among a convenience sample of prenatal and oral health providers. Inclusion criteria were as follows: (1) ≥21 years old, (2) licensed healthcare professional (DMD, MD/DO, CNM), and (3) currently provide prenatal or oral healthcare to pregnant patients. Six participants were recruited through the PAB and the SAB, an appropriate sample size for cognitive interviews using this methodology [33]. Participants completed the online survey and were emailed a copy to review during the interview with the research team.

The cognitive interview guide included questions assessing participants’ overall thoughts of the survey, flow, length, content, readability, and extent of ambiguity of the items and response options. A verbal probing approach was used and elicited insight on understanding, survey content, and organization [33]. The interviews lasted approximately 45 min and detailed notes were recorded. Each participant received $100 for their time. Findings were reviewed and items that were unclear, double-barreled, or misunderstood were revised or removed.

Results

Identifying High-Priority Constructs

In Round 1 of the modified-Delphi process, 24 of the 39 CFIR constructs had consensus based on ≥70% of responses spread across two consecutive categories. In Round 2, after presenting all 39 constructs and using modified question stems and response options, 20 constructs reached consensus. In Round 3, the remaining 17 constructs that did not reach consensus in the prior two rounds were presented and assessed using the same consensus definition from Round 2. A total of 14 constructs reached consensus after this round, but the consensus process and construct retainment were revisited as explained below.

Final deliberation on achieving consensus

In reviewing all three rounds and discussing the various approaches and language to survey items, it was decided to re-analyze Round 2 data (which included the operationalized 39 CFIR constructs) using a modified cut-off value calculated based on Round 2’s mean response across all constructs. For each of the five CFIR domains, item means were ranked ordered (from most to least important), and constructs with the highest mean rankings within each domain were retained. This calculation resulted in 19 retained constructs prior to SAB, PAB, and cognitive interview feedback. The SAB and PAB reviewed the final instrument to ensure that items were clear, scientifically relevant, and had utility to practice settings.

Pre-testing through cognitive interviews

The instrument with items translated from the retained CFIR constructs was pre-tested with six participants (4 prenatal and 2 oral health providers). The survey took an average of 13 min to complete and sociodemographic characteristics of cognitive interview participants are provided in Table 3.

Table 3 Socio-demographic characteristics of cognitive interview participants (n=6)
Full size table

Feedback from cognitive interviews

General modifications

Participants reported that survey items were acceptable and feasible for providers and were well organized within the survey. Participants suggested adding clarifying information at the beginning of the survey to advise participants to not look up information about the guidelines prior to their participation as they would receive more information later in the survey. Participants also recommended adding “I don’t know” as a response option regarding guideline awareness, and changing the wording of the stem for CFIR items to: When deciding to implement the prenatal oral health guidelines into practice, how important are the following things?

CFIR construct changes

Participants identified several areas of improvement in clarifying CFIR items (Table 4). Two items in particular, champions and formally appointed opinion leaders, were perceived as unclear because these roles often overlap in practice. One interviewee described the difference between the two constructs: “A champion is someone who is excited to do it or a colleague, a formally appointed opinion leader was someone who was higher up and yelled at you to do it”. After soliciting SAB and PAB feedback on these items, the champion item was expanded to include the definition of a champion as “a colleague who takes it upon themselves to promote and support the guidelines”.

Table 4 Changes to CFIR survey items after cognitive interviews
Full size table

Similarly, there were challenges differentiating between compatibility and adaptability. One participant felt that “Adaptability and compatibility with the workflow are the same – if it’s adaptable, you can change it to be compatible with your workflow. I might make the difference between the two more clear – one is more like changing the guidelines, one like changing your part in it.” Based on this feedback, these items were modified to clarify the distinct constructs.

Final instrument

The final survey instrument included 21 CFIR items across four domains after separating double-barreled items; implementation climate and leadership engagement were measured by two items each. Five constructs were included from the intervention characteristics domain, two from the process domain, two from the outer setting domain, and 12 from the inner setting domain. Table 5 includes the final operationalization of prioritized CFIR constructs.

Table 5 Final survey
Full size table

Discussion

This study describes a multi-step process to developing a theory-based and practice-informed survey to identify context-specific barriers and facilitators influencing prenatal oral health guideline implementation. This process used three rounds of a modified-Delphi technique, followed by cognitive interviews, and deliberations among the research team, SAB, and PAB members to identify, operationalize, and pre-test CFIR survey items. Many studies using the CFIR to implement and evaluate clinical practices rely largely on qualitative methods to identify salient constructs [34,35,36,37,38,39]. Our approach is unique in employing an interactive process that begins with examining all CFIR constructs to prospectively and quantitatively examine implementation barriers and facilitators [15, 17].

A key finding is the significant challenge researchers encounter when identifying high-priority constructs. It is vital to emphasize to stakeholders that the goal is to identify only those factors of the highest priority. It may be helpful to clarify at this stage that the task is not to classify the directionality of the factor (i.e., barrier vs. facilitator), but to identify all factors that may have an impact on implementation. Similarly, researchers should adhere to psychometric principles, including the language of items and other measurement issues, to facilitate readability and data analysis, production of cut-off values, and decisions about consensus.

The process of achieving consensus can be difficult. Ideally, researchers should establish how consensus will be operationalized and statistically analyzed prior to the study. Nonetheless, given that implementation can be complex in practice, quantitative approaches to measuring implementation constructs must be flexible. In this study, the PAB was emailed instructions, the rationale for this process, and a link to complete the survey for each round. This passive, one-directional communication process did not permit an interactive dialog where constructs could be discussed before completing the survey [27, 28, 30]. Future research should consider including a group forum to facilitate discussion, generation of meaning-making, and opportunity for participants to consider varying perspectives.

Pre-testing the survey via cognitive interviews is paramount to improve survey quality (e.g., clarity, flow) [33, 40,41,42]. The pre-testing process can illuminate how target participants think and behave in research and practice. This phase also underscores the need to utilize health literacy principles, such as explaining what is being asked of them and why, using laymen’s terms, and avoiding double-barreled questions and response options.

Study findings must be considered in light of limitations. A modified-Delphi technique was used to establish consensus instead of a traditional linear Delphi technique method with strict processes and cut-off points. The lack of racial/ethnic diversity and professional roles and use of convenience sampling to recruit participants limits generalizability. The modified-Delphi process could have benefitted from additional participants and methods, such as focus groups, to capture diverse perspectives, including those working across a range of practice settings and holding various positions. Nonetheless, this study’s strengths include an interdisciplinary research team and SAB and PAB members, which comprised stakeholders involved in implementation science, MCH oral health guideline implementation, and healthcare delivery. This study highlights the need for both scientific rigor and flexibility when systematically identifying, translating, and prioritizing theoretical constructs to real-world practice.

Conclusions

Our process and lessons learned may be useful to others who are applying an implementation science theoretical framework, such as CFIR, to develop a quantitative survey to identify barriers and facilitators to implementing guidelines in other settings. Key lessons learned include the importance of soliciting diverse input, distinguishing between importance/impact and direction of impact, and need for additional qualitative methods to elicit context and consensus-building. Balancing theoretical and practical applications remains critical while identifying high-priority implementation factors and advancing theoretical contributions to implementation science.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CFIR:

Consolidated Framework for Implementation Research

PAB:

Practice Advisory Board

SAB:

Scientific Advisory Board

References

  1. Corbella S, Taschieri S, Del Fabbro M, Francetti L, Weinstein R, Ferrazzi E. Adverse pregnancy outcomes and periodontitis: a systematic review and meta-analysis exploring potential association. Quintessence Int. 2016;47(3):193–204.

    PubMed  Google Scholar 

  2. Hartnett E, Haber J, Krainovich-Miller B, Bella A, Vasilyeva A, Lange KJ. Oral Health in Pregnancy. J Obstet Gynecol Neonatal Nurs. 2016;45(4):565–73.

    Article  PubMed  Google Scholar 

  3. Moore S, Ide M, Coward PY, Randhawa M, Borkowska E, Baylis R, et al. A prospective study to investigate the relationship between periodontal disease and adverse pregnancy outcome. Br Dent J. 2004;197(5):251–8 discussion 47.

    Article  PubMed  CAS  Google Scholar 

  4. Finlayson TL, Gupta A, Ramos-Gomez FJ. Prenatal Maternal Factors, Intergenerational Transmission of Disease, and Child Oral Health Outcomes. Dent Clin N Am. 2017;61(3):483–518.

    Article  PubMed  Google Scholar 

  5. American College of Obstetricians and Gynecologists. Oral health care during pregnancy and through the lifespan. Committee Opinion No. 569. 2013. Available from: https://www.acog.org/clinical/clinical-guidance/committee-opinion/articles/2013/08/oral-health-care-during-pregnancy-and-through-the-lifespan.

  6. March of Dimes. Dental health during pregnancy 2019. Available from: https://www.marchofdimes.org/pregnancy/dental-health-during-pregnancy.aspx.

  7. Boggess KA, Urlaub DM, Moos MK, Polinkovsky M, El-Khorazaty J, Lorenz C. Knowledge and beliefs regarding oral health among pregnant women. J Am Dent Assoc. 2011;142(11):1275–82.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Yee LM, Simon MA, Grobman WA, Rajan PV. Pregnancy as a "golden opportunity" for patient activation and engagement. Am J Obstet Gynecol. 2021;224(1):116–8.

    Article  PubMed  Google Scholar 

  9. Oral Health Care During Pregnancy Expert Workgroup. Oral health care during pregnancy: a national consensus statement - summary of an expert workgroup meeting. Washington, D.C.: National Maternal and Child Oral Health Resource Center; 2012.

    Google Scholar 

  10. Pina P, Douglass J. Practices and opinions of Connecticut general dentists regarding dental treatment during pregnancy. Gen Dent. 2011;59(1):e25–31.

    PubMed  Google Scholar 

  11. Vamos CA, Walsh ML, Thompson E, Daley EM, Detman L, DeBate R. Oral-systemic health during pregnancy: exploring prenatal and oral health providers' information, motivation and behavioral skills. Matern Child Health J. 2015;19(6):1263–75.

    Article  PubMed  Google Scholar 

  12. Agency for Healthcare Research and Quality. Clinical Guidelines and Recommendations Rockville, MD: 2021. Updated May 2021. Available from: https://www.ahrq.gov/prevention/guidelines/index.html.

  13. Beauchemin M, Cohn E, Shelton RC. Implementation of Clinical Practice Guidelines in the Health Care Setting: A Concept Analysis. ANS Adv Nurs Sci. 2019;42(4):307–24.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud P-AC, et al. Why don't physicians follow clinical practice guidelines? a framework for improvement. JAMA. 1999;282(15):1458–67.

    Article  PubMed  CAS  Google Scholar 

  15. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Kalkan A, Roback K, Hallert E, Carlsson P. Factors influencing rheumatologists' prescription of biological treatment in rheumatoid arthritis: an interview study. Implement Sci. 2014;9:153.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Forman J, Harrod M, Robinson C, Annis-Emeott A, Ott J, Saffar D, et al. First things first: foundational requirements for a medical home in an academic medical center. J Gen Intern Med. 2014;29(Suppl 2):S640–8.

    Article  PubMed  Google Scholar 

  20. Luck J, Bowman C, York L, Midboe A, Taylor T, Gale R, et al. Multimethod evaluation of the VA's peer-to-peer Toolkit for patient-centered medical home implementation. J Gen Intern Med. 2014;29(Suppl 2):S572–8.

    Article  PubMed  Google Scholar 

  21. Possemato K, Kuhn E, Johnson EM, Hoffman JE, Brooks E. Development and refinement of a clinician intervention to facilitate primary care patient use of the PTSD Coach app. Transl Behav Med. 2017;7(1):116–26.

    Article  PubMed  Google Scholar 

  22. Fernandez ME, Walker TJ, Weiner BJ, Calo WA, Liang S, Risendal B, et al. Developing measures to assess constructs from the Inner Setting domain of the Consolidated Framework for Implementation Research. Implement Sci. 2018;13(1):52.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Connell LA, McMahon NE, Watkins CL, Eng JJ. Therapists' use of the Graded Repetitive Arm Supplementary Program (GRASP) intervention: a practice implementation survey study. Phys Ther. 2014;94(5):632–43.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Cragun D, DeBate RD, Vadaparampil ST, Baldwin J, Hampel H, Pal T. Comparing universal Lynch syndrome tumor-screening programs to evaluate associations between implementation strategies and patient follow-through. Genet Med. 2014;16(10):773–82.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Shimada SL, Hogan TP, Rao SR, Allison JJ, Quill AL, Feng H, et al. Patient-provider secure messaging in VA: variations in adoption and association with urgent care utilization. Med Care. 2013;51(3 Suppl 1):S21–8.

    Article  PubMed  Google Scholar 

  26. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6(3):e34.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–9.

    Article  PubMed  Google Scholar 

  28. Dalkey N. The Delphi Method: An Experimental Study of Group Opinion. Santa Monica: RAND Corporation; 1969.

    Google Scholar 

  29. Vogel C, Zwolinsky S, Griffiths C, Hobbs M, Henderson E, Wilkins E. A Delphi study to build consensus on the definition and use of big data in obesity research. Int J Obes. 2019;43(12):2573–86.

    Article  Google Scholar 

  30. Eubank BH, Mohtadi NG, Lafave MR, Wiley JP, Bois AJ, Boorman RS, et al. Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Med Res Methodol. 2016;16(1):56.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Slade SC, Dionne CE, Underwood M, Buchbinder R. Standardised method for reporting exercise programmes: protocol for a modified Delphi study. BMJ Open. 2014;4(12):e006682.

    Article  PubMed  PubMed Central  Google Scholar 

  32. CFIR Research Team-Center for Clinical Management Research. CFIR constructs. 2022. Available from: https://cfirguide.org/constructs/.

  33. Knafl K, Deatrick J, Gallo A, Holcombe G, Bakitas M, Dixon J, et al. The analysis and interpretation of cognitive interviews for instrument development. Res Nurs Health. 2007;30(2):224–34.

    Article  PubMed  Google Scholar 

  34. Toomey CM, Kennedy N, MacFarlane A, Glynn L, Forbes J, Skou ST, et al. Implementation of clinical guidelines for osteoarthritis together (IMPACT): protocol for a participatory health research approach to implementing high value care. BMC Musculoskelet Disord. 2022;23(1):643.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Yaguchi-Saito A, Kaji Y, Matsuoka A, Okuyama A, Fujimori M, Saito J, et al. Factors affecting the implementation of guideline-based prophylactic antiemetic therapy for chemotherapy-induced nausea and vomiting in Japan: a protocol for a hospital-based qualitative study. BMJ Open. 2022;12(6):e055473.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Sothornwit J, Lumbiganon P, Saranrittichai K, Sangkomkamhang U, Singhdaeng T, Jampathong N. Barriers and Facilitators to Implementing Immediate Postpartum Contraceptive Implant Programs: A Formative Implementation Research. Int J Women's Health. 2022;14:945–56.

    Article  Google Scholar 

  37. Loo S, Mullikin K, Robbins C, Xiao V, Battaglia TA, Lemon SC, et al. Patient navigator team perceptions on the implementation of a citywide breast cancer patient navigation protocol: a qualitative study. BMC Health Serv Res. 2022;22(1):683.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Inness EL, Jagroop D, Andreoli A, Bayley M, Biasin L, Danells C, et al. Factors That Influence the Clinical Implementation of Aerobic Exercise in Stroke Rehabilitation: A Theory-Informed Qualitative Study. Phys Ther. 2022;102(6).

  39. Ablordeppey EA, Keating SM, Brown KM, Theodoro DL, Griffey RT, James AS. Implementation of ultrasound after central venous catheter insertion: A qualitative study in early adopters. J Vasc Access. 2021:11297298211053447.

  40. Drennan J. Cognitive interviewing: verbal data in the design and pretesting of questionnaires. J Adv Nurs. 2003;42(1):57–63.

    Article  PubMed  Google Scholar 

  41. García AA. Cognitive Interviews to Test and Refine Questionnaires. Public Health Nurs. 2011;28(5):444–50.

    PubMed  Google Scholar 

  42. Willis GB, Artino AR Jr. What Do Our Respondents Think We're Asking? Using Cognitive Interviewing to Improve Medical Education Surveys. J Grad Med Educ. 2013;5(3):353–6.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Practice Advisory Board members and participants for their time in this study. The authors would also like to thank Ms. Nicole Harris for her assistance with formatting the manuscript.

Funding

Research reported in this manuscript was supported by the National Institute of Dental & Craniofacial Research of the National Institutes of Health under Award Number R03DE024633. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and Affiliations

  1. College of Public Health, University of South Florida, 13201 Bruce B. Downs Blvd MDC 56, Tampa, FL, 33612, USA

    Cheryl A. Vamos, Ellen M. Daley, Morgan Richardson Cayama & Jason Beckstead

  2. School of Public Health, The University of North Texas Health Science Center at Fort Worth, 3500 Camp Bowie Blvd, Fort Worth, TX, 76107, USA

    Stacey B. Griner

  3. Department of Obstetrics and Gynecology, School of Medicine, University of North, Carolina at Chapel Hill, CB 7516, Chapel Hill, NC, 27599, USA

    Kim Boggess

  4. Department of Pediatric Dentistry, Schools of Dentistry, Pediatrics and Public Health, University of North Carolina at Chapel Hill, CB 7450, Chapel Hill, NC, 27599, USA

    Rocio B. Quinonez

  5. Implementation Pathways, Chelsea, Michigan, USA

    Laura Damschroder

Authors
  1. Cheryl A. Vamos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stacey B. Griner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ellen M. Daley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Morgan Richardson Cayama
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jason Beckstead
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kim Boggess
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Rocio B. Quinonez
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Laura Damschroder
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CV, ED, and SG conceptualized and designed the study. KB, RQ, and LD provided scientific guidance throughout the entire study. Data collection was performed by CV and SG. Data analysis was performed by CV, SG, and JB. Writing of the original draft was performed by CV, SG, and MRC. All authors (CV, SG, ED, MRC, JB, KB, RQ, LD) reviewed and edited manuscript drafts and approved the final manuscript.

Corresponding author

Correspondence to Cheryl A. Vamos.

Ethics declarations

Ethics approval and consent to participate

The University of South Florida’s Institutional Review Board reviewed and approved this study (Pro00037391). All participants provided, written, verbal or electronic consent before participating in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vamos, C.A., Griner, S.B., Daley, E.M. et al. Prenatal oral health guidelines: a theory- and practice-informed approach to survey development using a modified-Delphi technique and cognitive interviews. Implement Sci Commun 3, 126 (2022). https://doi.org/10.1186/s43058-022-00363-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-022-00363-6

Keywords

Implementation Science Communications

ISSN: 2662-2211