Skip to main content
  • Study protocol
  • Open access
  • Published:

A mixed-methods protocol for identifying successful sustainability strategies for nutrition and physical activity interventions in childcare



Despite the importance of sustainability for nutrition and physical activity in public health interventions, limited studies have explored the factors that promote and inhibit evidence-based program sustainment in the childcare setting. This study protocol describes a mixed-methods approach to develop novel sustainability strategies based on real-world settings and stakeholder feedback, with the goal of providing support for future obesity prevention programs and related studies on intervention sustainability. Two interventions, Together, We Inspire Smart Eating (WISE) and The Food Friends’ (FF) Fun with New Foods and Get Movin’ with Might Moves, are studied to this end.


The study will deploy an explanatory, sequential mixed-methods design. First, the research team will collect a quantitative survey to assess rates of sustainment among WISE and Food Friends sites. We expect to collect 150 surveys from WISE and FF sites combined. Data from these surveys will be used to purposively sample sites for 12 to 18 site visits. Specifically, we will purposively sample low, partial, and high sustaining sites where we will conduct key informant interviews and focus groups as well as validate self-reports on sustainability. Survey content, qualitative interviews, and coding will be based on the Dynamic Sustainability Framework. We will draw on findings from the quantitative survey on predictors of sustainment and the qualitative site visits to understand varying levels of program sustainment. Then, we will utilize evidence-based quality improvement sessions to engage stakeholders in developing a multi-component sustainability strategy.


This study will provide a stakeholder-informed sustainability strategy ready for testing in a full-scale trial examining effects on sustainment of evidence-based nutrition and physical activity practices in childcare. We expect this strategy to be relevant for educators and consistent with the views of administrators as a guide for future practice for the targeted nutrition and physical activity interventions and beyond.


Children between 2 and 5 years of age in the USA rarely meet dietary recommendations, and most fall short of achieving recommended levels of physical activity [1,2,3,4]. Further, 40% of children are overweight or obese by age 5 [5]. Such excess weight increases the risk for metabolic syndrome, cardiovascular disease, obesity, and some cancers in adulthood [6, 7]. Consuming a healthy diet (e.g., fiber and antioxidant-rich) [8,9,10], engaging in physical activity (PA), [10,11,12], and maintaining a healthy weight [10, 13, 14] provide significant protection against disease. Given that early life patterns track into adulthood, [15, 16] early childhood is an important time period for such prevention efforts. Because a large majority of young children in the USA spend an average of 30 h per week or more in early care and education programs [17], childcare is a promising setting for implementing and sustaining nutrition and physical activity programs.

Importance of sustainment for disease prevention

Sustainability is both (a) the process by which an innovation becomes integrated into existing systems [18] and (b) an outcome in which an innovation is sustained after initial implementation or funding ends [19]. Sustainability is a key target outcome for public health interventions and programs to have the desired effects on population health [20]. Yet, the sustainability of public health programs varies, with estimates indicating that 40 to 60% of programs cease or operate at lower levels after initial funding is withdrawn [21,22,23,24,25]. Thus, sustaining evidence-based programs in the community setting is a significant challenge, undermining investments in implementation. Reflective of this gap, leaders in the field of implementation science have identified sustainability as a priority area for future studies [26].

Target interventions to inform sustainment in childcare

Sustaining effective programs for nutrition and physical activity in childcare would promote disease prevention in a setting serving a large number of children; yet, little is known about factors that predict sustainability (or lack thereof) [27, 28]. Two existing interventions in the childcare setting provide an opportunity to study factors associated with sustainment in childcare—Together, We Inspire Smart Eating (WISE)® and Food Friends® (FF; Fun with New Foods and Get Movin’ with Mighty Moves). Both interventions include three elements: (a) a classroom curriculum, (b) educator training, and (c) parent education using materials for outreach [29].

WISE was designed to establish healthy early eating habits with food experiences and supporting activities executed weekly for children 3 to 8 years old across a 9-month school term in Arkansas. WISE has been disseminated for approximately 7 years. WISE has created positive changes in child and family eating behaviors that align with the current recommended Dietary Guidelines for Americans, specifically, increasing reported consumption of WISE fruits and vegetables, more fruits and vegetables in general, and decreasing intake of nutrient-poor food compared to children not experiencing WISE and before exposure to WISE [30, 31].

FF is a preschool program designed to address childhood obesity by establishing healthful eating and physical activity behaviors in preschool-aged children. The program consists of two components (1) Fun With New Foods®, a 12-week intervention, which focuses on helping children increase their willingness to try new foods, and (2) Get Movin’ With Mighty Moves®, an 18-week intervention, which aims to enhance preschoolers’ gross motor skill development. The program is based on constructs of the social cognitive theory, tenets of social marketing, and is embedded within Bronfenbrenner’s social ecological framework [32]. FF was implemented mainly in Colorado for over 20 years and has successfully demonstrated increases in both children’s willingness to try new foods (food preference) and gross motor performance in preschool-aged children in the short term [33, 34] and longitudinally [35, 36].

The current study aims to understand sustainment factors common across and unique to each intervention. We expect examining similarities and differences across two programs will contribute to knowledge that will be more generalizable to promote sustainment for a range of evidence-based practices and interventions in childcare. To this end, we will complete two aims:

Aim 1: Identify factors associated with sustaining nutrition/PA programs in childcare settings

We will use an explanatory, sequential mixed-methods design to (a) collect survey data and conduct multivariate analyses to understand predictors of sustainability and (b) conduct site visits to perform key informant interviews and focus groups; validate self-reports on sustainability; identify environmental contributors to sustainability; and document program adaptations. We will also conduct interviews with external stakeholders to understand decisions affecting support for nutrition/PA programs.

Aim 2: Develop a multi-faceted sustainability strategy with stakeholder input

We will use evidence-based quality improvement (EBQI) sessions [37,38,39] to engage stakeholders in developing a multi-component sustainability strategy. Data from aim 1 on the predictors of sustainability and innovation from sites with high sustainment will be used to inform stakeholder-selected strategies.

Theoretical framework

The proposed project will use the Dynamic Sustainability Framework (DSF) [40] to understand predictors of sustainability for interventions in childcare and to design sustainability strategies to be tested in this setting. The DSF posits that characteristics of the intervention (e.g., components and practitioners), the context (e.g., climate and staffing), the larger ecological system (e.g., policy and population characteristics), and the fit among these elements contribute to sustainability outcomes. Consistent with our focus on adaptation, the DSF recognizes that adaptation is inevitable and may be key to enhancing long-term sustainability. In the proposed study, this framework will inform the selection, adaption, and development of quantitative and qualitative measures in aim 1. Additionally, we will apply the Wiltsey-Stirman framework [41] for coding program modifications and adaptations to qualitative data gathered in aim 1. The Expert Recommendations for Implementing Change (ERIC) taxonomy of implementation strategies [42] will be used as a starting point for developing and selecting sustainment strategies in aim 2. The integration of these will provide a solid foundation for the proposed research.


Aim 1

Study design

Figure 1 provides an overview of study activities. All study procedures will be conducted in accordance with protocols approved by the University of Arkansas for Medical Sciences Institutional Review Board (IRB). Aim 1 of the current study will identify predictors of sustainability unique to each program and common across programs. We will focus on three aspects of sustainability: (1) “continued attention” to nutrition and physical activity promotion and obesity prevention [20], (2) the degree to which centers use original program components [20, 43], and (3) the extent and nature of modifications to components [43]. We will also examine different levels of sustainability: high, partial, and low (defined below). Our multi-pronged approach recognizes that sustainment is not a dichotomous outcome, and partial sustainment [43,44,45] is likely common. Thus, we will focus both on which aspects of each program are sustained in the current school year and to what degree they are sustained (e.g., frequency, length, and fidelity). First, we will use a quantitative survey to identify predictors of self-reported sustainability. Then, we will conduct purposive site visits to (1) validate the degree of sustainability with in-person fidelity assessments, (2) collect in-depth information to understand local-level decision making relative to sustainability, (3) identify and categorize adaptations to the programs, and (4) document strategies used by high sustaining programs which could be tested in the future.

Fig. 1
figure 1

Study activities and future plans

We will use an explanatory, sequential mixed-methods design to understand barriers and facilitators specific to sustainability (quant→QUAL) [46, 47]. See Table 1 for data sources for aim 1. First, we will complete a director survey of current sustainability. Quantitative data will be gathered with an online REDCap survey of directors from FF and WISE sites (N = 373 potential sites; NFF = 210; NWISE = 163). Because directors approve lesson plans and curricular purchases, they can report on program activities. We have set the parameters of the sampling frame for the director survey to be receipt of Child and Adult Care Food Program support (a proxy for serving families affected by low income) and being trained (or retrained) between 2010 and 2017. Notably, our databases include data on variables such as dates of training. The director survey will incorporate scales from the established Program Sustainability Assessment Tool (PSAT) [48] to measure funding/resource stability, community partnerships, organizational capacity, program evaluation and adaptation, communications, and strategic planning. The PSAT was developed for public health programs, but we will adapt it via cognitive interviews (n = 5) with teachers to be retrospective and to reflect the childcare setting. The survey also will use established measures to assess the organizational climate [49], perceptions of the innovation [50], and reported fidelity to core components [51].

Table 1 Data sources for aim 1

The survey will begin with questions about current nutrition/PA programming using broad questions and then, via branching logic, specific questions (program component usage; investment to sustain programming; use of technical assistance, follow-up training, contact with program administrators; and plans for future use). From this information, we will determine three categories of program sustainability: (1) high sustainability programs continue to implement core components with or without appropriate adaptations; (2) partial sustainability programs use some, but not all, components; and (3) low sustainability programs no longer use FF/WISE or exhibit continued attention to nutrition/PA programming.

Directors will be invited by email to complete the survey online. Completers will be entered into 1 of 15 drawings for $100. We expect 150 responses (40%) based on our experience in recruiting childcare participants. We will use additional strategies to increase participation (e.g., phone calls, mailed flyers, visits) if we cannot reach our target response rate [47]. For example, we will draw on relationships with county-based Extension educators, early childhood councils, and Child Care Resource and Referral agencies. We will also make presentations at state conferences and meetings to engage with potential participants.

Qualitative data will be collected based on survey data. Sites will be purposively sampled (criterion-i) [52] to reflect the 3 categories of sustainability. We expect to target 2–3 sites per category in Arkansas (AR) and Colorado (CO). Thus, our total sample will include 12–18 sites split evenly across AR and CO and across sustainment categories. During site visits (2–3 days), the research team will conduct semi-structured interviews with directors and mini focus groups with educators (groups of 4–6 per Krueger and Casey) [53, 54]. Director interviews will include questions about external factors influencing sustainability decisions (e.g., regulations, funding). We will query whether institutional knowledge about FF/WISE was transferred during leadership changes and how decisions to sustain (or not) were made. Mini focus groups with teachers are cost-effective and allow them to share experiences without directors present [54]. Interviews and focus groups, informed by the DSF, will ask about contextual elements (e.g., values, organizational culture) that influenced sustainability decisions, processes in place to support sustainability, and the decision-making process to reduce, abandon, or adapt use of the program.

We will also engage relevant external stakeholders in each state for in-depth interviews to understand external factors affecting sustainability. Targets include delegates from the state Departments of Education and Health, leaders from prior funding agencies, members of policy councils, and agency administration (which provides leadership beyond site-level directors). We expect to interview 3 to 5 external stakeholders per state. Interview guides for mini focus groups and interviews will align with constructs of the DSF [40] and be modified as needed [54]. To ensure the validity of the data, the research team will follow a standard training protocol, including multiple rounds of mock interviews [55]. Interviews and mini focus groups will last 30 to 60 min and be transcribed verbatim. Interview and focus group participants will receive a $25 incentive.

Observational data will include observer field notes [54, 56] and two quantitative measures, the Environmental and Policy Assessment and Observation (EPAO) [57] and an assessment of fidelity tailored to each program [58]. The EPAO, a reliable and valid measure of childcare nutrition and PA environments, will provide a standardized assessment of contextual factors that could support or interfere with the sustainability of the FF and WISE programs. We will prioritize EPAO modules that measure programmatic practices (e.g., planned nutrition and PA lessons) and behaviors that best capture influence of the program or training (e.g., provider interactions). We will exclude less relevant details (e.g., condiment use). The PIs will train research staff using the published online EPAO modules and certification tools [59]; research staff will also be trained to document observable characteristics that are important to program implementation (e.g., program materials present and interpersonal climate). Notable adaptations of the program will be analyzed with the Wiltsey-Stirman framework [41]. Researchers will also document the use of skills (taught by the programs) outside of programmatic lessons (e.g., encouraging sensory exploration of foods). Finally, the fidelity measure will assess the quality and dosage of program components during a classroom lesson (e.g., engagement with mascots, frequency of role-modeling) and build on our own published measures [58].

Participants, setting, and sample size

All directors and educators previously trained in FF or WISE will compose the potential participant pool for aim 1 surveys, interviews, and focus groups. We will recruit participants through contacts (first) from research staff and from county Extension or childcare resource and referral agents (second) to ensure representation across racial/ethnic groups and geographic regions of AR and CO. Purposively sampled sites for site visits will be recruited into the study through informational meetings and IRB-approved consenting procedures. Participation in aim 1 activities will be short term and will not require continued engagement after the initial data collection.

Aim 1 data analysis

Surveys will provide quantitative data on (1) the levels of sustainment across programs and components, (2) factors unique and common to predicting sustainability, and (3) the moderating effect of length of sustainment (lag) on identifying salient predictors of sustainability. We will examine the data for missing values, extreme scores, and the shape of distributions for each variable. Outliers will be excluded and will use full information maximum likelihood in analyses to manage missing data greater than 5%. We will also compare item and summary scores across program types to identify systematic differences between nutrition programs. The primary analysis will use a general linear model (multiple regression) with sustainment scores as the outcome and the following predictors: PSAT scores, program type, context characteristics (internal, external, and program factors), time since program implementation (lag), and the interactions of PSAT scores and lag. A statistically significant (p < .05) result from the t test for each predictor will be used to identify factors associated with program sustainability. We will also determine if there are interactions between PSAT scores and program type that suggest different models of sustainment for FF vs. WISE (p < .05). Assuming a response of 40% from our participant pool, 150 sites will be represented in the director survey. With N = 150 (two-sided α = .05), we would have 80% power to detect a Pearson correlation of 0.23. For comparisons of program type, assuming nWISE = 45 and nFF = 105 (two-sided α = .05), we would have 80% power to detect a Cohen’s d of 0.5 or a medium-sized effect. In the proposed multiple regression (two-sided α = .05, 10 predictors), we would have 80% power to detect an f2 effect size of 0.05 (a little larger than a “small” effect) [60]. For context, with a base model accounting for 25% of response variance, we would have 80% power for an added predictor accounting for an additional 3.9% of variance. These power estimates apply for tests of both main effects and interactions. Recruiting 30% of our participant pool (N = 112) would provide 80% power to detect a Pearson correlation of 0.26, an f2 effect size of 0.07, and a Cohen’s d of 0.54 for group comparisons, assuming nWISE = 49 and nFF = 63.

Qualitative interviews with targeted sub-groups will provide information on the barriers and facilitators to sustainability specific to each program and for those that co-exist across programs. Further, we will be able to compare self-reported perceptions of sustainability with observational data and to collect data on how agencies decide to reduce, abandon, or adapt the use of a program. Transcripts will be coded by the PIs and trained research assistants using directed content analysis [61]. Observer field notes will be matched to interviews. Nvivo software will be used for team coding. The DSF [40] will provide sensitizing concepts to build initial codes that will succinctly label significant, recurrent ideas across participants. Analysis will focus on identifying common and distinct themes across the three categories of sustainability. Inductive codes will be incorporated into the codebook as they develop [62]. Primary and secondary coders (at least one each from AR and CO) will code the same transcripts until inter-rater reliability is established. Afterward, coding will be independent, but we will hold weekly consensus meetings to reduce bias and ensure reliability and validity. At weekly meetings, disagreements will be resolved, and the codebook will be expanded as needed. Summaries of site-level findings will be presented back to participants for member checking.

Aim 2

Study design

We will use a stakeholder-driven process called evidence-based quality improvement (EBQI) [37,38,39, 63,64,65,66] to ensure that our sustainability strategy reflects cultural needs and preferences. EBQI has been used frequently to develop strategies and adapt interventions in primary care, outpatient care, and pharmacy practice [38, 64, 67, 68]. Consistent with the DSF [40], we expect that engaging stakeholders will contribute to the most effective strategies. EBQI panels will operate according to principles of community-based participatory research [69] and best-practices for engaging stakeholders in implementation [70]. Childcare directors and educators will contribute the local knowledge needed to select and tailor strategies; the research team will provide expertise on implementation approaches.

The EBQI process will (1) leverage strategies that worked for high sustainability sites from aim 1, (2) prioritize barriers and facilitators to sustainability reported from sites and external stakeholders from aim 1, (3) identify theoretically informed sustainability strategies that match priorities/needs, and (4) tailor sustainability strategies to the early childhood context. The EBQI panel will include 8 to 12 stakeholders combining FF and WISE users, and the diversity of the panel will reflect the diversity of the target populations.

EBQI is a flexible process conducted across a series of meetings with topic-driven agendas; each session will last 2 h. In EBQI session 1, the research team will present a summary of aim 1 findings, conduct a “member checking” exercise to assess validity of the findings, reach consensus on prioritized barriers and facilitators to inform the sustainment strategies, and vet sustainment strategies identified in aim 1. In session 2, we will present potential sustainment strategies mapped by the research team to the ERIC [42] taxonomy of implementation strategies, incorporating knowledge on strategies that worked in high sustainability sites and predictors of sustainability (from aim 1). To reach a consensus on sustainment strategies, we will use techniques outlined by Powell et al. [71], including concept mapping with virtual polling. This method provides quantifiable information and promotes the efficient collection of input in real time. Our existing relationships with Extension offices and regional medical facilities will facilitate virtual meetings in all counties. In session 3, we will present the draft strategies/tools, collect feedback for revisions, and receive final approval. In sessions 4 and 5, we will present our plan for a future pilot test of the sustainability strategy to assess acceptability and adoption. Stakeholders will receive $50 per session. By applying the EBQI process to the study of sustainability rather than implementation, our findings may be broadly applicable to the field of implementation science.

Participants, setting, and sample size

We will engage key community stakeholders to serve on our EBQI panel and provide input on the development of a sustainment strategy. This process will review the existing scientific evidence and data from aim 1 with the EBQI panel to solicit stakeholder input on how best to sustain nutrition/PA programs in childcare. EBQI sessions will be audio-recorded so that we can review the content covered in each session and rapidly code the reactions of the EBQI panel. These data will be collected anonymously. Panel members may also be asked to complete survey instruments to provide quantitative input on proposed strategies. The identity of EBQI members will not be included in reports or manuscripts. We expect to engage up to 12 stakeholders for the panel. The EBQI panel will meet 5 times virtually.

For aim 2, stakeholders and pilot participants will be recruited through IRB approved procedures, focusing on agencies with which we have continued partnerships. We expect these activities to include site visits, emails, contacts at early childhood conferences, and phone calls. Stakeholder recruitment will reflect the demographics of early-childhood educators in each state as published by the state departments of education. External stakeholders will be recruited through established relationships with governing, funding, and administrative bodies in each respective state.

Data analysis

We will use an online platform and database server to collect and store the EBQI panel’s perceptions of feasibility and importance of potential sustainment strategies. Then, we can query the database [72] in real time to plot potential strategies by their rated importance (x-axis) and feasibility (y-axis), consistent with a concept mapping approach [71]. Strategies above the mean for both criteria will be considered for inclusion in the sustainability strategy. After each EBQI session, research team members will document what they observed and heard, what was resolved, and what remains undecided. These meeting minutes will guide subsequent EBQI sessions. The research team will also assimilate panel feedback, translate it to actionable plans, and develop the next iteration of materials for panel input. Qualitative information from meeting minutes and audio recordings will be analyzed using directed content analysis [61] relative to the main goals of the EBQI process (e.g., matching barriers/facilitators to sustainability strategies). After data from the final EQBI meeting are analyzed and incorporated into the design, the sustainability approach will be fully specified.


This study will help address critical gaps in knowledge about the extent to which nutrition education and PA programs are sustained in childcare settings and the predictors of sustainability (internal and external factors within/between intervention programs). The results of this study are likely to provide useful strategies for sustaining evidence-based programs in childcare, as well as in other settings (e.g., home visiting, worksites, and schools). Successful implementation of sustainability strategies will inform other projects as to the process of engaging stakeholders to inform the selection and development of optimal sustainability strategies, an essential approach for sustaining program impact [73, 74]. Finally, our study will provide a “follow up of factors that contribute to the sustainability of evidence-based interventions in public health” and an opportunity to study the relationships between the “context and local capacity of clinical and community settings” and sustainability [75]. This addresses a critical gap in public health to identify and prioritize strategies to promote sustainability [76].

Several factors make our study of sustainability unique. First, we will study two interventions simultaneously—each in different stages of its life cycle. Specifically, FF has ended dissemination efforts after 20 years. However, some sites continue to use the program despite workforce turnover and withdrawal of state-level support. Analyzing the life-cycle of FF will help us understand and contribute to the design of novel sustainability strategies and ensure the full benefit of investments in newer programs (such as WISE). Second, our study will be the longest-range view of sustainability to date (over 20 years for some Food Friends sites), providing a unique opportunity to study long-range predictors of sustainability. In addition, our study will provide an opportunity, through site visits, to assess unintended consequences of sustainment, and the ability to compare self-reported and observed measures of sustainability. Furthermore, the proposed project will provide data about successful adaptations that have been made to programs to promote sustainability and how these decisions were made.

Challenges and limitations

To include a pilot study of the sustainability strategy would be ideal; however, this is beyond the scope of the current study. Alternatively, we will dedicate a follow-up study to a robust pilot of sustainability strategies before a future, large-scale implementation trial. An additional potential problem is coordinating the stakeholders across Arkansas and Colorado. However, the study team has experience managing the logistics of stakeholder panels with diverse members conducted via virtual meeting. Lastly, the COVID-19 pandemic has resulted in significant changes to the research and childcare landscape. For example, over 45 sites trained in WISE have closed permanently. We have included power estimates for a reduced sample size should COVID-19 affect our ability to recruit participants. Further, the research team will remain responsive to COVID-19 conditions in our methodologies (e.g., virtual site visits versus in-person site visits as proposed).


We expect our study to produce a multi-faceted package of sustainability strategies based on real-world settings. Our stakeholder-informed sustainability strategy is anticipated to be relevant for educators and consistent with the views of administrators. With the combination of mixed-methods approach, stakeholder engagement, and analysis across multiple interventions, our study has the potential to contribute towards the advancement the science of sustainability.

Availability of data and materials

The datasets used and/or analyzed during the current study will be available from the corresponding author on reasonable request.



Physical activity


Food Friends, Fun with New Foods & Get Movin’ with Mighty Moves


Together, We Inspire Smart Eating


Evidence-based quality improvement


Dynamic Sustainability Framework


Expert Recommendations for Implementing Change


Implementation science


Program Sustainability Assessment Tool


Environmental and Policy Assessment and Observation


  1. Ward ZJ, Long MW, Resch SC, Giles CM, Cradock AL, Gortmaker SL. Simulation of growth trajectories of childhood obesity into adulthood. N Engl J Med. 2017;377:2145–53 Massachusetts Medical Society.

    Article  PubMed  Google Scholar 

  2. Gu X, Tucker KL. Dietary quality of the US child and adolescent population: trends from 1999 to 2012 and associations with the use of federal nutrition assistance programs. Am J Clin Nutr. 2017;105:194–202 Oxford University Press.

    Article  CAS  PubMed  Google Scholar 

  3. Banfield EC, Liu Y, Davis JS, Chang S, Frazier-Wood AC. Poor adherence to US dietary guidelines for children and adolescents in the national health and nutrition examination survey population. J Acad Nutr Diet. 2016;116:21–7 Elsevier.

    Article  PubMed  Google Scholar 

  4. Katzmarzyk PT, Denstel KD, Beals K, Bolling C, Wright C, Crouter SE, et al. Results from the United States of America’s 2016 report card on physical activity for children and youth. J Phys Act Heal. 2016;13:S307–13 Available from:

    Article  Google Scholar 

  5. Skinner AC, Ravanbakht SN, Skelton JA, Perrin EM, Armstrong SC. Prevalence of obesity and severe obesity in US children, 1999-2016. Pediatrics. 2018;141:e20173459 American Academy of Pediatrics. Available from:

    Article  PubMed  PubMed Central  Google Scholar 

  6. Shah AS, Urbina EM. Childhood obesity, atherogenesis, and adult cardiovascular disease. Pediatr Obes. 2018:527–38 Cham: Springer International Publishing.

  7. Biro FM, Wien M. Childhood obesity and adult morbidities. Am J Clin Nutr. 2010;91:1499S–505S Oxford University Press.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Lohse T, Faeh D, Bopp M, Rohrmann S. Adherence to the cancer prevention recommendations of the World Cancer Research Fund/American Institute for Cancer Research and mortality: a census-linked cohort. Am J Clin Nutr. 2016;104:678–85 Oxford University Press.

    Article  CAS  PubMed  Google Scholar 

  9. Norat T, Aune D, Chan D, Romaguera D. Fruits and vegetables: updating the epidemiologic evidence for the WCRF/AICR lifestyle recommendations for cancer prevention. Cancer Treat Res. 2014;159:35–50 Springer, Berlin, Heidelberg.

    Article  CAS  PubMed  Google Scholar 

  10. World Cancer Research Fund/American Institute for Cancer Research. Diet, nutrition, physical activity and cancer: a global perspective | Third Expert Report. 2018.

    Google Scholar 

  11. Brenner DR, Poirier AE, Grundy A, Khandwala F, McFadden A, Friedenreich CM. Cancer incidence attributable to inadequate physical activity in Alberta in 2012. C Open. 2017;5:E338–44 Canadian Medical Association. Available from:

    Article  Google Scholar 

  12. Thomson CA, McCullough ML, Wertheim BC, Chlebowski RT, Martinez ME, Stefanick ML, et al. Nutrition and physical activity cancer prevention guidelines, cancer risk, and mortality in the women’s health initiative. Cancer Prev Res (Phila). 2014;7:42–53 Available from: American Association for Cancer Research.

    Article  Google Scholar 

  13. Bray GA, Heisel WE, Afshin A, Jensen MD, Dietz WH, Long M, et al. The science of obesity management: an endocrine society scientific statement. Endocr Rev. 2018;39:79–132.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Lauby-Secretan B, Scoccianti C, Loomis D, Grosse Y, Bianchini F, Straif K. Body fatness and cancer — viewpoint of the IARC Working Group. N Engl J Med. 2016;375:794–8 Massachusetts Medical Society.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Mennella JA. Ontogeny of taste preferences: basic biology and implications for health. Am J Clin Nutr. 2014;99:704S–11S American Society for Nutrition.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Simmonds M, Llewellyn A, Owen CG, Woolacott N. Predicting adult obesity from childhood obesity: a systematic review and meta-analysis. Obes Rev. 2016;17(10.1111):95–107 Wiley/Blackwell.

    Article  CAS  PubMed  Google Scholar 

  17. Laughlin L. Who’s minding the kids? Child care arrangements; 2013.

    Google Scholar 

  18. Johnson K, Hays C, Center H, Daley C. Building capacity and sustainable prevention innovations: a sustainability planning model. Eval Program Plann. 2004;27:135–49 Pergamon. Available from:

    Article  CAS  Google Scholar 

  19. Cooper BR, Bumbarger BK, Moore JE. Sustaining evidence-based prevention programs: correlates in a large-scale dissemination initiative. Prev Sci. 2015;16:145–57 Springer US. Available from:

    Article  PubMed  Google Scholar 

  20. Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67 Available from:

    Article  PubMed  PubMed Central  Google Scholar 

  21. Tibbits M, Bumbarger B, Kyler S, Perkins D. Sustaining evidence-based interventions under real-world conditions: results from a large-scale diffusion project. Prev Sci. 2010;11:252–62.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Walker B, Wolford B, Sasser D, Verbois C, Bell L. Launching a comprehensive SNAP-Ed social marketing campaign utilizing the cooperative extension model. J Nutr Educ Behav. 2016;48:S84 Elsevier. Available from:

    Article  Google Scholar 

  23. Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26:320–47 Sage Publications Sage CA: Thousand Oaks, CA.

    Article  Google Scholar 

  24. Bond GR, Drake RE, McHugo GJ, Peterson AE, Jones AM, Williams J. Long-term sustainability of evidence-based practices in community mental health agencies. Adm Policy Ment Heal Ment Heal Serv Res. 2014;41:228–36 Springer US.

    Article  Google Scholar 

  25. Savaya R, Spiro S, Elran-Barak R. Sustainability of social programs. Am J Eval. 2008;29:478–93 SAGE PublicationsSage CA: Los Angeles, CA.

    Article  Google Scholar 

  26. Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National institutes of health approaches to dissemination and implementation science: current and future directions. Am J Public Health. 2012;102:1274–81.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Lemacks J, Wells BA, Ilich JZ, Ralston PA. Interventions for improving nutrition and physical activity behaviors in adult African American populations: a systematic review, January 2000 through December 2011. Prev Chronic Dis. 2013;10:E99 Centers for Disease Control and Prevention.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Cassar S, Salmon J, Timperio A, Naylor P-J, van Nassau F, Contardo Ayala AM, et al. Adoption, implementation and sustainability of school-based physical activity and sedentary behaviour interventions in real-world settings: a systematic review. Int J Behav Nutr Phys Act. 2019;16:120.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Whiteside-Mansell L, Swindle TM. Together We Inspire Smart Eating: a preschool curriculum for obesity prevention in low-income families. J Nutr Educ Behav. 2017;49:789–792.e1 Elsevier.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Whiteside-Mansell L, Swindle T, Davenport K. Evaluation of “Together, We Inspire Smart Eating”(WISE) nutrition intervention for young children: assessment of fruit and vegetable consumption with parent reports and measurements of skin carotenoids as biomarkers. J Hunger Environ Nutr. 2019:1–11.

  31. Whiteside-Mansell L, Swindle TM. Evaluation of Together We Inspire Smart Eating: pre-school fruit and vegetable consumption. Health Educ Res. 2019;34:62–71 Oxford University Press. Available from:

    Article  PubMed  Google Scholar 

  32. Bronfenbrenner U. The ecology of human development. Cambridge: Harvard University Press; 1979.

    Google Scholar 

  33. Bellows LL, Davies PL, Anderson J, Kennedy C. Effectiveness of a physical activity intervention for head start preschoolers: a randomized intervention study. Am J Occup Ther. 2013;67:28–36 American Occupational Therapy Association.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Johnson SL, Bellows L, Beckstrom L, Anderson J. Evaluation of a social marketing campaign targeting preschool children. Am J Health Behav. 2007;31(1):44–55.

    Article  PubMed  Google Scholar 

  35. Bellows LL, Davies PL, Courtney JB, Gavin WJ, Johnson SL, Boles RE. Motor skill development in low-income, at-risk preschoolers: a community-based longitudinal intervention study. J Sci Med Sport. 2017;20:997–1002 Elsevier.

    Article  PubMed  Google Scholar 

  36. Johnson SL, Ryan SM, Kroehl M, Moding KJ, Boles RE, Bellows LL. A longitudinal intervention to improve young children’s liking and consumption of new foods: findings from the Colorado LEAP study. Int J Behav Nutr Phys Act. 2019;16(1):49.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Rubenstein L, Stockdale S, Sapir N, Altman L. A patient-centered primary care practice approach using evidence-based quality improvement: rationale, methods, and early assessment of implementation. J Gen. 2014.

  38. Curran GM, Mukherjee S, Allee E, Owen RR. A process for developing an implementation intervention: QUERI Series. Implement Sci. 2008;3:17 BioMed Central.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, et al. Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Implement Sci. 2012;7:30.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Chambers D, Glasgow R. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. 2013;8(1):117.

    Google Scholar 

  41. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. 2013;8:65 BioMed Central.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Powell B, Waltz T, Chinman M, Damschroder L, Smith J, Matthieu M, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17 Available from:

    Article  PubMed  PubMed Central  Google Scholar 

  44. Ament SMC, de Groot JJA, Maessen JMC, Dirksen CD, van der Weijden T, Kleijnen J. Sustainability of professionals’ adherence to clinical practice guidelines in medical care: a systematic review. BMJ Open. 2015;5:e008073 British Medical Journal Publishing Group.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Brookman-Frazee L, Stadnick N, Roesch S, Regan J, Barnett M, Bando L, et al. Measuring sustainment of multiple practices fiscally mandated in children’s mental health services. Adm Policy Ment Health. 2016;43:1009–22 Springer US. Available from:

    Article  PubMed  PubMed Central  Google Scholar 

  46. Meissner HI, Creswell JW, Klassen AC, Clark VLP, Smith KC. Best practices for mixed methods research in the health sciences. Vol. 2013. Bethesda (Maryland): National Institutes of Health; 2011. p 541–5.

  47. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Heal Ment Heal Serv Res. 2011;38:44–53 Springer US.

    Article  Google Scholar 

  48. Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:130184 Centers for Disease Control and Prevention.

    Article  PubMed  Google Scholar 

  49. Helfrich C, Li Y, Sharp N, Sales A. Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) Framework. Implement Sci. 2009;4:38.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Steckler A, Goodman RM, McLeroy KR, Davis S, Koch G. Measuring the Diffusion of Innovative Health Promotion Programs. Am J Heal Promot. 1992;6:214–24 SAGE PublicationsSage CA: Los Angeles, CA.

    Article  CAS  Google Scholar 

  51. Swindle T, Selig JP, Rutledge JM, Whiteside-Mansell L, Curran G. Fidelity monitoring in complex interventions: a case study of the WISE intervention. Arch Public Heal. 2018;76.

  52. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015;42:533–44 Springer US. Available from:

    Article  PubMed  PubMed Central  Google Scholar 

  53. Krueger R, Casey M. Focus groups: a practical guide for applied research - Richard A. Krueger, Mary Anne Casey - Google Books. Thousand Oaks: Sage Publications; 2014.

    Google Scholar 

  54. Green CA, Duan N, Gibbons RD, Hoagwood KE, Palinkas LA, Wisdom JP. Approaches to mixed methods dissemination and implementation research: methods, strengths, caveats, and opportunities. Adm Policy Ment Heal Ment Heal Serv Res. 2014.

  55. Goodell LS, Stage VC, Cooke NK. Practical qualitative research strategies: training interviewers and coders. J Nutr Educ Behav. 2016;48:578–585.e1 Elsevier.

    Article  PubMed  Google Scholar 

  56. Cooper J, Lewis R, Urquhart C. Using participant or non-participant observation to explain information behaviour. Inf Res. 2004;9:1368–613 Dept. of Information Studies, University of Sheffield.

    Google Scholar 

  57. Ward D, Hales D, Haverly K, Marks J, Benjamin S, Ball S, et al. An instrument to assess the obesogenic environment of child care centers. Am J Health Behav. 2008;32:380–6.

    Article  PubMed  Google Scholar 

  58. Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Heal Ment Heal Serv Res. 2011;38:32–43 Springer US.

    Article  Google Scholar 

  59. The University of North Carolina at Chapel Hill. Resources | Childrens Healthy Weight Research Group. 2018.

    Google Scholar 

  60. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: L. Erlbaum Associates; 1988.

    Google Scholar 

  61. Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88 Sage PublicationsSage CA: Thousand Oaks, CA.

    Article  PubMed  Google Scholar 

  62. Fereday J, Muir-Cochrane E. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods. 2006;5:80–92 SAGE Publications.

    Article  Google Scholar 

  63. Rubenstein LV, Parker LE, Meredith LS, Altschuler A, DePillis E, Hernandez J, et al. Understanding team-based quality improvement for depression in primary care. Health Serv Res. 2002;37:1009–29 Blackwell Publishing, Inc.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Curran GM, Pyne J, Fortney JC, Gifford A, Asch SM, Rimland D, et al. Development and implementation of collaborative care for depression in HIV clinics. AIDS Care. 2011;23:1626–36 Taylor & Francis Group.

    Article  PubMed  Google Scholar 

  65. Smith JL, Williams JW, Owen RR, Rubenstein LV, Chaney E. Developing a national dissemination plan for collaborative care for depression: QUERI Series. Implement Sci. 2008;3:59 BioMed Central.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Mendel P, Meredith LS, Schoenbaum M, Sherbourne CD, Wells KB. Interventions in organizational and community context: a framework for building evidence on dissemination and implementation in health services research. Adm Policy Ment Heal Ment Heal Serv Res. 2008;35:21–37 Springer US.

    Article  Google Scholar 

  67. Pyne JM, Fortney JC, Curran GM, Tripathi S, Atkinson JH, Kilbourne AM, et al. Effectiveness of collaborative care for depression in human immunodeficiency virus clinics. Arch Intern Med. 2010;171:23–31 American Medical Association.

    Article  Google Scholar 

  68. Swindle T. Application of evidence-based quality improvement to design implementation strategies for nutrition promotion in childcare. Washington, DC: Natl IDeA Symp Biomed Res Excell; 2018.

    Google Scholar 

  69. Israel BA, Eng E, Schulz AJ, Parker EA. Methods for community based participatory research. Second. San Fransico: Jossey-Bass; 2013.

    Google Scholar 

  70. Shea CM, Young TL, Powell BJ, Rohweder C, Enga ZK, Scott JE, et al. Researcher readiness for participating in community-engaged dissemination and implementation research: a conceptual framework of core competencies. Transl Behav Med. 2017;7:1–12 Springer US.

    Article  Google Scholar 

  71. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Heal Serv Res. 2017;44:177–94.

    Article  Google Scholar 

  72. W3schools. PHP 5 Form Handling. 2017.

    Google Scholar 

  73. Holt CL, Chambers DA. Opportunities and challenges in conducting community-engaged dissemination/implementation research. Transl Behav Med. 2017;7:389–92 Springer.

    Article  PubMed  PubMed Central  Google Scholar 

  74. Boothroyd RI, Flint AY, Lapiz AM, Lyons S, Jarboe KL, Aldridge WA. Active involved community partnerships: co-creating implementation infrastructure for getting to and sustaining social impact. Transl Behav Med. 2017;7:467–77 Oxford University Press Available from:

    Article  PubMed  PubMed Central  Google Scholar 

  75. Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39:55–76 Annual Reviews.

    Article  PubMed  Google Scholar 

  76. Chambers DA, Vinson CA, Norton WE. Advancing the science of implementation across the Cancer Continuum - Google Books. New York: Oxford University Press; 2018.

    Book  Google Scholar 

Download references


We would like to acknowledge our community partners who were integral in the development of this protocol and who will be critical to the success of this project.


The project was funded by NIH R21CA237984 (TS and LLB). TS is also supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number 5P20GM109096. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies

Author information

Authors and Affiliations



TS and LLB led the conception and design of this study in addition to obtaining funding; TS led the drafting of this manuscript; DZ contributed to the drafting and editing of this manuscript; SIJ and LWM contributed to the design of this study and editing of this manuscript; GMC contributed to the conception and design of the study and editing of this manuscript; JM contributed to the drafting, editing, and formatting of this manuscript; JS contributed to the conception and design of the study and editing of this manuscript; LLB contributed to the conception and design of the study and editing of this manuscript. All authors approved the manuscript before submission.

Corresponding author

Correspondence to Taren Swindle.

Ethics declarations

Ethics approval and consent to participate

This study involved the University of Arkansas for Medical Sciences (UAMS), Cornell University, and The University of Denver Anshutz Medical Campus. UAMS serves as the IRB of record.

This study will be conducted in accordance with all applicable government regulations and University of Arkansas for Medical Sciences research policies and procedures. This protocol and any amendments will be submitted and approved by the UAMS Institutional Review Board (IRB) to conduct the study.

All directors, educators, and external stakeholders participating in surveys, interviews, or classroom observations will be consented into the study per an IRB-approved protocol.

Consent for publication

Not applicable.

Competing interests

Dr. Leanne Whiteside-Mansell, Dr. Taren Swindle, and UAMS have a financial interest in the technology (WISE) discussed in this presentation/publication. These financial interests have been reviewed and approved in accordance with the UAMS conflict of interest policies.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Swindle, T., Zhang, D., Johnson, S.L. et al. A mixed-methods protocol for identifying successful sustainability strategies for nutrition and physical activity interventions in childcare. Implement Sci Commun 2, 8 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: